Re: NUMA page allocation from next Node

On Mon, 2010-11-01 at 19:48 +0530, Tharindu Rukshan Bamunuarachchi
wrote:
> Tim,
> 
> I found that default value for "zone_reclaim_mode" is zero in HP
> machine. But It is one in IBM.
> Why does it set 1 or 0 in different hardware

Because the SLIT on the IBM platform has distances > 20.  Looks like IBM
is populating the SLIT on those platforms with "real" values.  The HP
bios is not supplying a slit, letting the remote distances default to
20.  That is the threshold for setting zone_reclaim_mode.  A patch was
submitted recently to bump the threshold to ~30.  Now that vendors are
starting to populate the SLIT with values > 20, we've begun to see the
behavior that you experienced. 

Regards,
Lee
 
> 
> On Sat, Oct 30, 2010 at 1:22 AM, Tim Pepper <[email protected]> wrote:
> >
> > It would be interesting to see the output of "numactl --hardware" for each
> > of these scenarios.
> >
> 
> 1. SLES11 + IBM HW
> After consuming all memory in node1, it shows following ...
> 
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3 4 5
> node 0 size: 24564 MB
> node 0 free: 23025 MB
> node 1 cpus: 6 7 8 9 10 11
> node 1 size: 24576 MB
> node 1 free: 16 MB
> node distances:
> node   0   1
>   0:  10  21
>   1:  21  10
> 
> 
> 2. SLES 11 + HP
> 
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 19929 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 3043 MB
> node distances:
> node   0   1
>   0:  10  20
>   1:  20  10
> 
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 19912 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 335 MB
> node distances:
> node   0   1
>   0:  10  20
>   1:  20  10
> 
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 17066 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 16 MB
> node distances:
> node   0   1
>   0:  10  20
>   1:  20  10
> 
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 10468 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 16 MB
> node distances:
> node   0   1
>   0:  10  20
>   1:  20  10
> 
> 
> 
> > --
> > Tim Pepper  <[email protected]>
> > IBM Linux Technology Center
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-numa" in
> the body of a message to [email protected]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html