Re: numa_alloc_onnode does not allocate on node passed as argument

On Tue, Oct 21, 2014 at 11:47 PM, Andres N=C3=B6tzli <noetzli@stanford.=
edu> wrote:
> Hi everyone,
>
> I am experiencing a weird problem. When using numa_alloc_onnode repea=
tedly to allocate memory, it does not allocate memory on the node passe=
d as an argument.
>
> Sample code:
> #include <numa.h>
> #include <numaif.h>
> #include <iostream>
> using namespace std;
>
> void find_memory_node_for_addr(void* ptr) {
>    int numa_node =3D -1;
>    if(get_mempolicy(&numa_node, NULL, 0, ptr, MPOL_F_NODE | MPOL_F_AD=
DR) < 0)
>       cout << "WARNING: get_mempolicy failed" << endl;
>    cout << numa_node << endl;
> }
>
> int main() {
>    int64_t* x;
>    int64_t n =3D 5000;
>    //numa_set_preferred(1);
>
>    numa_run_on_node(2);
>    for(int i =3D 0; i < 20; i++) {
>       size_t s =3D n * sizeof(int64_t);
>       x =3D (int64_t*)numa_alloc_onnode(s, 1);
>       for(int j =3D 0; j < n; j++)
>          x[j] =3D j + i;
>       find_memory_node_for_addr(x);
>    }
>
>    return 0;
> }
>
> Output:
> 1
> 1
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
> 1
> 2
>
> When uncommenting the line "numa_set_preferred(1);=E2=80=9D, the outp=
ut is all 1s as expected. Am I doing something wrong? Have you seen sim=
ilar issues?
>
> I am running Ubuntu 12.04.5 LTS:
> $ cat /proc/version
> Linux version 3.2.0-29-generic (buildd@allspice) (gcc version 4.6.3 (=
Ubuntu/Linaro 4.6.3-1ubuntu5) ) #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC =
2012
>
> I am using libnuma 2.0.10 but I=E2=80=99ve had the same problem with =
2.0.8~rc3-1.
>
> Thank you very much,
> Andres
> --
> To unsubscribe from this list: send the line "unsubscribe linux-numa"=
 in
> the body of a message to [email protected]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi Andres

Can you try to use strict policy by calling numa_set_strict?

If you comment out setting the preferred node, the default policy is
in action (I assume you did no change it, not for the process, not
system wide) which is preferred also.
But here you set preferred to a specific node and manual says, the
default for process is to allocate on the node it runs.
So I wonder what is the cpu affinity for this process looks like...
Also maybe just to confirm you can check the policy from within your
running code?

Can you also post the machine NUMA config?

--=20
Elena