RE: Memory policy question for NUMA arch....

Hello,

PS - Please 'CC' me on the emails.I have not subscribed to the list.

> Hi Andy,
> 
> --- On Wed, 4/7/10, Andi Kleen <[email protected]>
> wrote:
> > On Tue, Apr 06, 2010 at 01:46:44PM -0700, Rick Sherm
> wrote:
> > > On a NUMA host, if a driver calls
> __get_free_pages()
> > then
> > > it will eventually invoke
> > ->alloc_pages_current(..). The comment
> > > above/within alloc_pages_current() says
> > 'current->mempolicy' will be
> > > used.So what memory policy will kick-in if the
> driver
> > is trying to
> > > allocate some memory blocks during driver load
> > time(say from probe_one)? System-wide default
> > policy,correct?
> >
> > Actually the policy of the modprobe or the kernel boot
> up
> > if built in
> > (which is interleaving)
> >

I may be wrong but I think there's a difference. system-wide run-time default policy is M_PREFERRED | M_LOCAL and not Interleaving.

So, if current->mempolicy is set then default_policy will not be used. 
And now if you don't want the default_policy mode then what?
I'm stuck in this confused state too. So we have two cases to take care off - 

Case1) current->mempolicy is initialized and so we can just set it to whatever we like and then reset it once we are done with __get_free_pages(..) etc.

Case2) current->mempolicy is not initialized. Then default_policy is used. Now if we have to muck with the default_policy then we will need to lock it down. Otherwise some other consumer will get affected by it.

But both the above solutions are twisted.Why not just create a different wrapper? This way we can leave both current & default_policy alone.

#ifdef CONFIG_NUMA
__get_free_policy_pages(policy,mask,order)??
endif

For now I may end up hacking my kernel and implementing the above mentioned quick and dirty solution. But if there's a cleaner approach then please let me know.

PS - We should create some wrapper's that will automatically figure out the MSIX-affinity(if present/set) and then default the allocation to that node? Also, is there a way to configure irqbalance and ask it to leave these guys alone? Like a config file that says - leave these irqs/pci-devices alone.For now I've shut down irqbalance.

> 
> > -Andi
> >
> 
> Thanks
> Rick
> 

thanks
Chetan Loke