Re: Umem Questions

On Thu, May 20, 2021 at 8:14 AM Magnus Karlsson
<[email protected]> wrote:
>
> On Wed, May 19, 2021 at 11:09 PM Dan Siemon <[email protected]> wrote:
> >
> >
> > > > 2) It looks like there is a limit of 2GB on the maximum Umem size?
> > > > I've
> > > > tried with and without huge pages. Is this fundamental? How hard
> > > > would
> > > > it be to increase this?
> > >
> > > This was news to me. Do you know where in the xdp_umem_reg code it
> > > complains about this? I guess it is xsk_umem__create() that fails,
> > > or?
> > > The only limit I see from a basic inspection of the code is that the
> > > number of packet buffers cannot be larger than a u32 (4G). But you
> > > are
> > > not close to that limit.
> >
> > Yes, the failure is in xsk_umem__create(). I don't know where
> > specifically but there are a couple spots in kernel side of that which
> > return ENOMEM which is the return value.

I think I have found it.

static int xdp_umem_pin_pages(struct xdp_umem *umem, unsigned long address)
{
  unsigned int gup_flags = FOLL_WRITE;
long npgs;
        int err;

        umem->pgs = kcalloc(umem->npgs, sizeof(*umem->pgs),
                            GFP_KERNEL | __GFP_NOWARN);
        if (!umem->pgs)
                return -ENOMEM;

This structure becomes too large to allocate with kcalloc(). It needs
to be turned into a kvcalloc() so that it can use vmalloc instead for
requests that are this large. Will spin a patch.

Thanks: Magnus

> Can you issue a "ulimit -a" on your system and share the result? Just
> to verify that there is no per process limit that kicks in.