Re: AF_XDP sendto kick returning EPERM

On Thu, Apr 29, 2021 at 5:47 PM Srivats P <[email protected]> wrote:
>
> On Tue, Apr 27, 2021 at 12:58 PM Magnus Karlsson
> <[email protected]> wrote:
> >
> > On Fri, Apr 23, 2021 at 5:44 PM Srivats P <[email protected]> wrote:
> > >
> > > Hi,
> > >
> > > I'm using sendto() to kick tx in my AF_XDP program after I submit
> > > descriptors to the tx ring -
> > >
> > > ret = sendto(xsk_socket__fd(xsk_), NULL, 0, MSG_DONTWAIT, NULL, 0);
> > >
> > > However, I'm receiving EPERM as the return value every time. AFAIK
> > > this is not an expected return value. Since this is with i40e, I
> > > checked i40e_xsk_wakeup() - but that also doesn't return EPERM. I am
> > > running as root and I don't see any problems with creating the xsk,
> > > configuring umem etc.
> > >
> > > Also, no packets seem to go out either.
> > >
> > > # uname -a
> > > Linux Ostinato-1 5.11.15-1-default #1 SMP Fri Apr 16 16:47:34 UTC 2021
> > > (64fb5bf) x86_64 x86_64 x86_64 GNU/Linux
> > >
> > > I don't see the problem on another machine with i40e but older kernel 5.4 series
> > >
> > > Any suggestions on what to look for or how to proceed?
> >
> > Weird. Have not seen this before. What is your command line for
> > xdpsock? Is it unmodified?
>
> This is not xdpsock, but my own AF_XDP program.
>
> >
> > Using bpftrace, we can get the call stack of xsk_sendmsg. Somewhere in
> > this stack there must be an EPERM. You can run the same command on
> > your system, but use ftrace to see what a sendto call hits. Then see
> > where the code terminates.
> >
> > mkarlsso@kurt:~/src/dna-linux$ sudo bpftrace -e 'kprobe:xsk_sendmsg {
> > @[kstack()] = count(); }'
> > Attaching 1 probe...
> > ^C
> >
> > @[
> >     xsk_sendmsg+1
> >     sock_sendmsg+94
> >     __sys_sendto+238
> >     __x64_sys_sendto+37
> >     do_syscall_64+51
> >     entry_SYSCALL_64_after_hwframe+68
> > ]: 2244805
>
> Ostinato-1:~ # bpftrace -e 'kprobe:xsk_sendmsg {
> @[kstack()] = count(); }'
> Attaching 1 probe...^C@[
>     xsk_sendmsg+1
>     sock_sendmsg+94
>     __sys_sendto+238
>     __x64_sys_sendto+37
>     do_syscall_64+51
>     entry_SYSCALL_64_after_hwframe+68
> ]: 1253307
>
> Which doesn't seem to suggest any error - I've looked at the source
> code for all these functions, but don't see any reference to EPERM.

It must be in there somewhere :-). Could you plesae use ftrace
(through perf for example) and trace all functions that a sendto hits
in your case? Then we might see what it hits.

Are you running in SKB mode or in zero-copy mode? Guess it is
zero-copy from your mail, but just want to verify. Does Rx work as
expected?

Could you share your AF_XDP program?

/Magnus

> Srivats