Re: AF_XDP sendto kick returning EPERM

On Tue, Apr 27, 2021 at 12:58 PM Magnus Karlsson
<[email protected]> wrote:
>
> On Fri, Apr 23, 2021 at 5:44 PM Srivats P <[email protected]> wrote:
> >
> > Hi,
> >
> > I'm using sendto() to kick tx in my AF_XDP program after I submit
> > descriptors to the tx ring -
> >
> > ret = sendto(xsk_socket__fd(xsk_), NULL, 0, MSG_DONTWAIT, NULL, 0);
> >
> > However, I'm receiving EPERM as the return value every time. AFAIK
> > this is not an expected return value. Since this is with i40e, I
> > checked i40e_xsk_wakeup() - but that also doesn't return EPERM. I am
> > running as root and I don't see any problems with creating the xsk,
> > configuring umem etc.
> >
> > Also, no packets seem to go out either.
> >
> > # uname -a
> > Linux Ostinato-1 5.11.15-1-default #1 SMP Fri Apr 16 16:47:34 UTC 2021
> > (64fb5bf) x86_64 x86_64 x86_64 GNU/Linux
> >
> > I don't see the problem on another machine with i40e but older kernel 5.4 series
> >
> > Any suggestions on what to look for or how to proceed?
>
> Weird. Have not seen this before. What is your command line for
> xdpsock? Is it unmodified?

This is not xdpsock, but my own AF_XDP program.

>
> Using bpftrace, we can get the call stack of xsk_sendmsg. Somewhere in
> this stack there must be an EPERM. You can run the same command on
> your system, but use ftrace to see what a sendto call hits. Then see
> where the code terminates.
>
> mkarlsso@kurt:~/src/dna-linux$ sudo bpftrace -e 'kprobe:xsk_sendmsg {
> @[kstack()] = count(); }'
> Attaching 1 probe...
> ^C
>
> @[
>     xsk_sendmsg+1
>     sock_sendmsg+94
>     __sys_sendto+238
>     __x64_sys_sendto+37
>     do_syscall_64+51
>     entry_SYSCALL_64_after_hwframe+68
> ]: 2244805

Ostinato-1:~ # bpftrace -e 'kprobe:xsk_sendmsg {
@[kstack()] = count(); }'
Attaching 1 probe...^C@[
    xsk_sendmsg+1
    sock_sendmsg+94
    __sys_sendto+238
    __x64_sys_sendto+37
    do_syscall_64+51
    entry_SYSCALL_64_after_hwframe+68
]: 1253307

Which doesn't seem to suggest any error - I've looked at the source
code for all these functions, but don't see any reference to EPERM.

Srivats