Re: AF_XDP busy poll receives packets in batches of 8 on i40e
Federico Parola <[email protected]>
| Newsgroups | org.kernel.vger.xdp-newbies |
|---|---|
| Message-ID | <[email protected]> |
On 17/03/22 15:59, Jay Vosburgh wrote: > Federico Parola <[email protected]> wrote: > >> Hello everybody, >> I'm experiencing a strange problem when running an AF_XDP application with >> busy poll enabled on a Intel XL710 NIC (i40e driver). >> The problem can be replicated running the xdpsock kernel sample in rx or >> l2fwd mode. >> The first packet I send to the machine is correctly received by the >> application. After this, packets are only received in batches of 8. >> If I send 7 packets the application sees nothing, while the 8th one >> triggers the reception of all 8 packets. >> Disabling the busy poll mode everything works fine and packets are >> immediately received as they are sent. >> >> I tried changing kernel (5.12, 5.14 and 5.16) but all present the same >> problem. >> I also tried using another NIC, an Intel X540 with ixgbe driver and the >> problem isn't there, so I guess is NIC/driver related. >> >> I tried monitoring ethtool statistics. When sending packets between 1 and >> 7 these counters are increased: >> stat: 64 ( 64) <= port.rx_bytes /sec >> stat: 1 ( 1) <= port.rx_size_64 /sec >> stat: 1 ( 1) <= port.rx_unicast /sec >> stat: 1 ( 1) <= rx_unicast /sec >> >> While the 8th one triggers this updates: >> stat: 64 ( 64) <= port.rx_bytes /sec >> stat: 1 ( 1) <= port.rx_size_64 /sec >> stat: 1 ( 1) <= port.rx_unicast /sec >> stat: 477 ( 477) <= rx-0.bytes /sec >> stat: 8 ( 8) <= rx-0.packets /sec >> stat: 477 ( 477) <= rx_bytes /sec >> stat: 8 ( 8) <= rx_packets /sec >> stat: 1 ( 1) <= rx_unicast /sec >> >> As far as I understand the first set of counters are hardware counters, so >> it makes me think that packets are kept in the NIC and not even sent to >> memory. >> >> Does anyone have any suggestion on what could be causing this problem? >> Does enabling busy poll set some flag on the NIC? > > We observed similar "batching" behavior on i40e devices late > last year in ordinary use (not XDP, but using SR-IOV VFs). We > instrumented the drivers at the send and receive sides, and determined > that it appeared to be a behavior of the receiving device itself, i.e., > packets 1 - 7 would be held indefinitely (as I recall, no interrupt or > update of the RX ring pointers) until packet 8 arrives, at which point > all 8 were delivered simultaneously. > > The issue was evidently in the firmware, and was resolved after > a firmware upgrade. Hi Jay, I just updated the firmware to the latest version (v8.50 from v8.30) but unfortunately the problem is still there. However I'm experiencing the problem only when using AF_XDP in busy poll mode, all other modes (standard AF_XDP and normal packet reception) work just fine. Maybe the two problems are correlated in some way. > > -J > >> >> Best regards, >> Federico Parola > > --- > -Jay Vosburgh, [email protected]