Re: AF_XDP busy poll receives packets in batches of 8 on i40e

On 3/18/2022 11:33 AM, Jay Vosburgh wrote:
>> I tested it on 3 different XL710 NICs, all updated to the latest 8.50
>> firmware, and they all present the same 8 packets "batching" behavior.
>> How do I check the PCI Cache Line Size (I didn't find such entry in lspci
>> -vvv)? Normal cache line size is 64B on my system, but I guess they are
>> two different things.
> 
> 	It's in the lspci -vvv, near the top, e.g.,
> 
> 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
>          Subsystem: Intel Corporation Ethernet Converged Network Adapter XL710-Q2
>          Physical Slot: 1
>          Control: [...bunch of stuff...]
>          Status: [...bunch of stuff...]
>          Latency: 0, Cache Line Size: 64 bytes
> 
> 	Checking the PCIe spec, though, I'm not sure why this would
> matter, as PCIe 4.0 7.5.1.3 Cache Line Size Register says it "has no
> effect on any PCI Express device behavior."

Thanks Jay, that might just be the sound of my brain short circuiting, 
as I thought that held some relevance for some reason. Our receive 
descriptors are 16 bytes (with options for 32 bytes sometimes), and the 
transmit are 16. 8*16=128, so I was suspecting something related to 
cache line size in the PCIe is triggering a writeback when we get to 
that boundary, it's probably some other trigger I don't remember.

I know the XL710 has a documented erratum where while interrupts are 
disabled (like while polling) that hardware won't write back until it 
hits a writeback criteria, or unless the interrupt is re-enabled. We 
have a bunch of workarounds in the driver trying to address this, I was 
thinking maybe one of those workarounds isn't hitting in the XDP code.

Oh, and WB_ON_ITR didn't appear until the X722 generation, so it's not 
there on XL710. Hope this helps!

Jesse