Questions about Offloads and XDP-Hints regarding a Cloud-Provider Use-Case

Marcus Wichelmann <[email protected]> Wed, 28 Sep 2022 15:54:03 +0200
Newsgroups org.kernel.vger.xdp-newbies
Message-ID <[email protected]>
Hello,

I'm working for a cloud hosting provider and we're working on a new 
XDP-based networking stack for our VM-Hosts that uses XDP to accelerate 
the connectivity of our qemu/KVM VMs to the outside.

For this, we use XDP_REDIRECT to forward packets between the physical 
host NIC and the VM tap devices. The main issue we have now is, that our 
VM guests have some virtio NIC offloads enabled: rx/tx checksumming, 
TSO/GSO, GRO and Scatter-Gather.
Currently, these offloads (especially TSO/GSO) are incompatible with 
XDP_REDIRECT and result in packets being dropped. Because disabling 
these offloads in all our customer VMs is not a good option, we're 
searching for ways to support these offloads with XDP.

The XDP multi-buffer support needed for TSO/GSO seems to be mostly there 
already, but, to our understanding, the last missing part for full 
TSO/GSO support is a way to tell the physical NIC to perform the TSO/GSO 
offload.

I've seen  the latest LPC 2022 talk from Jesper Dangaard Brouer 
regarding the planned XDP-Hints feature. But this was mainly about 
Checksum and VLAN offloads. Is supporting TSO/GSO also one of the goals 
you have in mind with these XDP-Hints proposals?

Will the multi-buffer and hints patches be all what's needed to make 
XDP_REDIRECT between a VM (without disabled offloads) and the host NIC 
possible, or are there more things missing in XDP that will become an 
issue in that use-case?

The "XDP Cloud-Provider" project page describes a very similar use-case 
to what we plan to do. What's the goal of this project?

We're very interested in the work on XDP-Hints and in the performance 
benefits that the offloading support could bring to XDP and I would be 
thankful if you could help us with some of our questions.

Best regards,
Marcus Wichelmann
Hetzner Cloud GmbH