Re: [PATCH] virtio-net: introduce TSO limit feature

On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <[email protected]> wrote:
>
> thanks for the patch! yet something to improve.
> I note issues only when encountered 1st time but please
> do go and check all patch for each issue.
>
>
> On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > This patch introduces TSO limit feature which allows the device to
> > advertise:
> >
> > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> >   tunnel is support
> > - Maximum number of segment that can be produced by the device after
> >   segmentation of TSO or inner TSO packet of a UDP tunnel
> >
> > This is a must to implement TCP jumbogram, as networking stack needs
> > to know the limitation of the device in order to produce TSO packet as
> > large as possible.
>
> I am not sure I understand the use-case. So how is it used,
> exactly?

It works like a device advertising the TSO limit so the kernel stack
can try to send a TSO packet up to that limit to the device to perform
TCP segmentation offload. E.g without this device information
networking stack can only send a size at most 64K packet to the
device.

>
> Why is TSO singled out as compared to USO?

AFAIK no support for big USO in Linux.

>
> >
> > And it would also help for the case where host has a different TSO
> > limitation than the assumption (for example, Linux assumes 64K to be
> > the maximum number of segs and payload length).
>
> how does it assume it? 64K segments?

Yes.

>
> > Signed-off-by: Jason Wang <[email protected]>
>
>
>
>
> > ---
> >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> >
> > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > index 415c7fd..e56df75 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> >
> > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > +  length and the number of segments when performing TCP segmentation.
> > +
> >  \end{description}
> >
> >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> >  \end{description}
> >
> >  \begin{note}
> > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >          le16 rss_max_indirection_table_length;
> >          le32 supported_hash_types;
> >          le32 supported_tunnel_types;
> > +        le32 tso_max_size;
> > +        le32 tso_max_segs;
> >  };
> >  \end{lstlisting}
> >
> > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> >
> > +The following field, \field{tso_max_size} only exists if
> > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > +It specifies the maximum TCP length
>
> what is TCP length?

It's defined in the rfc793:

"""
      The TCP Length is the TCP header length plus the data length in
      octets (this is not an explicitly transmitted quantity, but is
      computed), and it does not count the 12 octets of the pseudo
      header.
"""

>
> > of a TSO packet
>
> what is a TSO packet?

Packet for device to perform TCP segmentation offload.

> is length likely to be same for TCP6 and TCP4?

Technically we can have different limitations for ipv4 and ipv6. But
considering we are designing a fresh new feature, it would be better
to use the same limitation for simplicity.

>
> > that the
> > +device can process.
>
> process in which direction? you mean device can receive?

It works only for TX (as TSO works only for TX).

>
> > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > +that the device can process.
>
> Rest of spec talks of " GSO over UDP tunnels packets" is this the same?

Not exactly the same, this is only for TSO not genreal GSO.

>
> even if it's actually unused?
>
> this, on the assumption that the length for tunnel is smaller?

It means the device should have the same limitation for plain TSO and
tunnel TSO.

>
> I think this kind of things should be explicit.
>
>
> > +
> > +The following field, \field{tso_max_segs} only exists if
> > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > +It specifies the maximum number of segments that can be produced by
> > +the device after performing segmentation on TSO packet or a UDP tunnel
> > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
>
> I don't get this field at all. the assumption is that all segments
> are the same size, right? Then it is just based on length?

It's the device side limitation, for example a device can produce 100
segments at most, even if the tso_max_size is 256K, when MTU is 1500,
the driver can't send a TSO packet whose TCP length is greater than
(1500 - 20 - 20) * 100 = 146K.

>
>
>
> > +
> >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> >
> >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> >  does not offer VIRTIO_NET_F_CTRL_VQ.
> >
> > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > +segment that fully utilizes the configured MTU can be processed by TSO
> > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > +without extension headers: at least \field{mtu} - 40). This
> > +recommendation does not account for IPv4 options or IPv6 extension
> > +headers, which reduce the effective segment size.
> > +
> > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > +set \field{tso_max_segs} to at least 64.
>
> where does this 64 come from? pls document.

A simple backward compatibility which makes sure the value can make
sure 64K TSO can be segmented with 1500 MTU.

>
> > +
> >  \drivernormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> >
> >  The driver MUST NOT write to any of the device configuration fields.
> > @@ -379,6 +409,22 @@ \subsubsection{Legacy Interface: Device configuration layout}\label{sec:Device T
> >  which provided a way for drivers to update the MAC without
> >  negotiating VIRTIO_NET_F_CTRL_MAC_ADDR.
> >
> > +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> > +transmit TSO packets with TCP length exceeding \field{tso_max_size}.
> > +
> > +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> > +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> > +packets with inner TCP length exceeding \field{tso_max_size}.
> > +
> > +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> > +transmit TSO packets with \field{gso_size} that would cause the device
> > +to generate more than \field{tso_max_segs} segments.
>
> and how does it know?

Driver knows the tso_max_segs, so when it finds the TSO packet will
exceed the limitation it would allocate a new skb for example.

>
>
> > +
> > +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> > +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> > +packets with \field{gso_size} that would cause the device to generate
> > +more than \field{tso_max_segs} segments.
> > +
> >  \subsection{Device Initialization}\label{sec:Device Types / Network Device / Device Initialization}
> >
> >  A driver would perform a typical initialization routine like so:
> > --
> > 2.42.0

Thanks

>