Re: [RFC PATCH 1/4] net: Introduce read_sock_cmsg proto_ops for control message delivery

On Thu, Feb 19, 2026 at 2:12 AM Chuck Lever <[email protected]> wrote:
>
>
>
> On Wed, Feb 18, 2026, at 10:52 AM, Hannes Reinecke wrote:
> > On 2/18/26 15:33, Chuck Lever wrote:
> >> On 2/18/26 2:29 AM, Hannes Reinecke wrote:
>
> >>> The way I see it, ->read_sock() has an advantage when you can work
> >>> with skbuffs (ie network packets) directly. Then it has the benefit
> >>> of being paced with the packets arriving as they do on the network.
> >>>
> >>> But for TLS this is no longer true; the TLS skbuffs are synthesized
> >>> on top of the TCP stream, and the boundary of a TLS skbuff is completely
> >>> decoupled from the TCP skbuff (in theory, at least).
> >>>
> >>> This can result in TLS skbuff stalls when waiting for the remainder
> >>> of the TCP data stream to arrive.
> >>
> >> An entire TLS Record has to be present before it can be decrypted and
> >> passed to the socket consumer. Is that what you mean?
> >>
> > Yes. And that makes ->read_sock() a bit pointless (from my POV); looking
> > at the network stack the skbuffs are assembled/decoded by the TLS layer,
> > only to be converted into skbuffs again for ->read_sock().
> > Which seems a bit pointless.
>
> Is this true only of the software kTLS implementation? For
> offload, I'd think the decrypted records are placed directly
> in the skbs that are handed to consumers.
>
>
> > But the main point with ->read_sock() is that we can only process
> > skbuffs, ie we have to wait for the TLS layer to assemble the entire
> > record.
> > With recvmsg() we at least have a theoretical choice of returning
> > a short read, allowing the code to do something clever in the meantime.
>
> Do you mean that read_sock is always synchronous? Perhaps
> judicious construction of the consumer can help there; but
> I expect that plain TCP has similar pathologies during, for
> instance, a network partition.
>
>
> > Mind you, that is all conjecture. Might well be that ->read_sock()
> > is beneficial for TLS, too. Or, at least, doesn't do any harm.
> > It's just that I don't trust the author of the TLS read_sock()
> > implementation; his network stack knowledge isn't _that_ great.
>
> Time to get out our measuring sticks, I suppose.
>
>
> > (Problem is that my performance measurements always ran into
> > some obnoxious occasional stall, rendering the entire measurement
> > pretty worthless. So I couldn't tell whether the recvmsg()
> > implementation delivers a benefit or not)
>
> Generally speaking (ie, hand-wave mode) that kind of stall arises
> because the consumer expects it will get another data_ready and the
> network layer does not conform with the implementer's expectations.
>
> So I like the read_sock_cmsg design here because with recvmsg,
> the CMSG payload can appear in the consumer's data buffer. For
> direct I/O, that's bad. We also really don't want to place the
> TLS Alerts in page cache pages. With read_sock_cmsg, the CMSG
> payload can be efficiently directed to an independent buffer
> that is never visible to user space.
>
> And I prefer having a single receive path for both non-TLS and
> TLS processing. That's so much less code to deal with.
>
> This approach seems to me to be more architecturally sound.
> We just have to get over the implementation bumps IMO. I think
> we can certainly examine ->read_sock with software kTLS to see
> if there's room for improvement.

I completely agree!

The current read_sock approach is very brittle, as soon as a control
message appears the whole thing falls over. In the NVMe world we are
seeing devices issuing Key-Updates, which is breaking things.

So far the approach has been to convert to use sock_recvmsg() instead
of ->read_sock(), but at least for the nvme-host conversion (not
upstream) that has exposed a range of subtle and hard to narrow down
bugs. All this churn just to handle a few control messages.

This approach (in this series) seems like a great step to handle
control messages without having to redesign existing otherwise working
recv flows to use sock_recvmsg(), which doesn't seem to have any other
benefit, besides handling control messages.

Alistair

>
> --
> Chuck Lever
>