Re: [RFC PATCH 1/4] net: Introduce read_sock_cmsg proto_ops for control message delivery

On Thu, Feb 19, 2026 at 6:10 PM Hannes Reinecke <[email protected]> wrote:
>
> On 2/18/26 17:12, Chuck Lever wrote:
> >
> >
> > On Wed, Feb 18, 2026, at 10:52 AM, Hannes Reinecke wrote:
> >> On 2/18/26 15:33, Chuck Lever wrote:
> >>> On 2/18/26 2:29 AM, Hannes Reinecke wrote:
> >
> >>>> The way I see it, ->read_sock() has an advantage when you can work
> >>>> with skbuffs (ie network packets) directly. Then it has the benefit
> >>>> of being paced with the packets arriving as they do on the network.
> >>>>
> >>>> But for TLS this is no longer true; the TLS skbuffs are synthesized
> >>>> on top of the TCP stream, and the boundary of a TLS skbuff is completely
> >>>> decoupled from the TCP skbuff (in theory, at least).
> >>>>
> >>>> This can result in TLS skbuff stalls when waiting for the remainder
> >>>> of the TCP data stream to arrive.
> >>>
> >>> An entire TLS Record has to be present before it can be decrypted and
> >>> passed to the socket consumer. Is that what you mean?
> >>>
> >> Yes. And that makes ->read_sock() a bit pointless (from my POV); looking
> >> at the network stack the skbuffs are assembled/decoded by the TLS layer,
> >> only to be converted into skbuffs again for ->read_sock().
> >> Which seems a bit pointless.
> >
> > Is this true only of the software kTLS implementation? For
> > offload, I'd think the decrypted records are placed directly
> > in the skbs that are handed to consumers.
> >
> >
> >> But the main point with ->read_sock() is that we can only process
> >> skbuffs, ie we have to wait for the TLS layer to assemble the entire
> >> record.
> >> With recvmsg() we at least have a theoretical choice of returning
> >> a short read, allowing the code to do something clever in the meantime.
> >
> > Do you mean that read_sock is always synchronous? Perhaps
> > judicious construction of the consumer can help there; but
> > I expect that plain TCP has similar pathologies during, for
> > instance, a network partition.
> >
> >
> >> Mind you, that is all conjecture. Might well be that ->read_sock()
> >> is beneficial for TLS, too. Or, at least, doesn't do any harm.
> >> It's just that I don't trust the author of the TLS read_sock()
> >> implementation; his network stack knowledge isn't _that_ great.
> >
> > Time to get out our measuring sticks, I suppose.
> >
> >
> >> (Problem is that my performance measurements always ran into
> >> some obnoxious occasional stall, rendering the entire measurement
> >> pretty worthless. So I couldn't tell whether the recvmsg()
> >> implementation delivers a benefit or not)
> >
> > Generally speaking (ie, hand-wave mode) that kind of stall arises
> > because the consumer expects it will get another data_ready and the
> > network layer does not conform with the implementer's expectations.
> >
> Naa. Tested that one. I've seen stalls affecting _all_ queues at
> the same time, in regular intervals spaced with the power of 2.
> Current reasoning is HW-based LLDP frames handling.
>
> > So I like the read_sock_cmsg design here because with recvmsg,
> > the CMSG payload can appear in the consumer's data buffer. For
> > direct I/O, that's bad. We also really don't want to place the
> > TLS Alerts in page cache pages. With read_sock_cmsg, the CMSG
> > payload can be efficiently directed to an independent buffer
> > that is never visible to user space.
> >
> > And I prefer having a single receive path for both non-TLS and
> > TLS processing. That's so much less code to deal with.
> >
> > This approach seems to me to be more architecturally sound.
> > We just have to get over the implementation bumps IMO. I think
> > we can certainly examine ->read_sock with software kTLS to see
> > if there's room for improvement.
> >
> And that, I guess, is the real argument. We've seen with the
> recent TLS alert CVE that posting TLS messages in the recvmsg
> payload is a _bad_ idea, as this puts far too many restrictions
> on the format of the message.
>
> So you see me convinced. I'll give it a go to convert nvme-tcp.

Just a quick update that I have done this for nvme-tcp with KeyUpdate.
Just chasing down one last issue. Hopefully I can send some patches in
the next week or two.

Alistair

>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke                  Kernel Storage Architect
> [email protected]                                +49 911 74053 688
> SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
> HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
>