Re: [RFC PATCH 1/4] net: Introduce read_sock_cmsg proto_ops for control message delivery

On 2/18/26 15:33, Chuck Lever wrote:
> On 2/18/26 2:29 AM, Hannes Reinecke wrote:
>> On 2/17/26 23:20, Chuck Lever wrote:
>>> From: Chuck Lever <[email protected]>
>>>
>>> Kernel TCP consumers that use the read_sock interface
>>> (proto_ops.read_sock) cannot receive TLS control messages (Alerts,
>>> Handshake records) when kTLS is active. The current
>>> tls_sw_read_sock() method rejects non-data records with -EINVAL, and
>>> the sk_read_actor_t callback has no channel for delivering record-
>>> type metadata.
>>>
>>> Four kernel subsystems are affected: NFSD (sunrpc svcsock), NFS
>>> client (sunrpc xprtsock), NVMe target (nvmet-tcp), and NVMe host
>>> (nvme-tcp). Each of these either falls back to the slower
>>> sock_recvmsg() API or lacks TLS alert handling entirely.
>>>
>> And that is the contentious topic.
>> Is it really slower?
> 
> I measured 5-10% throughput increases and latency drops with NFSD,
> /without/ TLS. The primary benefit of this series is that it handles
> CMSG (TLS Alerts) much more cleanly.
> 
> 
>> The way I see it, ->read_sock() has an advantage when you can work
>> with skbuffs (ie network packets) directly. Then it has the benefit
>> of being paced with the packets arriving as they do on the network.
>>
>> But for TLS this is no longer true; the TLS skbuffs are synthesized
>> on top of the TCP stream, and the boundary of a TLS skbuff is completely
>> decoupled from the TCP skbuff (in theory, at least).
>>
>> This can result in TLS skbuff stalls when waiting for the remainder
>> of the TCP data stream to arrive.
> 
> An entire TLS Record has to be present before it can be decrypted and
> passed to the socket consumer. Is that what you mean?
> 
Yes. And that makes ->read_sock() a bit pointless (from my POV); looking
at the network stack the skbuffs are assembled/decoded by the TLS layer,
only to be converted into skbuffs again for ->read_sock().
Which seems a bit pointless.

But the main point with ->read_sock() is that we can only process
skbuffs, ie we have to wait for the TLS layer to assemble the entire
record.
With recvmsg() we at least have a theoretical choice of returning
a short read, allowing the code to do something clever in the meantime.

Mind you, that is all conjecture. Might well be that ->read_sock()
is beneficial for TLS, too. Or, at least, doesn't do any harm.
It's just that I don't trust the author of the TLS read_sock()
implementation; his network stack knowledge isn't _that_ great.

(Problem is that my performance measurements always ran into
some obnoxious occasional stall, rendering the entire measurement
pretty worthless. So I couldn't tell whether the recvmsg()
implementation delivers a benefit or not)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich