Re: [RFC PATCH 1/4] net: Introduce read_sock_cmsg proto_ops for control message delivery

On 2/19/26 05:06, Alistair Francis wrote:
> On Thu, Feb 19, 2026 at 2:12 AM Chuck Lever <[email protected]> wrote:
>>
>>
>>
>> On Wed, Feb 18, 2026, at 10:52 AM, Hannes Reinecke wrote:
>>> On 2/18/26 15:33, Chuck Lever wrote:
>>>> On 2/18/26 2:29 AM, Hannes Reinecke wrote:
>>
>>>>> The way I see it, ->read_sock() has an advantage when you can work
>>>>> with skbuffs (ie network packets) directly. Then it has the benefit
>>>>> of being paced with the packets arriving as they do on the network.
>>>>>
>>>>> But for TLS this is no longer true; the TLS skbuffs are synthesized
>>>>> on top of the TCP stream, and the boundary of a TLS skbuff is completely
>>>>> decoupled from the TCP skbuff (in theory, at least).
>>>>>
>>>>> This can result in TLS skbuff stalls when waiting for the remainder
>>>>> of the TCP data stream to arrive.
>>>>
>>>> An entire TLS Record has to be present before it can be decrypted and
>>>> passed to the socket consumer. Is that what you mean?
>>>>
>>> Yes. And that makes ->read_sock() a bit pointless (from my POV); looking
>>> at the network stack the skbuffs are assembled/decoded by the TLS layer,
>>> only to be converted into skbuffs again for ->read_sock().
>>> Which seems a bit pointless.
>>
>> Is this true only of the software kTLS implementation? For
>> offload, I'd think the decrypted records are placed directly
>> in the skbs that are handed to consumers.
>>
>>
>>> But the main point with ->read_sock() is that we can only process
>>> skbuffs, ie we have to wait for the TLS layer to assemble the entire
>>> record.
>>> With recvmsg() we at least have a theoretical choice of returning
>>> a short read, allowing the code to do something clever in the meantime.
>>
>> Do you mean that read_sock is always synchronous? Perhaps
>> judicious construction of the consumer can help there; but
>> I expect that plain TCP has similar pathologies during, for
>> instance, a network partition.
>>
>>
>>> Mind you, that is all conjecture. Might well be that ->read_sock()
>>> is beneficial for TLS, too. Or, at least, doesn't do any harm.
>>> It's just that I don't trust the author of the TLS read_sock()
>>> implementation; his network stack knowledge isn't _that_ great.
>>
>> Time to get out our measuring sticks, I suppose.
>>
>>
>>> (Problem is that my performance measurements always ran into
>>> some obnoxious occasional stall, rendering the entire measurement
>>> pretty worthless. So I couldn't tell whether the recvmsg()
>>> implementation delivers a benefit or not)
>>
>> Generally speaking (ie, hand-wave mode) that kind of stall arises
>> because the consumer expects it will get another data_ready and the
>> network layer does not conform with the implementer's expectations.
>>
>> So I like the read_sock_cmsg design here because with recvmsg,
>> the CMSG payload can appear in the consumer's data buffer. For
>> direct I/O, that's bad. We also really don't want to place the
>> TLS Alerts in page cache pages. With read_sock_cmsg, the CMSG
>> payload can be efficiently directed to an independent buffer
>> that is never visible to user space.
>>
>> And I prefer having a single receive path for both non-TLS and
>> TLS processing. That's so much less code to deal with.
>>
>> This approach seems to me to be more architecturally sound.
>> We just have to get over the implementation bumps IMO. I think
>> we can certainly examine ->read_sock with software kTLS to see
>> if there's room for improvement.
> 
> I completely agree!
> 
> The current read_sock approach is very brittle, as soon as a control
> message appears the whole thing falls over. In the NVMe world we are
> seeing devices issuing Key-Updates, which is breaking things.
> 
Indeed. That's why I thought recvmsg() would be a better fit.

> So far the approach has been to convert to use sock_recvmsg() instead
> of ->read_sock(), but at least for the nvme-host conversion (not
> upstream) that has exposed a range of subtle and hard to narrow down
> bugs. All this churn just to handle a few control messages.
> 
Ah; didn't know this. Was wondering why you had kept quiet, but that
explains it.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich