Re: Network.framework and libcurl

Demi Marie Obenour via curl-library <[email protected]> Wed, 27 Aug 2025 17:07:00 -0400
Newsgroups	gmane.comp.web.curl.library
Message-ID	<[email protected]>
On 8/27/25 04:16, Daniel Stenberg wrote:
> On Tue, 26 Aug 2025, Demi Marie Obenour wrote:
> 
>> 1. A userspace networking stack, which implements TCP, UDP, and everything
>>   below.  This replaces the kernelspace networking stack used by BSD sockets,
>>   and has both improved performance and more features.  For instance, it
>>   supports MPTCP and avoids an extra copy from kernel buffers into user
>>   buffers.
> 
> In theory this should not have to affect curl in any particular way, but it 
> depends on the specifics. From demos I've seen, Network.framework likes using 
> multiple threads to provide this functionality (I presume since for example 
> timers can get tricky to manage otherwise when done entirely in user space). 
> The impression I've got is that this gets complicated, especially since they 
> seem to like callbacks - from different threads.

I'm not surprised.  Unless some of those callbacks need synchronous
processing, this could be handled by using AF_UNIX sockets to notify
the main thread.  That said, I expect this behavior is for performance
reasons, as that is the documented reason for MsQuic to use callbacks
on a thread pool MsQuic owns.

>> 2. APIs that take a domain name as input and (asynchronously) return a
>>   connection to it.  This is what is necessary to make VPN On Demand work.
>>   There is also a deprecated API that can be used to do the same with BSD
>>   sockets.
> 
> We can't use name based API that does all this magic under the hood. Magic 
> that curl needs to own and control itself.

Unfortunately, any program that doesn’t do this will break if VPN On Demand
is in use.  VPN On Demand startup is based on target domain name, not just
on IP address.  This means that for a program to work correctly if VPN On
Demand is in use, using such an API is required.

My recommendation would be to consider this a necessary workaround for a
platform quirk, and accept that this will result in differing behavior.

>> 3. A TLS implementation based on BoringSSL.  This is usually used with (1)
>>   and (2), but it can also be used with any other transport (including
>>   non-TCP transports) via a custom framer.
> 
> That's basically the only part we would like to access and use but I have not 
> seen anyone work on that or show how that would work.

I'll leave that to the Apple experts.

>> 4. Implementations of QUIC, HTTP/1.x, HTTP/2, and HTTP/3.  These use (1),
>>   (2), and (3) internally.
> 
> Not terribly interesting for us.

Fair.  I could see the QUIC, HTTP/2, and HTTP/3 code being useful as an
alternative to ngtcp2, nghttp2, and nghttp3, but that would be the least
important part.

> I would like to emphasize that we would not be against anyone implementing 
> network framework support - if and only if it can be done in a clean enough 
> way. It would be welcomed. I have not seen or heard from anyone working on 
> this and I have not seen anyone interested in paying for it to happen either. 
> I'm not holding my breath for it.
Definitely do not consider yourself under any obligation to implement this
yourself.  I do question if requiring identical behavior on all platforms
is worth being broken (in a user-visible way) on Apple devices that have
VPN On Demand configured.

As an aside, I don't think watchOS support is a worthwhile motivation.
watchOS requires using the high-level URLSession APIs and those built on
top of them for all but a few purposes, where "high-level" is defined as
"the entire transfer happens in the background and the app is woken up
once the transfer is complete".  In other words, one can only provide
the data to be uploaded as a file or memory buffer, and the entire
response is always written to a file or a memory buffer before one's
app is notified.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html
OpenPGP_0xB288B55FFF9C22C1.asc (application/pgp-keys, 7 KB)
OpenPGP_signature.asc (application/pgp-signature, 833 B)