Re: Re: A more general question about curl

Hello everyone,


wget2 community is hearing our conversation and is reaching out to us for collaboration:


story - short:
> On 10. Sep 2022, at 20:53, Tim Rühsen <[email protected]> wrote:
>
> just found this issue: https://gitlab.com/gnuwget/wget2/-/issues/550
>
> Feel free to add the details needed for GNUnet.


story - medium:
> On 10. Sep 2022, at 20:47, Tim Rühsen <[email protected]> wrote:
>
> TL;DR the wget2 project is willing to help and a multi API may be easy
> to implement. But someone has to write down the exact needs.


Full story, its rest - https://lists.gnu.org/archive/html/wget-dev/2022-09/msg00008.html :
> On 10. Sep 2022, at 20:53, Tim Rühsen <[email protected]> wrote:
> I try to explain what libwget does today and that it seems very straight forward to implement an API - and by the way - everybody is invited to do that :-)
>
> Libwget has several layers of abstraction of accessing the network stack(s).
>
> So you have indeed the synchronously, super simplified
>   response = wget_http_get(...) like in example/http_get.c
>
> Then there is async HTTP API layer with more control over the details, see example/http_get2.c.
>
>   err = wget_http_open(&conn, url); // comes back immediately
>   err = wget_http_send_request(conn, req); // comes back immediately
>
> resp = wget_http_get_response(conn); // waits until error/timeout or response
>
>   wget_http_close(&conn);
>
> While you could send several requests over a single connection, HTTP/1.1 has issues with it. But it works fine with HTTP/2. In this case wget_http_get_response(conn) can be called in a loop, returning the finished response in the order they came in. You can also have as many open connections as you like - but what is missing, if I understood correctly, is an API that fetches the responses from more than one connection at once, like
>
>   resp = wget_http_get_response(conn1, conn1, ..., NULL);
>
> Now there is also a TCP+SSL API (used by the above mentioned high level functions). This API is works asynchronously. It is like
>
>   tcp = wget_tcp_init()
>   ... set all kind of configurations to the 'tcp' handle ...
>   err = int wget_tcp_connect(tcp, host, port) // returns immediately
>   wget_tcp_write() // returns after timeout or immediately if no timeout
>   wget_tcp_read() // returns after timeout or immediately if no timeout
>   wget_tcp_close()
>   wget_tcp_deinit()
>
> Internally, this API uses select/poll, but just uses a single 'tcp' handle.
>
> Now, what a "multi"  API basically would look like is e.g.
>
> a function wget_tcp_select(array of tcp handles, timeout) which can be called in a loop and which returns an array of "ready" tcp handles (ready for write or read, configurable per tcp handle).
>
>
> For me it looks like this is straight forward to implement (depends on the details / requirements).
>
>
> Internally, wget_ready_2_transfer(int fd, int timeout, int mode) in
> io.c just needs a companion function that takes a list/array of fds.
>
> If there really is interest from the GNUnet community, why not open an issue at https://gitlab.com/gnuwget/wget2/issues to discuss the details and the needs. Once we agree upon the details, the implementation can be done by anyone - whoever likes to pick it up.


Greetings,
Bastian Schmidt