Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0

Demi Marie Obenour via curl-library <[email protected]> Tue, 6 Jan 2026 14:27:43 -0500
Newsgroups	gmane.comp.web.curl.library
Message-ID	<[email protected]>
On 1/6/26 06:59, Stefan Eissing via curl-library wrote:
> I hear that you have a very specific application scenario where the curl 8.6.0 "almost" (your words) helped you, but the versions after that are no longer that "almost". Especially for cases where the overall network transfer is less than a second (or just a few).
> 
> The curl rate limit works in "bytes per second". It does not say when *during* the second the bytes are received. The bytes may be received in the first few nanoseconds. If the transfer has not ended by that, libcurl will wait the remainder of the second before doing the next receive, obeying the limit.
> 
> This is in accordance to what you want, except at the end of a transfer. Here, our definitions of what "rate limit" means differ. (And I would really prefer you not calling our view on things as "not working" or a "regression". It is working fine as we define rate limits. It is just not what you want. Respect each other's views.)
> 
> An example: a transfer of 100 bytes is configured with a rate limit of 1000 bytes/s.
> - In your definition, the transfer should complete exactly after 100ms.
> - In our definition, the transfer completes when the 100 bytes have been received, never delivering more than 1000 bytes in a second. And there was no second that it delivered more.
> 
> For longer transfers, this all does not matter much as the last second has decreasing impact.
> 
> Now, besides your libcurl application, there are other applications that use rate limits but want to have transfers reported as complete when all data has arrived. I would hate my steam downloads to pause at the end, for example. You have a specific application case and your proposed solution is incompatible with other cases.
> 
> I do not know your application and cannot judge how to best solve that. It seems to be in need to some "holding queue" where finished libcurl transfers are held to idle the last second remainder before the application acts on the completion. This is what you are asking libcurl to implement, so your application does not have to.
> 
> tl;dr
> 
> The current rate limit implementation is what should be natural for many libcurl applications. For longer transfers, it works for you was well. For short ones, unfortunately, it does not give you the precision that you want in your application. Acknowledged. Adding a separate rate limit implementation in libcurl, so your application does not have to, is not a convincing argument.

I think the reason that this is a severe problem for Dmitry Karpov
is that they rely on third-party libraries that cannot be changed.
They can pass a rate limit to libcurl via those libraries, but they
can't add a holding queue.

My personal view is that this breaks backwards compatibility in
a corner case.  Whether that is acceptable is up to the libcurl
maintainers, and I'm fine with either decision.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html
OpenPGP_0xB288B55FFF9C22C1.asc (application/pgp-keys, 7 KB)
OpenPGP_signature.asc (application/pgp-signature, 833 B)