Re: working around status code 429

Hi Daniel,

I got a simplified test case:

calling

dirk@dirk-System-Product-Name:~/hs1$ curl --retry 30 -s -w "%{http_code} 
%{url} %{filename_effective}\n" --compressed -K ./curlrun1 --parallel -L 
| grep "429 "

result in

429 
https://commons.wikimedia.org/wiki/File:1-Acylglycerin-3-phosphocholin.svg 
/tmp/MediaWiki2LaTeXImages-3943f5a7379c6219/601.4.html
429 https://commons.wikimedia.org/wiki/File:Glycerin-3-phosphocholin.svg 
/tmp/MediaWiki2LaTeXImages-3943f5a7379c6219/603.4.html
429 https://de.wikibooks.org/wiki/File:Phosphatidylglycerin.svg 
/tmp/MediaWiki2LaTeXImages-3943f5a7379c6219/591.1.html

and more...

where curlrun1 is in 7zip file I attached

Yours Dirk

On 21.01.24 12:54, Daniel Stenberg wrote:
> On Sun, 21 Jan 2024, Dirk Hünniger via curl-users wrote:
>
>> I am downloading about 1000 URLs in parallel with a singe curl call. 
>> Some of them return status code 429 and need to be retried. 
>> Unfortunately curl does not seem to be able to retry them itself with 
>> the --retry-all-errors --retry 30 options.
>
> curl *should* retry on 429 with just --retry. I believe we even have 
> test cases verifying this...
>

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html