Re: You've been blocked

On 10/18/25 6:44 AM, ToddAndMargo via curl-users wrote:
>>> On Sat, Oct 18, 2025 at 1:22 PM ToddAndMargo via curl-users <curl- 
>>> [email protected] <mailto:[email protected]>> wrote:
>>>
>>>     On 10/18/25 3:06 AM, Daniel Stenberg wrote:
>>>      > On Sat, 18 Oct 2025, ToddAndMargo via curl-users wrote:
>>>      >
>>>      >> How do I get around "You've been blocked" on this web sire:
>>>      >
>>>      > Presumably you get blocked by a site if you somehow violate their
>>>     terms
>>>      > or use or their perception of good behavior.
>>>      >
>>>      > A primary way to not get blocked would be to not do that. To
>>>     understand
>>>      > the exact specifics and reasons, you would have to ask the admins
>>>     of the
>>>      > website in question.
>>>      >
>>>
>>>     I am not doing anything different than I ever do.
>>>     Do you see anything wrong with my code?
>>>     --     Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl- 
>>> users
>>>     <https://lists.haxx.se/mailman/listinfo/curl-users>
>>>     Etiquette: https://curl.se/mail/etiquette.html <https://curl.se/
>>>     mail/etiquette.html>
>>>
>>>
> 
> On 10/18/25 6:29 AM, Bastian Jesuiter via curl-users wrote:
>> You didnt even post code.
> 
>  From my original post:
> 
> curl -L https://www.softpedia.com/get/System/Back-Up-and-Recovery/ 
> Icedrive.shtml#download -o eraseme.html
> 
>> But regardless of that - if you get blocked, you'll need to resolve 
>> that with the admin of the page.
>> Maybe you can even ask (them) if you may get an API Documentation for 
>> Developers and or explain what your use case is and how you should 
>> proceed.
>> Some websites are explicitly custom fetching, others - like the 
>> icedrive website you shared recently - do not.
>>
>> You are most likely being catched by an AI Crawler blocker. Presumably 
>> because your requests are similar to the behavior of an AI or 
>> otherwise automated scraper.
>> Dont do that.
> 
> Curl does that?
> 
>> Try to find dev documentation or ask the service admin as Daniel said.
>>
>> ---
>> Bastian
> 
> The web page that I get states:
> 
>        What can I do to resolve this?
> 
>        You can email the site owner to let them know you
>        were blocked. Please include what you were doing
>        when this page came up and the Cloudflare Ray ID
>        found at the bottom of this page.
> 
>        Cloudflare Ray ID: 990732037c51c798 • Your IP:
>        • Performance & security by Cloudflare
> 
> And there is noting at the bottom of the page.
> 


I just wrote Softpedia
-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html