Re: I Nerd help downloading a page
ToddAndMargo via curl-users <[email protected]> Sat, 11 Oct 2025 17:40:47 -0700
| Newsgroups | gmane.comp.web.curl.general |
|---|---|
| Message-ID | <[email protected]> |
On 10/11/25 5:39 PM, ToddAndMargo via curl-users wrote: > On 10/11/25 5:03 PM, ToddAndMargo via curl-users wrote: >>>> On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <curl- >>>> [email protected] <mailto:[email protected]>> wrote: >>>> >>>> Hi All, >>>> >>>> curl-8.9.1-4.fc41.x86_64 >>>> >>>> How come Firefox can see this page, but curl can not? >>>> >>>> >>>> First attempt: >>>> >>>> $ curl -L https://icedrive.net/apps/desktop-laptop <https:// >>>> icedrive.net/apps/desktop-laptop> -o eraseme.html >>>> % Total % Received % Xferd Average Speed Time Time >>>> Time >>>> Current >>>> Dload Upload Total Spent >>>> Left >>>> Speed >>>> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:-- >>>> --:--:-- >>>> 51058 >>>> >>>> >>>> Second attempt (that is Brave Browser user agent. >>>> Firefox's did not work either): >>>> >>>> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64) >>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 >>>> <http://141.0.0.0> Safari/537.36" >>>> https://icedrive.net/apps/desktop-laptop/ <https://icedrive.net/ >>>> apps/desktop-laptop/> -o eraseme.html >>>> % Total % Received % Xferd Average Speed Time Time >>>> Time >>>> Current >>>> Dload Upload Total Spent >>>> Left >>>> Speed >>>> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:-- >>>> --:--:-- >>>> 48422 >>>> >>>> >>>> Yours in confusion, >>>> -T >> >> On 10/11/25 4:47 PM, Bastian Jesuiter via curl-users wrote: >>> Hi, >>> >>> Check the headers. >>> Some sites might block curls User-Agent. >>> >>> Alternatively, the server might block you because curl is missing >>> cookies or other identifiable headers, for example authorization >>> headers. >>> >>> But we can only guess. >>> >>> If you need the download links again, just check the JavaScript, or >>> monitor network calls from browser when you click the drop down menu. >>> It will be filled somewhere. >>> Either at initial page load, or later via xhr or fetch. >>> >>> I might take a quick look later today. >>> >>> Just experiment a bit though. >>> Use the browser network tab as help. >>> >>> The browser offers copy as curl in the conxt menu for each network call. >>> >>> Compare both curl calls, yours and the browsers. >>> >>> Good luck, >>> >>> Bastian >>> >>> >> I am seeing these two under Firefox's "Network": >> >> https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js >> https://icedrive.net/assets/builds/icedrive.frontend.4029.min.js > > Does this help? > > $ curl -L > https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js -o - > curl: (2) no URL specified > curl: try 'curl --help' for more information > bash: https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js: > No such file or directory > > > $ wget https://icedrive.net/apps/desktop-laptop -o - > [0] Downloading 'https://icedrive.net/apps/desktop-laptop' ... > HTTP ERROR response 403 [https://icedrive.net/apps/desktop-laptop] > > The page I do get start with: <!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title> "Just a moment...". Hmmmmmm -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.html