Re: I Nerd help downloading a page
ToddAndMargo via curl-users <[email protected]> Sat, 11 Oct 2025 17:39:29 -0700
| Newsgroups | gmane.comp.web.curl.general |
|---|---|
| Message-ID | <[email protected]> |
On 10/11/25 5:03 PM, ToddAndMargo via curl-users wrote: >>> On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <curl- >>> [email protected] <mailto:[email protected]>> wrote: >>> >>> Hi All, >>> >>> curl-8.9.1-4.fc41.x86_64 >>> >>> How come Firefox can see this page, but curl can not? >>> >>> >>> First attempt: >>> >>> $ curl -L https://icedrive.net/apps/desktop-laptop <https:// >>> icedrive.net/apps/desktop-laptop> -o eraseme.html >>> % Total % Received % Xferd Average Speed Time Time >>> Time >>> Current >>> Dload Upload Total Spent >>> Left >>> Speed >>> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:-- >>> --:--:-- >>> 51058 >>> >>> >>> Second attempt (that is Brave Browser user agent. >>> Firefox's did not work either): >>> >>> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64) >>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 >>> <http://141.0.0.0> Safari/537.36" >>> https://icedrive.net/apps/desktop-laptop/ <https://icedrive.net/ >>> apps/desktop-laptop/> -o eraseme.html >>> % Total % Received % Xferd Average Speed Time Time >>> Time >>> Current >>> Dload Upload Total Spent >>> Left >>> Speed >>> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:-- >>> --:--:-- >>> 48422 >>> >>> >>> Yours in confusion, >>> -T > > On 10/11/25 4:47 PM, Bastian Jesuiter via curl-users wrote: >> Hi, >> >> Check the headers. >> Some sites might block curls User-Agent. >> >> Alternatively, the server might block you because curl is missing >> cookies or other identifiable headers, for example authorization headers. >> >> But we can only guess. >> >> If you need the download links again, just check the JavaScript, or >> monitor network calls from browser when you click the drop down menu. >> It will be filled somewhere. >> Either at initial page load, or later via xhr or fetch. >> >> I might take a quick look later today. >> >> Just experiment a bit though. >> Use the browser network tab as help. >> >> The browser offers copy as curl in the conxt menu for each network call. >> >> Compare both curl calls, yours and the browsers. >> >> Good luck, >> >> Bastian >> >> > I am seeing these two under Firefox's "Network": > > https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js > https://icedrive.net/assets/builds/icedrive.frontend.4029.min.js Does this help? $ curl -L https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js -o - curl: (2) no URL specified curl: try 'curl --help' for more information bash: https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js: No such file or directory $ wget https://icedrive.net/apps/desktop-laptop -o - [0] Downloading 'https://icedrive.net/apps/desktop-laptop' ... HTTP ERROR response 403 [https://icedrive.net/apps/desktop-laptop] -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.html