Downloading user agent
Andreas Kemnade <[email protected]>
| Newsgroups | gmane.comp.web.dillo.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi, I am thinking about the User-Agent which is sent during downloads. Some sites block wget, probably because they do not want to learn robots.txt and do not like a tool doing massdownloads. So if you use dillo to download from such sites, you get a 403 forbidden. If wget would use dillo as user-agent (can be set via a parameter) in such cases, there would be no problem and it is not really wrong. Regards, Andreas