Downloading user agent

Andreas Kemnade <[email protected]>
Newsgroups gmane.comp.web.dillo.devel
Message-ID <[email protected]>
Hi,

I am thinking about the User-Agent which is sent during downloads.
Some sites block wget, probably because they do not want to learn
robots.txt and do not like a tool doing massdownloads.

So if you use dillo to download from such sites, you get a 403
forbidden.
If wget would use dillo as user-agent (can be set via a parameter) in
such cases, there would be no problem and it is not really wrong.

Regards,
Andreas
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.