Re: Alternative User-Agent header
Tim Chase <[email protected]>
| Newsgroups | gmane.comp.web.lynx.devel |
|---|---|
| Message-ID | <[email protected]> |
On 2021-06-24 13:49, Stef Caunter wrote: > alias lynx='/usr/local/bin/lynx -nopause -tna -useragent=IE . ' > > "IE" seems to be the most innocuous choice for the major news > sites. I have not tried "googlebot" ;) I'm a big fan of using the googlebot User Agent on the grounds that I've encountered a number of sites where the paywall allows Google through to index the full article content (and occasionally doesn't bother to serve advertisements to the Googlebot) but prevents other user-agent strings from receiving the article. -tim _______________________________________________ Lynx-dev mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/lynx-dev