Re: following redirects
Daniel Naber <[email protected]> Thu, 15 Jan 2004 20:33:08 +0100
| Newsgroups | gmane.comp.web.perlfect-search |
|---|---|
| Message-ID | <[email protected]> |
On Thursday 15 January 2004 13:13, BOCZ Peter wrote: > Is it a feature that web crawler follows and indexes redirects to > external URLs? No, it's a bug. The last lines in the get_url() function in tools.pl have a comment about that. The problem is that the check_accept_url() call should be done again after this line, but it isn't: $url = $response->base; You could try that and report if it helps. Regards Daniel -- http://www.danielnaber.de _______________________________________________ perlfect-search mailing list [email protected] To unsubscribe, set other personal options or view the list archives please visit: http://perlfect.com/mailman/listinfo/perlfect-search