Re: Are spider's bad?
Adrenolin <[email protected]> 5 Aug 2003 19:15:00 -0000
| Newsgroups | gmane.comp.web.oscommerce.tips |
|---|---|
| Message-ID | <4a87c1c172a28bcda01e84fe6a08236b@osCommerce-Forums> |
This message was sent from: Tips and Tricks http://forums.oscommerce.com/viewtopic.php?p=208100#208100 ---------------------------------------------------------------- Roger, Thats correct but there are also areas of a website you dont want spidered, or to be on topic.. You dont want spiders indexing your site with SIDs. Thats what the spider killer does.. it removes the SID part of the URL for known spiders so they dont index them. [quote="RPettyJr"]I thought that spiders are good because they are the information retrieveal components of a search engine and thusly help with your placements in searches. Looking for clarificaton. Roger[/quote]