RE: New Spider

This message was sent from: Tips and Tricks
http://forums.oscommerce.com/viewtopic.php?p=209977#209977
----------------------------------------------------------------

The problem with the PHP sessions is that, by default, the code will try to set a cookie storing the session number. Because you cannot set a cookie for a spider, the SID is passed in the URL instead. However, the sID is different for every session. 

Spiders like to revisit pages to see if the content has changed, etc. It can take several spider visits to the same page for it to be properly indexed. However, when the sID is appended to the URL , the spider can never find the same URL twice (because the sID has changed).  This can kill a site's search-engine rankings. 

The session-killer simply removes the sID from the URL for known spiders. It does not stop any pages from being indexed; robot-text does that. In-fac, it does the opposite, and helps your pages be spiered.