RE: New Spider
wizardsandwars <[email protected]> 6 Sep 2003 15:48:25 -0000
| Newsgroups | gmane.comp.web.oscommerce.tips |
|---|---|
| Message-ID | <0f9dc5a892aa092e29147367be389cc6@osCommerce-Forums> |
This message was sent from: Tips and Tricks http://forums.oscommerce.com/viewtopic.php?p=225272#225272 ---------------------------------------------------------------- Jason, You can prohibit the spider from entering your completly, via your htaccess file, however, I wouldn't recommend this, as it could affect your listing wiht that seach engine. Just wait it out. It's really not doing anything other than sucking up some bandwidth. I've got this new bot on my page as well, and I cought it early, and it's still spidering hte hell out of me even wihtout the SIDs. Hopefuly, we'll all be enjoying some fresh lisintgs on yahoo soon.