RE: New Spider
Adrenolin <[email protected]> 4 Aug 2003 13:18:10 -0000
| Newsgroups | gmane.comp.web.oscommerce.tips |
|---|---|
| Message-ID | <ad8a7e559623e696b0a30a7df9775334@osCommerce-Forums> |
This message was sent from: Tips and Tricks http://forums.oscommerce.com/viewtopic.php?p=207259#207259 ---------------------------------------------------------------- Ohh just as a side note.. Here are log entries which you may find useful if you dont have access to yours. [url]http://www.searchengineworld.com/cgi-bin/sim_spider.cgi[/url] westhost33.westhost.net - - [04/Aug/2003:09:23:19 +0000] "GET / HTTP/1.1" 200 31805 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT 5.0)" and [url]http://www.webconfs.com/search-engine-spider-simulator.php[/url] 203.199.124.111 - - [04/Aug/2003:09:23:57 +0000] "GET / HTTP/1.0" 200 31699 "-" "[email protected]" Searchengineworld was not in the Spider Killer contrib I use on MS1 and displayed the SIDs. Once I added the IP for testing it no longer displayed them. WebConf shows up as "[email protected]" which was already added so it didn't display them at all.