RE: New Spiders - lots of them for Burt's SID killer fix

This message was sent from: Tips and Tricks
http://forums.oscommerce.com/viewtopic.php?p=197678#197678
----------------------------------------------------------------

Actually, Burt, I'm having the same problem.

And here's my spider killer. the inktomi bot's ip is 66.196.73. and the agent is "Mozilla/5.0 (Slurp/cat; [email protected]; http://www.inktomi.com/slurp.html)".

Both of those should have registered.

[code]
      $spider_footprint = array( "inktomi", "grub", "Googlebot", "googlebot", "InternetSeer", "WebCrawler", "FST", "FST2.1", "Scooter", "FST2.0", "ia_archiver", "Robozilla", "Slurp");
      $spider_ip = array( "66.207.121.", "66.196.73.", "217.120.83.", "12.154.86.15", "64.140.49.", "216.239.46.", "64.209.181.53", "64.208.33.33", "64.209.181.52", "209.185.108.", "209.185.253", "216.239.49.", "204.123.", "204.74.103.", "203.108.10.", "195.4.183.", "195.242.46.", "198.3.97.", "204.62.245.", "193.189.227.", "209.1.12.", "204.162.96.", "204.162.98.", "194.121.108.", "128.182.72.", "207.77.91.", "206.79.171.", "207.77.90.", "208.213.76.", "194.124.202.", "193.114.89.", "193.131.74.", "131.84.1.", "208.219.77.", "206.64.113.", "195.186.1.", "195.3.97.", "194.191.121.", "139.175.250.", "209.73.233.", "194.191.121.", "198.49.220.", "204.62.245.", "198.3.99.", "198.2.101.", "204.192.112.", "206.181.238", "208.215.47.", "171.64.75.", "204.162.98.", "204.162.96.", "204.123.9.52", "204.123.2.44", "204.74.103.39", "204.123.9.53", "204.62.245.", "206.64.113.", "204.138.115.", "94.22.130.", "164.195.64.1", "205.181.75.169", "129.170.24.57", "204.162.96.", "204.162.96.", 
 "204.162.98.", "204.162.96.", "207.77.90.", "207.77.91.", "208.200.146.", "204.123.9.20", "204.138.115.", "209.1.32.", "209.1.12.", "192.216.46.49", "192.216.46.31", "192.216.46.30", "203.9.252.2");

      $agent = getenv('HTTP_USER_AGENT');
      $host_ip = getenv('REMOTE_ADDR');
      $is_spider = 0;

      // Is it a spider?

      $i = 0;
      while ($i < (count($spider_footprint)))
      {
      if (eregi($spider_footprint[$i],$agent))
      {
      $is_spider = 1;
      break;
      }
      $i++;
      }

      if (! $is_spider)
      {
        $i = 0;
        while ($i < (count($spider_ip)))
      {
        if (strstr($host_ip, $spider_ip[$i]))
      {
      $is_spider = 1;
      break;
      }
      $i++;
      }
      }
[/code]