robots.txt
[email protected] (Andrei Zmievski) Mon, 17 Jul 2006 09:44:19 -0700
| Newsgroups | php.gtk.webmaster |
|---|---|
| Message-ID | <[email protected]> |
Should we update our robots.txt file? I see a lot of crawling going on, both from search engine crawlers and other stuff. -Andrei