possibilty to exclude /nonav/...

Vincent Brabant <[email protected]> Sat, 21 May 2005 17:06:27 +0200

Newsgroups	gmane.comp.java.netbeans.webteam
Message-ID	<[email protected]>

I think we have to add the
Disallow: /nonav/
line in the robots.txt file
Otherwize, it causes somes problems with bots that are scaning all 
sources and images available in /nonav/sources and /nonav/images.

Could you check if that robots.txt file is well existing for ALL 
subdomain. By example, for FR, it was not the case.

Vincent