svn commit: r1360523 - /xalan/site/docs/xalan/robots.txt
[email protected] Thu, 12 Jul 2012 04:19:49 -0000
| Newsgroups | gmane.text.xml.xalan.cvs |
|---|---|
| Message-ID | <[email protected]> |
Author: shathaway
Date: Thu Jul 12 04:19:49 2012
New Revision: 1360523
URL: http://svn.apache.org/viewvc?rev=1360523&view=rev
Log:
Add robots.txt to limit web crawlers access to specific content
Added:
xalan/site/docs/xalan/robots.txt
Added: xalan/site/docs/xalan/robots.txt
URL: http://svn.apache.org/viewvc/xalan/site/docs/xalan/robots.txt?rev=1360523&view=auto
==============================================================================
--- xalan/site/docs/xalan/robots.txt (added)
+++ xalan/site/docs/xalan/robots.txt Thu Jul 12 04:19:49 2012
@@ -0,0 +1,10 @@
+#
+# Xalan website control: robots.txt
+#
+# Keep web crawlers out of auto-generated api documentation pages
+#
+User-agent: *
+Disallow: /xalan-c/apiDocs/
+Disallow: /xalan-j/apidocs/
+Disallow: /old/xalan-c/apiDocs/
+Disallow: /old/xalan-j/apidocs/