git: 6af7769111 - main - documentation/robots.txt: Disallow Past Releases Documentation
"Danilo G. Baio" <[email protected]>
| Newsgroups | gmane.os.freebsd.devel.cvs.doc |
|---|---|
| Message-ID | <[email protected]> |
The branch main has been updated by dbaio: URL: https://cgit.FreeBSD.org/doc/commit/?id=6af77691110d19019b389fad3a0700286fdf0ef1 commit 6af77691110d19019b389fad3a0700286fdf0ef1 Author: Danilo G. Baio <[email protected]> AuthorDate: 2021-08-08 14:12:03 +0000 Commit: Danilo G. Baio <[email protected]> CommitDate: 2021-08-13 22:51:16 +0000 documentation/robots.txt: Disallow Past Releases Documentation It was noticed that Google/DDG sometimes are sending users to the past releases documentation. To avoid misinformation, disallow this archive from the search engines. This robots.txt only covers docs.FreeBSD.org and not www.FreeBSD.org, so also remove unnecessary items. Differential Revision: https://reviews.freebsd.org/D31459 --- documentation/themes/beastie/layouts/robots.txt | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-) diff --git a/documentation/themes/beastie/layouts/robots.txt b/documentation/themes/beastie/layouts/robots.txt index b94bcd43cf..f59c629169 100644 --- a/documentation/themes/beastie/layouts/robots.txt +++ b/documentation/themes/beastie/layouts/robots.txt @@ -1,19 +1,2 @@ -# $FreeBSD$ - User-agent: * -#Crawl-delay: 10 -Disallow: /cgi/confirm-code.cgi -Disallow: /cgi/cvsweb.cgi -Disallow: /cgi/dosendpr.cgi -Disallow: /cgi/getmsg.cgi -Disallow: /cgi/mailindex.cgi -Disallow: /cgi/man.cgi?apropos=2& -Disallow: /cgi/mid.cgi -Disallow: /cgi/mirror.cgi -Disallow: /cgi/missing_handler.cgi -Disallow: /cgi/ports.cgi -Disallow: /cgi/query-pr-summary.cgi -Disallow: /cgi/query-pr.cgi -Disallow: /cgi/search.cgi -Disallow: /cgi/url.cgi -Disallow: /statistic +Disallow: /doc/ _______________________________________________ [email protected] mailing list https://lists.freebsd.org/mailman/listinfo/dev-commits-doc-all To unsubscribe, send any mail to "[email protected]"