Webboard: exclude mime types

[email protected] Thu, 13 Oct 2016 08:31:39 +0400 (MSK)

Newsgroups	gmane.comp.web.mnogosearch.general
Message-ID	<[email protected]>

Author: Alexander Barkov
Email: 
Message:
> And to be more precise, i finally want to index only html pages and not all other types of data (css/js/pictures/pdf/rss/...) .
> 

Something like this should do the trick:

NoIndexIf NoMatch Content-Type text/html*


Additionally, try to use the Disallow command to reduce the number of URLs that indexer has actually to download.
See here for details:
http://www.mnogosearch.org/board/message.php?id=21793


> Fabien.
> 
<cut>


Reply: <http://www.mnogosearch.org/board/message.php?id=21794>

_______________________________________________
General mailing list
[email protected]
http://lists.mnogosearch.org/listinfo/general