Webboard: How to speed up the crawl delay after each URL ?
[email protected] Wed, 4 May 2016 17:41:50 +0400 (MSK)
| Newsgroups | gmane.comp.web.mnogosearch.general |
|---|---|
| Message-ID | <[email protected]> |
Author: rafikCyc Email: Message: Hello, I've tried this : ./indexer -p 0 but it doesn't work :( The indexer sleeps for at least one seconde after each URL. It seems impossible to index faster than 1s after each url. To index 300 000 document on my website for example, the crawl takes 2 full days ! Is there a solution ? Reply: <http://www.mnogosearch.org/board/message.php?id=21760> _______________________________________________ General mailing list [email protected] http://lists.mnogosearch.org/listinfo/general