Webboard: Frequent pauses in indexer

[email protected]
Newsgroups gmane.comp.web.mnogosearch.general
Message-ID <[email protected]> 
Author: Michael
Email: 
Message:
When indexer runs, it pauses for 10 to 15 seconds after processing a small batch of URLs (usually 20 or 30). The result is that indexing takes a very long time. 



On our other servers, the indexing run finishes in about 4 hours. But on this one it goes for closer to 12.



The machine has a fair amount of RAM (4GB) and is not doing anything else at the time. The URLs being crawled are local to this server. Wget is able to crawl the same URLs with no delays. A "show processlist" in MySQL during the delay shows only one entry, in status "sleep".



Any idea what is causing the delays?

Reply: <http://www.mnogosearch.org/board/message.php?id=21292>
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.