Re: speeding searches by pre-indexing
Andrew Daviel <[email protected]>
| Newsgroups | gmane.mail.imap.uw.c-client |
|---|---|
| Message-ID | <[email protected]> |
On Fri, 12 Jun 2009, Mark Crispin wrote: > On Fri, 12 Jun 2009, Andrew Daviel wrote: >> 129,634 messages in 23 hours > > Yikes! What is that? Indexing time? Search time? Indexing time. Sorry, stripped off the original detail. I was trying to get a global search for all my mailboxes by pointing htdig at my webmail application. So what I end up with is a web search engine that returns links into webmail. Works, friendlier than "grep whatever mail/.mix*", but indexing too slow for real world use. The slowness is almost all in htdig (the indexing robot). It's single-threaded, and when it was written hammering websites at >> 2 requests/minute was considered rude.