Using multiple temporary indexes during updates
Jean-Francois Dockes <[email protected]> Fri, 15 Mar 2024 20:15:55 +0100
| Newsgroups | gmane.comp.search.xapian.general |
|---|---|
| Message-ID | <[email protected]> |
Hi, I have been playing at converting the index update stage of the Recoll indexer to use multiple temporary indexes and a final merge. This yields an improvement factor of almost 3 (on my quad-core CPU), for the total indexing time for "easy" files like HTML pages. This is nice (!) and I wanted to share my admiration for the "compact()" method. If someone is interested in a bit more detail: https://www.recoll.org/pages/idxthreads/threadingRecoll.html#_the_xapian_bottleneck_and_how_it_was_resolved_thanks_to_xapian Cheers, jf