Using multiple temporary indexes during updates

Jean-Francois Dockes <[email protected]> Fri, 15 Mar 2024 20:15:55 +0100
Newsgroups gmane.comp.search.xapian.general
Message-ID <[email protected]>
Hi,

I have been playing at converting the index update stage of the Recoll indexer to use
multiple temporary indexes and a final merge.

This yields an improvement factor of almost 3 (on my quad-core CPU), for the total
indexing time for "easy" files like HTML pages. This is nice (!) and I wanted to share my
admiration for the "compact()" method.

If someone is interested in a bit more detail:
https://www.recoll.org/pages/idxthreads/threadingRecoll.html#_the_xapian_bottleneck_and_how_it_was_resolved_thanks_to_xapian

Cheers,

jf