Using a document id as metadata key and merges

Jean-Francois Dockes <[email protected]> Thu, 12 Dec 2024 09:51:44 +0100
Newsgroups gmane.comp.search.xapian.general
Message-ID <[email protected]>
Hi,

Following a discussion a few years ago, Recoll stores the documents text
contents in database metadata entries, with keys derived from document ids.

More recently an index creation method using several temporary indexes
merged on completion was implemented. This is still a bit experimental. It
brings a significant speed increase in some cases.

I just realised that the merge lost many metadata entries because of the
document id collisions (I was just using add_document() on the temporary
dbs). It was not immediately obvious because this only affects snippets
generation. 

Would using replace_document() on the temporary dbs, with unique document
ids (modulo) ensure that the document ids are preserved during the merge so
that the metadata keys remain valid ?

Or is there another obvious approach which I am missing ?

Cheers,

J.F. Dockes