[issue2551190] Allow reindex to work in batches

New submission from John Rouillard:

Reindexing the database can take a while. It can require taking
the tracker offline.

Provide some way to incrementally reindex classes/items in the database. Consider
allowing the roundup-admin reindex command to take a class name and  range of numbers.

roundup-admin> reindex issues 1-10000
......
roundup-admin> reindex msgs 1-10000
....
roundup-admin> reindex files 1-10000

will reindex the fields in the first 10000 issues, the content in the first 10000 messages
and the same for files. These can be run overnight or during a long weekend. By limiting
the size, the reindexing can be scheduled in windows rather than starting it and
letting it run as long as it has to run.

Concerns:

Should there be a way to eliminate classes from reindex?

  "reindex" reindexes everything, but if I need to do incremental reindexing,
  I would never use reindex (as it would index the classes I want to incremental index
  as well). So I either need to scour the schema and identify every class in the full
  text index and increment every class using "reindex class" or "reindex class 1-<index
  of last element in class>". Can reindex accept a list of classes to skip indexing:

     reindex -files -msgs -issues

  to reindex all classes except these three. Then these three will be incrementally indexed.

Should it be possible to index using a different indexer from the one in config.ini?

   One use case for reindexing is to change the indexer. It would be good to keep
   the current indexing active while building/loading the new indexer.

   Setting indexing backend (and options like stoplist, indexer_language) could be
   another use for an options/pragma setting in roundup-admin see issue2551103.

----------
components: Command-line interface, Database
messages: 7445
nosy: rouilj
severity: normal
status: new
title: Allow reindex to work in batches
type: rfe

_________________________________________________
Roundup tracker <[email protected]>
<https://issues.roundup-tracker.org/issue2551190>
_________________________________________________