Re: DatabaseModifiedError while iterating on mset

Olly Betts <[email protected]> wrote:
> On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote:
> > I'm already retrying the ->get_mset operations; but now I'm
> > wondering where I'd hit DatabaseModifiedErrors while inside a
> > Xapian::MSetIterator loop.
> > 
> > I assume ->get_document is a place where it gets thrown;
> > but once a document is retrieved, can iterating through
> > terms in one document (using TermIterator) also throw DB modified?
> 
> If you only look at the terms and wdfs then you could only get
> DatabaseModifiedError on the call to create the TermIterator since the
> list of terms and wdfs is stored in a single entry per document which
> is fetched when the iterator is created (it is conceivable this might
> be different for a new database backend in the future I suppose).

Oh wow.  In Perl, I only had a retry_reopen wrapper only around
the get_mset call to reopen the DB because documents get added
frequently:

	my $mset = retry_reopen(sub { $enq->get_mset(0, 1000) });
	for my $m ($mset->items) {
		...
	}

But the above was actually unsafe from modifications and
I should be doing the following?:

	my $mset = retry_reopen(sub { $enq->get_mset(0, 1000) });
	my $cur = retry_reopen(sub { $mset->begin });
	my $end = retry_reopen(sub { $mset->end });
	for (; $cur != $end; retry_reopen(sub { $cur++ })) {
		...
	}

And similar for C++.

On the other hand, modifications to existing documents are not
common in my use cases (but possible) so I've never noticed
errors while though an MSet.

I suppose DocumentNotFound errors can also happen while
iterating an MSet if a writer is deleting documents, too, right?

> If you call methods like get_doclength() which need to consult the
> database those could throw.

OK, will do.  Thanks.