modifying the DB while iterating is user error, right?

Eric Wong <[email protected]>
Newsgroups gmane.comp.search.xapian.general
Message-ID <[email protected]>
Hello, I noticed a problem with DatabaseCorruptError exceptions
with public-inbox and I guess it's user error...

The problem is public-inbox was calling replace_document to
modify the DB while iterating through a PostingIterator.  At
first I thought it was a glass problem, but I've hit it with
chert on my dataset, too.

I have a standalone Perl script to reproduce the problem at
https://yhbt.net/skel.bug.perl and 81M gzipped dataset which
reproduces the problem at https://yhbt.net/skel.bug.gz

(each line is: MID [REFERENCES-SEPARATED-BY-SPACES])

Usage:

For failure:
  curl https://yhbt.net/skel.bug.gz | zcat | perl -w /path/to/skel.bug.perl

For success:
  curl https://yhbt.net/skel.bug.gz | zcat | \
        BATCH_SIZE=1000 perl -w /path/to/skel.bug.perl


I've also informed notmuch of the problem, since I stole
the thread linking logic from that for public-inbox:
https://www.mail-archive.com/search?l=mid&q=20180227224414.GA31804@dcvr
(currently greylisted, should show up momentarily)
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.