Re: squatter fatal error, assertion failed - v. 3.6.0-beta3-1+b1

Hi,

Unfortunately not, but I cannot locate the specific emails, which cause
this.
I cannot provide too much useful information.

It seems, when I put a lot of v in the options of squatter, it can
squatter all the messages, at least there is no assertion and it did
not stop.

        # incremental index changed mailboxes (fulltext) approximately every hour
        squatter1       cmd="/usr/sbin/cyrus squatter -vvvvv -p -i" period=180

        # reindex all mailboxes (fulltext) daily
        squattera       cmd="/usr/sbin/cyrus squatter -vvvvv -p" at=0130

In the other hand, the log contains additional info, around the
problematic folders.

Like this:
2022-11-13T01:32:11.937642+01:00 cyrus36 cyrus/squatter[19126]: indexing mailbox user/csaba/Sent... 
2022-11-13T01:32:12.071267+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part ''
2022-11-13T01:32:12.071595+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 10 bytes into message 32091
2022-11-13T01:32:12.078024+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 'h32091'
2022-11-13T01:32:12.078185+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 27 bytes into message 32091
2022-11-13T01:32:12.083676+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 'f32092'
2022-11-13T01:32:12.083837+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 44 bytes into message 32092
2022-11-13T01:32:12.087780+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 't32092'
2022-11-13T01:32:12.088028+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 48 bytes into message 32092
2022-11-13T01:32:12.090078+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 's32092'

(red part is suspicious for me)

But I do not find 1:1  between the additional logs and assertion
problems: seems more logs than assertion. (Maybe all of them could
cause assertion, but squatter normally dies at the first one/account,
so, the rest is invisible at this point.)
And unfortunately I was not able to find the exact mails, which caused
that.
Using mbexamine, I thought I found some lead, but the theory failed
when I tried to prove (emails with similar symptoms could be
squattered).

I had no time to modify the squatter sourcecode with some more debug
(print the message id/filename) and compiling, yet.
I have the idea to print all filenames which opened for squattering and
where the assertion happened, I got the problematic email.
It is still a theory.

Do you think, I could any useful information at this moment?

Thanks, 
István




2022. 11. 14, hétfő keltezéssel 10.43-kor ellie timoney ezt írta:
> Hi István,
> 
> I haven't seen a bug report about this come through GitHub yet, did
> you end up solving the problem yourself?
> 
> Cheers,
> 
> ellie
> 
> On Tue, 8 Nov 2022, at 11:51 PM, Pongrácz István wrote:
> > Thank you, I check it!
> > 
> > cheers,
> > István
> > 
> > 
> > 
> > 
> > 2022. 11. 8, kedd keltezéssel 12.01-kor Nic Bernstein ezt írta:
> > > I would suggest exploring a bit with 'mbexamine(8)' to see what
> > > it shows.  See the man page here:
> > > > https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html
> > > Specifically, if you have either the UID or the sequence number
> > > of a message, you may use the "-u uid" or "-s seqnum" options to
> > > narrow down the output.
> > > 
> > > If you then decide to create a bug report, the output from
> > > mbexamine(8) will be of use.
> > > 
> > > Cheers,
> > >     -nic
> > > On 11/8/22 11:55, Sebastian Hagedorn wrote:
> > > > 
> > > > My suggestion would be to create a bug report on GitHub.
> > > > Cheers, 
> > > > Sebastian
> > > > On 8 Nov 2022, at 12:47, Pongracz Istvan wrote:
> > > > > Hi,
> > > > > 
> > > > > I tried to reconstruct one of the problematic account with
> > > > > this:
> > > > > 
> > > > > /usr/lib/cyrus/bin/reconstruct -r -f -R -u pongraczi
> > > > > 
> > > > > Didn't work, squatter still assert failured:
> > > > > 
> > > > > cyrus/squatter[12044]: ERROR: message has more than 1000
> > > > > header lines not caching any more
> > > > > fatal error: Internal error: assertion failed:
> > > > > imap/squat_internal.c: 134: v64 >= 0
> > > > > 
> > > > > In case of 2 out of 7 assertion issues, that more than 1000
> > > > > header lines exists, but the rest (5) there is no other
> > > > > information, only the folder name available.
> > > > > 
> > > > > Any idea?
> > > > > 
> > > > > Thank you!
> > > > > IStván
> > > > > 
> > > > > 
> > > > > 
> > > > > Pongrácz István <[email protected]> ezt írta
> > > > > (időpont: 2022. nov. 7., H, 16:13):
> > > > > > Hi,
> > > > > > 
> > > > > > We use the cyrus version 3.6.0-beta3-1+b1 and I found an
> > > > > > issue with squatter.
> > > > > > In some accounts there are a folder, where the squatter
> > > > > > dies with fatal error and the whole squatter process cannot
> > > > > > move on.
> > > > > > 
> > > > > > I mean, there are about 40 accounts, called a.... to v....
> > > > > > The squatter starts and can finish about 4 accounts, up to
> > > > > > user 'foo' where it run into a folder, I assume it found an
> > > > > > email and id dies with the following message:
> > > > > > 
> > > > > > process type:EVENT name:squatter1 path:/usr/sbin/cyrus
> > > > > > age:138.664s pid:31781 exited, status 70
> > > > > > 
> > > > > > When I issue the command in command line, as cyrus user
> > > > > > /usr/lib/cyrus/bin/squatter -v -p -u foo
> > > > > > 
> > > > > > I got the following, more specific error message:
> > > > > > Indexing mailbox user/foo/Archives/2013... fatal error:
> > > > > > Internal error: assertion failed: imap/squat_internal.c:
> > > > > > 134: v64 >= 0
> > > > > > 
> > > > > > As I checked the squatter_internal.c it did not change for
> > > > > > years (github).
> > > > > > 
> > > > > > The problem with this, the periodic squatter just dies in
> > > > > > the very beginning and other accounts never will be
> > > > > > squattered.
> > > > > > 
> > > > > > This server populated using recent imapsync from old cyrus
> > > > > > server.
> > > > > > The used squatter is: squat (not xapian).
> > > > > > 
> > > > > > Now I have to run squatter "manually", directly for every
> > > > > > user to get search function usable, more or less.
> > > > > > (using the formula: /usr/lib/cyrus/bin/squatter -v -p -u
> > > > > > foo )
> > > > > > 
> > > > > > Do you know any kind of trick or method, how to
> > > > > > eliminate/solve this issue?
> > > > > > 
> > > > > > Thank you!
> > > > > > István
> > > > 
> > > > 
> > > > -- 
> > > > .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer
> > > > 2.13.:. 
> > > > .:.Regionales Rechenzentrum (RRZK).:. 
> > > > .:.Universität zu Köln / Cologne University - ✆ +49-221-470-
> > > > 89578.:.
> > > 
> > > 
> 
> Cyrus / Info / seediscussions +participants
> +delivery optionsPermalink

------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/Te0aac4ca5a14db4f-Me33f802d9ef0e1a538c96b38
Delivery options: https://cyrus.topicbox.com/groups/info/subscription