Re: cyr_expire suddenly segfaulting

Bron Gondwana <[email protected]>
Newsgroups gmane.mail.imap.cyrus
Message-ID <1489365158.3301376.908986416.229CF301@webmail.messagingengine.com> 
So everything is 2.4 defaults, which are:

brong@wot:~/src/cyrus-imapd$ grep '_db"' lib/imapoptions 
{ "annotation_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist")}
{ "duplicate_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
{ "mboxkey_db", "skiplist", STRINGLIST("berkeley", "skiplist") }
{ "mboxlist_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist")}
{ "ptscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist")}
{ "quota_db", "quotalegacy", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "quotalegacy")}
{ "seenstate_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist")}
{ "subscription_db", "flat", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist")}
{ "statuscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist") }
{ "tlscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
{ "userdeny_db", "flat", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql")}

So you shouldn't have any berkeley databases at all.  It should be perfectly safe to shut down Cyrus, delete the $confdir/db folder, and restart Cyrus.

that's:

rm -rf /var/lib/imap/db/

While Cyrus is shut down :)

(Normal warnings about backing up your stuff and making sure that Centos packages aren't doing something different with defaults than upstream)

Cheers,

Bron.

On Mon, 13 Mar 2017, at 08:28, Nick Howitt wrote:
> 
> 
> On 12/03/2017 21:09, Patrick Boutilier wrote:
> > On 03/12/2017 02:24 PM, Nick Howitt wrote:
> >> Hi,
> >> I've been running cyrus-imap for a few years not. I am using 2.4.17-8 on
> >> ClearOS (a Centos derivative). I've been running with "delprune
> >> cmd="cyr_expire -E 1 -X 7 -D 7" at=0001" in cyrus.conf, but in the last
> >> week or so it has started segfaulting. I've tried moving the command to
> >> cron.daily "/usr/lib/cyrus-imapd/cyr_expire -E 1 -X 7 -D 7" but it still
> >> segfaulting.
> >>
> >> In /var/log/maillog I get:
> >> Mar 12 03:15:53 server cyr_expire[15065]: Repacking mailbox 
> >> user.mail-ebay
> >> Mar 12 03:15:53 server master[15064]: setrlimit: Unable to set file
> >> descriptors limit to -1: Operation not permitted
> >> Mar 12 03:15:53 server master[15064]: retrying with 4096 (current max)
> >> Mar 12 03:15:53 server cyr_expire[15065]: Repacking mailbox
> >> user.mail-ebay.Drafts
> >> Mar 12 03:15:54 server cyr_expire[15065]: Repacking mailbox user.nick
> >> Mar 12 03:15:54 server cyr_expire[15065]: Repacking mailbox 
> >> user.ourfamily
> >> Mar 12 03:15:54 server cyr_expire[15065]: Repacking mailbox
> >> user.ourfamily.Drafts
> >> Mar 12 03:15:54 server cyr_expire[15065]: Expunged 0 out of 9150
> >> messages from 57 mailboxes
> >> Mar 12 03:15:54 server cyr_expire[15065]: duplicate_prune: pruning back
> >> 0.00 days
> >> Mar 12 03:15:54 server ctl_cyrusdb[15068]: recovering cyrus databases
> >> Mar 12 03:15:55 server ctl_cyrusdb[15068]: skiplist: checkpointed
> >> /var/lib/imap/mailboxes.db (57 records, 4816 bytes) in 0 seconds
> >> Mar 12 03:15:55 server ctl_cyrusdb[15068]: skiplist: checkpointed
> >> /var/lib/imap/annotations.db (0 records, 144 bytes) in 0 seconds
> >> Mar 12 03:15:55 server ctl_cyrusdb[15068]: done recovering cyrus 
> >> databases
> >> Mar 12 03:15:55 server master[15064]: unable to setsocketopt(IP_TOS):
> >> Operation not supported
> >> Mar 12 03:15:55 server ctl_cyrusdb[15073]: checkpointing cyrus databases
> >> Mar 12 03:15:55 server ctl_cyrusdb[15073]: done checkpointing cyrus
> >> databases
> >> Mar 12 03:15:55 server imaps[15075]: error sending to idled: 3
> >> Mar 12 03:15:55 server imap[15074]: error sending to idled: 3
> >> Mar 12 03:15:55 server imap[15078]: error sending to idled: 3
> >> Mar 12 03:15:55 server imap[15076]: error sending to idled: 3
> >> Mar 12 03:15:55 server imaps[15081]: error sending to idled: 3
> >> Mar 12 03:15:55 server imap[15080]: error sending to idled: 3
> >> Mar 12 03:15:55 server imaps[15077]: error sending to idled: 3
> >> Mar 12 03:15:55 server imaps[15079]: error sending to idled: 3
> >> Mar 12 03:15:56 server tls_prune[15071]: skiplist: checkpointed
> >> /var/lib/imap/tls_sessions.db (575 records, 122352 bytes) in 0 seconds
> >> Mar 12 03:15:56 server cyr_expire[15072]: skiplist: checkpointed
> >> /var/lib/imap/deliver.db (37 records, 5464 bytes) in 0 seconds
> >> Mar 12 03:15:56 server cyr_expire[15072]: Expunged 0 out of 9112
> >> messages from 57 mailboxes
> >> Mar 12 03:15:56 server cyr_expire[15072]: duplicate_prune: pruning back
> >> 3.00 days
> >> Mar 12 03:15:56 server cyr_expire[15072]: duplicate_prune: purged 0 out
> >> of 37 entries
> >>
> >> The "error sending to idled: 3" is probably a red herring as cyrus-imap
> >> has just restarted following a db backup, so earlier idled connections
> >> have broken.
> >>
> >> Correspondingly in /var/log/messages I get:
> >> Mar 12 03:15:56 server kernel: cyr_expire[15065]: segfault at
> >> 7f9c8a79be78 ip 00007f9c8a91c5ed sp 00007ffd44b51d38 error 4 in
> >> cyr_expire[7f9c8a8c9000+15c000]
> >>
> >> It may also be worth mentioning that on some nights, like the night
> >> before the segfault, in /var/log/maillog I see:
> >> Mar 11 03:42:09 server cyr_expire[3237]: Expunged 0 out of 9145 messages
> >> from 61 mailboxes
> >> Mar 11 03:42:09 server cyr_expire[3237]: duplicate_prune: pruning back
> >> 1.00 days
> >> Mar 11 03:42:11 server cyr_expire[3237]: duplicate_prune: purged 58 out
> >> of 147 entries
> >> Mar 11 03:45:39 server ctl_cyrusdb[25201]: checkpointing cyrus databases
> >> Mar 11 03:45:39 server ctl_cyrusdb[25201]: done checkpointing cyrus
> >> databases
> >> Mar 11 03:49:57 server master[31633]: exiting on SIGTERM/SIGINT
> >> Mar 11 03:50:58 server master[27624]: setrlimit: Unable to set file
> >> descriptors limit to -1: Operation not permitted
> >> Mar 11 03:50:58 server master[27624]: retrying with 4096 (current max)
> >> Mar 11 03:50:58 server cyr_expire[27625]: Repacking mailbox 
> >> user.mail-ebay
> >> Mar 11 03:50:58 server cyr_expire[27625]: Repacking mailbox 
> >> user.ourfamily
> >> Mar 11 03:50:59 server cyr_expire[27625]: Repacking mailbox
> >> user.ourfamily.Drafts
> >> Mar 11 03:50:59 server ctl_cyrusdb[27628]: recovering cyrus databases
> >> Mar 11 03:50:59 server ctl_cyrusdb[27628]: skiplist: checkpointed
> >> /var/lib/imap/mailboxes.db (61 records, 5148 bytes) in 0 seconds
> >> Mar 11 03:50:59 server ctl_cyrusdb[27628]: skiplist: checkpointed
> >> /var/lib/imap/annotations.db (0 records, 144 bytes) in 0 seconds
> >> Mar 11 03:50:59 server ctl_cyrusdb[27628]: done recovering cyrus 
> >> databases
> >> Mar 11 03:50:59 server cyr_expire[27625]: Expunged 0 out of 9145
> >> messages from 61 mailboxes
> >> Mar 11 03:50:59 server cyr_expire[27625]: duplicate_prune: pruning back
> >> 0.00 days
> >> Mar 11 03:50:59 server master[27624]: unable to setsocketopt(IP_TOS):
> >> Operation not supported
> >> Mar 11 03:50:59 server ctl_cyrusdb[27633]: checkpointing cyrus databases
> >> Mar 11 03:50:59 server ctl_cyrusdb[27633]: done checkpointing cyrus
> >> databases
> >> Mar 11 03:51:00 server tls_prune[27631]: skiplist: checkpointed
> >> /var/lib/imap/tls_sessions.db (303 records, 64348 bytes) in 1 second
> >> Mar 11 03:51:00 server cyr_expire[27632]: skiplist: checkpointed
> >> /var/lib/imap/deliver.db (84 records, 12424 bytes) in 0 seconds
> >> Mar 11 03:51:00 server cyr_expire[27632]: Expunged 0 out of 9109
> >> messages from 61 mailboxes
> >> Mar 11 03:51:00 server cyr_expire[27632]: duplicate_prune: pruning back
> >> 3.00 days
> >> Mar 11 03:51:00 server tls_prune[27631]: tls_prune: purged 3 out of 303
> >> entries
> >> Mar 11 03:51:00 server cyr_expire[27632]: duplicate_prune: purged 0 out
> >> of 83 entries
> >> Mar 11 03:51:04 server cyr_expire[27625]: duplicate_prune: purged 89 out
> >> of 89 entries
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR db5: BDB1581 File
> >> handles still open at environment close
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR db5: BDB1582 Open file
> >> handle: /var/lib/imap/db/__db.001
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR db5: BDB1582 Open file
> >> handle: /var/lib/imap/db/__db.002
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR db5: BDB1582 Open file
> >> handle: /var/lib/imap/db/__db.003
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR db5: BDB0060 PANIC:
> >> fatal region error detected; run recovery
> >> Mar 11 03:51:04 server cyr_expire[27625]: DBERROR: critical database
> >> situation
> >>
> >> I'm afraid I've absolutely no idea how to troubleshoot this. Can anyone
> >> please help?
> >>
> >> Thanks,
> >>
> >> Nick
> >
> >
> > Looks like you might have a corrupted BerlkeyDB . What is the output of ?
> >
> > grep _db /etc/imapd.conf
> Nothing? I can't see any alternative file that may hold this info. 
> imapd.conf is:
> 
> configdirectory: /var/lib/imap
> partition-default: /var/spool/imap
> admins: root
> sievedir: /var/lib/imap/sieve
> sendmail: /usr/sbin/sendmail
> hashimapspool: true
> sasl_pwcheck_method: saslauthd
> sasl_mech_list: PLAIN
> tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
> tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
> tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
> 
> flushseenstate:       1
> allowplaintext:       yes
> reject8bit:           no
> munge8bit:            no
> lmtp_over_quota_perm_failure: 1
> timeout:              30
> imapidlepoll:         60
> idlesocket:           /var/lib/imap/socket/idle
> lmtpsocket:           /var/lib/imap/socket/lmtp
> allowapop:            no
> altnamespace:         0
> unixhierarchysep:     yes
> lmtp_downcase_rcpt:   yes
> username_tolower:     1
> autocreatequota:      -1
> createonpost:         1
> virtdomains:          off
> expire-days:    1
> expunge-days:    1
> delete-days:    7
> 
> >
> >
> >
> >>
> >> ----
> >> Cyrus Home Page: http://www.cyrusimap.org/
> >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> >> To Unsubscribe:
> >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> >
> >
> >
> > ----
> > Cyrus Home Page: http://www.cyrusimap.org/
> > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> > To Unsubscribe:
> > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


-- 
  Bron Gondwana
  [email protected]
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.