Re: Consistency check on all db

Le 15/11/2024 à 14:57:01+0100, Michael Menge a écrit

Hi, 

> > Hi everyone.
> > 
> > One user with I don't know what, and I don't know how was able to create a
> > recursive tree of mailbox:
> > 
> >   user.toto.mailbox1.mailbox2.mailbox1.mailbox2. ...etc
> >   user.toto.mailbox1.mailbox3.mailbox1.mailbox3. etc..
> >   etc..
> > 
> > I was able to manually destroy almost all the mailboxes.....but few remain
> > undeleteable...meaning I cannot see them with
> > 
> 
> did you see an error message? Or are there related errors in the logs?

No error message in normal situation, but when I launch squatter they going
to tell me they are some maiboxes issues
 
> >   cyradm + lm user.toto.*
> > 
> > so for the user it's ok.
> > 
> > But if I do a
> > 
> >   ctl_mboxlist -d
> 
> what do you see here. If it has "(null)" in the output it is a tombstone
> record,

no, no null at all. 

> afaik it was added for syncronisation/backup to indicate that a mailbox did
> exist
> but was removed "recently"

No I remove the mailboxes ~one mounth ago. And I don't use sync_repl since
we got a lot a issue with that. We just use zfs/send/recv

> 
> There was an issue to create dokumentation for this feature
> but it is still open https://github.com/cyrusimap/cyrus-imapd/issues/2265
> 
> 
> 
> If it is under DELETED.user.toto it is there to allow admins to restore
> deleted mailboxes
> see "delete_mode:" in man imapd.conf

Yes I know the delete_mode, it's why (even at the time it's mostly for mail
not mailboxes) I switch to cyrus ;-) ;-)

> >   ctl_mboxlist -d > list.json
> >   vi list.json  + remove those mailboxes
> >   ctl_mboxlist -u < list.json
> > 
> 
> This was stupid. Sorry that i am blunt but any mailbox
> created/renamed/deleted acls changed

No worries, I prefer that than something incomprehensible 

> between dumping the mboxlist and updating the mboxlist will most likely be
> lost and
> create an inconsistency between mailboxes.db and the filesystem

Ok. 

> 
> With the uuid Mailbox location, reconstruct might be able to fix some of
> the problems but i don't use uuid mailbox locations jet, so i am not sure.


> > At the end I do something mark «as not to do» such as
> > 
> >   systemctl stop cyrus-imapd
> >   ctl_mboxlist -d > list.json
> >   vi list.json + remove those mailboxes
> >   rm -rf MAILBOX/uuid/*/*/uniqueid of those mailboxes
> >   ctl_mboxlist -u < list.json -f /tmp/mailboxes.db
> >   mv /tmp/mailboxes.db /var/lib/cyrus
> >   chown cyrus:mail /var/lib/cyrus/mailboxes.db
> >   systemctl start cyrus-imapd
> > 
> 
> This is the safer way, but still if the problem is not visible/critical

Ok thanks. 

> to the user, so that there is no need to rush, asking on the list
> for advise is the way to go, espacially if it is marked as "not to do"

Well, it's on a pre-production server, who going to production end of next
week, so I still go enough time to resynchronize everything. 


> > now I don't see them anymore....piouff...
> > 
> > But I'm now worry I introduce some inconsistency somewhere.
> > 
> 
> for the future some general tips.
> 
> - Stoping Cyrus before you change/remove files.
> - Keeping a backup of files you change (e.g. mailboxes.db)
> - Moving the Files/Directories  outside of the cyrus directories instead of
>   removing them with "rm"
>   You can delete them later if you have confirmed that there is no problem

Yeah...I did that, the list of the commands are just for explanation, it'
s not copy/paste. 

The server is a lxc running on zfs, so I even stop the lxc, take a zfs
snapshot and restart the lxc just in case I broke so much I was unable to
restart cyrus. 

I eventually find out the «real» issue, or maybe I should say I'm guessing
it's the real issue. 

In the list dump by ctl_info every line are unique but...those problematic
mailboxes who show 2 times each with exactly the same infos. 

Here a example (I change the name or private infos): 

"DELETED.user.toto.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.2 - THINGS.E": {"name": "DELETED.user.toto.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.2 - THINGS.E", "mtime": "1731619948", "uidvalidity": "1726619337", "createdmodseq": "7876", "foldermodseq": "18587", "mbtype": "e", "partition": "default", "acl": {"toto": "lrswipkxtecdan", "cyrus": "xc"}, "uniqueid": "ylb11hedkbo084glacjmo5bt", "name_history": []},

"DELETED.user.toto.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.2 - THINGS.E": {"name": "DELETED.user.toto.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.divers.IR - MIRS.2 - THINGS.E", "mtime": "1727701437", "uidvalidity": "1726619337", "createdmodseq": "7876", "foldermodseq": "18587", "mbtype": "e", "partition": "default", "acl": {"toto": "lrswipkxtecdan", "cyrus": "xc"}, "uniqueid": "ylb11hedkbo084glacjmo5bt", "name_history": []},

I've no idea how and why. Maybe because cyrus cut the name of the mailboxes
at some lenght and because it's so long the cut part is identical. 

> > Is they are any tool I can check every db and their relations to check
> > everything is ok ?
> 
> check manpage for reconstruct.
> 

I will do that. 

Thanks for the help and tips. 

Regards.


-- 
Albert SHIH 🦫 🐸
France
Heure locale/Local time:
ven. 15 nov. 2024 15:03:02 CET

------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/Tb03cd1a701e8d6d4-Mf8cedec07d32fb77714fbe24
Delivery options: https://cyrus.topicbox.com/groups/info/subscription