Re: duplicate mailboxes upon migrating (different name forms)

Hi Brain,

to answer some of your original questions.

> Where I screwed up? in imapd.conf:
> OLD server "unixhierarchysep: no"
> NEW server "unixhierarchysep: yes"

yes. this is probably the cause of your problem.

"unixhierarchysep: no" means "." is uses as hierarchyseperator.
and "unixhierarchysep: yes" means "/" is the seperator.

> Everything appeared to copy over fine, with permissions all correct, but
> cyrus now sees duplicate mailboxes, like so:
>
>
> [email protected] <- what I created
> user^foo@example^com <- what's this?


In case of "unixhierarchysep: yes" "." is a valid character in folder names
and is replaced by "^" in internal representation and on filesystem-

Where do you see these two different mailboxes, both in cyradm?
If you have seen it in the reconstruct error message, this is a "problem" of
reconstruct using the "normal" and "internal" mailbox name in the same output.

So if you see "[email protected]" in cyradm and  
user^foo@example^com in the filesystem
or some error logs this is only one mailbox

> After copying over with rsync, and checking that permissions looked ok, I
> ran reconstruct ...
>

If you copy a cyrus server via rsync you have to copy some extra  
files/directories in addition
to the directory configured by  "partition-default:" in impd.conf
You also need the files from the "configdirectory:", "sievedir:" and  
also files from any metapartition
and archivepartition you had used (if you used these features).

If you have lost/did not copy the files from the configdirectory you  
are missing the mailboxes.db .
This database has a list of all mailboxes and some other information  
including acls for each mailbox,
and is used by cyrus to know if a mailbox does exist, and is also used  
by reconstruct for finding the
starting folders used by the "-r" and the "-f" options.

There used to by a "-m" option for reconstruct that would by very  
handy now but

>
> -m     NOTE: CURRENTLY UNAVAILABLE
>
>              Rebuild  the  mailboxes file.  Use whatever data in the existing
>              mailboxes file it can scavenge, then scans all partitions listed
>              in the imapd.conf(5) file for additional mailboxes.
>


> bally@server:~$ su -c "/usr/lib/cyrus/bin/reconstruct -r -f user.*" cyrus
> -s /bin/bash
> Password:
> user^foo@example^com: failed to read index header
> [email protected]
> ...
>

you are still using the "unixhierarchysep: yes" and it seems that you  
mailboxes.db has been
"recreated" with only one mailbox using the wrong seperator regarding  
the seperator used by
the filesystem.


> Oh, dear. Let's list the mailboxes:
>
> bally@server:~$ cyradm --user cyrus --server localhost
> Password:
> localhost> lm
> [email protected] (\HasNoChildren)

If you did not use acls, the mailboxes.db could be recreated from the  
files in the "partition-default:" directory, by hand but you are  
missing mailbox subscriptions, seen status (can't remember if this was  
still
stored in "$configdirectory/users/$hash/*.seen" in cyrus 2.4"), and  
sieve scripts.

How many users/mailboxes do you have?

Also before you do anything else, stop cyrus, and check your old  
configuration and new configuration for any other changes  (e.g.  
hashimapspool:, fulldirhash:)




Quoting brian ally <[email protected]>:

> Thanks, Ellie. 2.4.18-3 to 3.0.13-5
>
> Unfortunately, i no longer have the possibility of replicating. I have to
> fix it in place on the new server or accept that everything is lost.
>
>
> On Sun, Aug 22, 2021 at 7:57 PM ellie timoney <[email protected]> wrote:
>
>> Hi brian,
>>
>> It might be informative to specify what Cyrus version is on your old and
>> new servers, in case there's quirks that need highlighting.
>>
>> Is there a safe way for me to massage this into working order? (Without
>> having to transfer all the files again.)
>>
>>
>> Even if there was, would you trust it?
>>
>> Should i delete, and recreate, the mailboxes?
>>
>>
>> We usually recommend people upgrade Cyrus by replicating from the original
>> server to a new server.  This will fairly transparently take care of
>> configuration/feature differences between the two installs, unlike a file
>> system copy, because the receiving server will just store the things it
>> receives in the way it expects them to be stored.
>>
>> This isn't always an option for people (not everyone has the luxury of
>> redundant hardware), but since you're moving to a new server anyway, I
>> think this is what you should do (after deleting all the broken stuff from
>> the previous attempt).
>>
>> We have some upgrading (to 3.4) documentation here:
>> https://www.cyrusimap.org/imap/download/upgrade.html
>>
>> If you're upgrading to some older version, there's old upgrade
>> documentation at:
>> https://www.cyrusimap.org/3.2/imap/download/upgrade.html
>> https://www.cyrusimap.org/3.0/imap/download/upgrade.html
>>
>> If you're not upgrading, but just moving to a new server running the same
>> version, I'd still suggest replicating.
>>
>> You may wish to upgrade your old server in place to the latest release
>> from its current series first -- e.g. if you're running 3.0.7, first
>> upgrade to 3.0.15.  Please check all the release notes for any versions
>> you're skipping over.
>>
>> If you're currently running something older than the 2.4 series, then I
>> think -- sorry, my memory is woolly -- you'll probably need to upgrade in
>> place to 2.4 before you'll be able to replicate to something newer.
>>
>> Cheers,
>>
>> ellie
>> *Cyrus <https://cyrus.topicbox.com/latest>* / Info / see discussions
>> <https://cyrus.topicbox.com/groups/info> + participants
>> <https://cyrus.topicbox.com/groups/info/members> + delivery options
>> <https://cyrus.topicbox.com/groups/info/subscription> Permalink
>> <https://cyrus.topicbox.com/groups/info/T56fa392e69ba2632-Mff3df3f564c8863adb8f6f09>
>>



--------------------------------------------------------------------------------
Michael Menge                          Tel.: (49) 7071 / 29-70316
Universität Tübingen                   Fax.: (49) 7071 / 29-5912
Zentrum für Datenverarbeitung          mail:  
[email protected]
Wächterstraße 76
72074 Tübingen


------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/T56fa392e69ba2632-M48865f284883e91f03bb862e
Delivery options: https://cyrus.topicbox.com/groups/info/subscription