Re: Moving from 2.4.17 (CentOS 7) to 3.0.7 (RHEL 8)

----- Message from Javier Angulo <[email protected]> ---------
     Date: Mon, 15 Mar 2021 10:19:46 +0100
     From: Javier Angulo <[email protected]>
Reply-To: Info <[email protected]>
  Subject: Re: Moving from 2.4.17 (CentOS 7) to 3.0.7 (RHEL 8)
       To: [email protected]


> On 3/14/21 3:20 PM, [email protected] wrote:
>> Third query, on 'reconstruct'.
>>
>> I scp'ed test data from a dummy user to the new server, then tried  
>> to reconstruct it, but reconstruct is not behaving as I'd expect.  
>> To get reconstruct to work it appears I have to do two steps:
>>
>> First (as per the upgrade page):
>>
>> reconstruct -V max
>>
>> But that doesn't add all of the emails... then running:
>> |reconstruct -f|
>> ...does, like this:
>>
>> [root@emp87 server]# reconstruct -V max
>> user/server
>> Repacked user/server to version 13
>> user.server.Deleted Items updating quota_mailbox_used: 77944 => 0
>> user.server.Deleted Items: updating exists 14 => 0
>> user.server.Deleted Items: updating sync_crc 1694230313 => 0
>> user/server/Deleted Items
>> Repacked user/server/Deleted Items to version 13
>> user/server/Drafts
>> Repacked user/server/Drafts to version 13
>> user.server.Sent updating quota_mailbox_used: 6274857 => 0
>> user.server.Sent: updating exists 3 => 0
>> user.server.Sent: updating sync_crc 3564848791 => 0
>> user/server/Sent
>> Repacked user/server/Sent to version 13
>> user.server.Spam updating quota_mailbox_used: 299039 => 0
>> user.server.Spam: updating exists 1 => 0
>> user.server.Spam: updating sync_crc 4077200434 => 0
>> user/server/Spam
>> Repacked user/server/Spam to version 13
>> user.server.Trash updating quota_mailbox_used: 5214 => 0
>> user.server.Trash: updating exists 2 => 0
>> user.server.Trash: updating sync_crc 1430816766 => 0
>> user/server/Trash
>> Repacked user/server/Trash to version 13
>>
>>
>> [root@emp87 server]# reconstruct -f
>> user/server
>> user.server.Deleted Items uid 1 rediscovered - appending
>> user.server.Deleted Items uid 2 rediscovered - appending
>> user.server.Deleted Items uid 3 rediscovered - appending
>> user.server.Deleted Items uid 4 rediscovered - appending
>> user.server.Deleted Items uid 5 rediscovered - appending
>> user.server.Deleted Items uid 6 rediscovered - appending
>> user.server.Deleted Items uid 7 rediscovered - appending
>> user.server.Deleted Items uid 8 rediscovered - appending
>> user.server.Deleted Items uid 9 rediscovered - appending
>> user.server.Deleted Items uid 10 rediscovered - appending
>> user.server.Deleted Items uid 11 rediscovered - appending
>> user.server.Deleted Items uid 12 rediscovered - appending
>> user.server.Deleted Items uid 13 rediscovered - appending
>> user.server.Deleted Items uid 14 rediscovered - appending
>> user/server/Deleted Items
>> user/server/Drafts
>> user.server.Sent uid 1 rediscovered - appending
>> user.server.Sent uid 2 rediscovered - appending
>> user.server.Sent uid 3 rediscovered - appending
>> user/server/Sent
>> user.server.Spam uid 1 rediscovered - appending
>> user/server/Spam
>> user.server.Trash uid 7 rediscovered - appending
>> user.server.Trash uid 8 rediscovered - appending
>> user/server/Trash
>>
>> Otherwise the emails in subfolders are not visible over IMAP.
>>
>> What is the correct process for running reconstruct from 2.4 to 3.0?
>>
>> Also, once visible it appears messages are all "Unread" flagged -  
>> can the seen flags not be brought through? These are emails from an  
>> existing 2.4 server.
>> *Cyrus <https://cyrus.topicbox.com/latest>* / Info / see  
>> discussions <https://cyrus.topicbox.com/groups/info> + participants  
>> <https://cyrus.topicbox.com/groups/info/members> + delivery options  
>> <https://cyrus.topicbox.com/groups/info/subscription> Permalink  
>> <https://cyrus.topicbox.com/groups/info/T843907ad61cf4402-Mefda5e5d6186c76f29eacd18>
>
> Hi Simon,
>
> looks like you did not import a dump of mailboxes database on  
> destination. And also did not copy user,quota,sieve folders.
>
> From my memory:
>
> a) On $origin server:
> - dump mailboxes.db to text file
> # ctl_mboxlist -d > /tmp/foo
>
> b) on $destination server:
> - copy from _$origin with permissions(assuming default paths on rhel7/8):
> /var/spool/imap/*
> /var/lib/imap/sieve/*
> /var/lib/imap/user/*
> /var/lib/imap/quota/*
> your /tmp/foo mailboxes dump file
>
> - import mailboxes
> # ctl_mboxlist -u < /tmp/foo
> (check uid/gid of /var/lib/imap/mailboxes.db after import)
>
> - run reconstruct
> # reconstruct -r -V max user/whatever
> 
> Note that if you have user names with dots (e.g. foo.bar), you will
> have to rename sieve folders from foo^bar to foo.bar
> 
> Hope it helps. I could have forgotten something, though.
> 
> Javier.
> 


----- End message from Javier Angulo <[email protected]> -----

That's great thank you... I'll give it a try. You are correct, looks  
like I missed some steps.

In the meantime I've been working to get sync_client / sync_server  
working, and am making good headway there. I had to run reconstruct a  
few times on the source, including with GUID corrections with -G and  
-R, and had to delete a couple of emails completely, but have been  
able to get a fully completed test replication of all users, so I am  
also thinking that may be easier. Does replication bring over sieve  
scripts?

I still have the following questions though:

1. How to set up squatter in rolling mode on 3.0.7.
2. Whether I need to do anything new for quota management
     Incidentally - the "quota" utility is not present on my RHEL 8  
install of Cyrus 3.0.7 - is this a known issue?
3. I have set altnamespace: 0 to match what I have on the old server -  
will that retain the old behaviour? Any issues with migration, sieve?  
Mailboxes appear to have replicated ok, including one with a dot,  
which has a folder structure of name^name.
4. Is a "shutdown file missing, updating recovery stamp" message  
normal on restart of cyrus-imapd
5. New question - after replication from 2.4 to 3.0 and once I have  
stopped replication and am running with the new one, do I need to run  
reconstructs on the 3.0 replicated mailboxes? e.g. -V max, etc.?

Simon


-- 
Simon Wilson
M: 0400 12 11 16


------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/T843907ad61cf4402-M616f3ac301536a7348b41eba
Delivery options: https://cyrus.topicbox.com/groups/info/subscription