Re: Is it possible to migrate a single mailbox from cyrus 3.4.3 to 3.6.0~beta2-1 storage?

On 3/23/22 08:49, Andy Dorman wrote:
> On 3/22/22 21:53, ellie timoney wrote:
>> On Wed, 23 Mar 2022, at 12:53 PM, Andy Dorman wrote:
>>> Hi Ellie. Something else has come up and I hope you or the Cyrus team
>>> has a good solution cause I am at a loss.
>>>
>>> We had a user with many thousands of emails (2GB+) on one of the servers
>>> that was affected by the segfault.  While I was working the issue and
>>> testing cyrus reconstruct to fix the mailboxes, one of our support
>>> people helped this very busy user by creating a new mailbox for them on
>>> the same server.
>>>
>>> So now this user has a new mailbox using uuid storage (which Cyrus sees)
>>> and their old mailbox (which Cyrus doesn't see) at
>>> /var/spool/cyrus/mail/domain/B/redacted/S/user/redacted/
>>
>> To clarify: does the new account have the same name as the original 
>> one?  Or does the new account have a new name?  In other words, could 
>> the two accounts sensibly coexist?
>>
> 
> The new account has the exact same name.
> 
>>> So is there any simple way to move a single mailbox from the old to the
>>> new uuid storage space?
>>
>> This is what the missing relocate_by_id tool is for.  You could build 
>> Cyrus from source to get it, but if you can wait a little, I think 
>> it'd be less risky to wait for an updated Debian package that includes 
>> it.  Since presumably if you're using the Debian packages, then you 
>> aren't already in a habit of building Cyrus from source.
>>
>> But a mailbox being in the old storage location is not a problem, as 
>> long as Cyrus expects it to be.  It sounds like from your previous 
>> success that reconstruct can figure out if it's in the wrong place and 
>> sort it out, so you'd only need the relocate_by_id tool for switching 
>> mailboxes to uuid storage when you're ready to.
>>
> 
> I can not find the relocate_by_id tool and we use the debian packages. 
> So I guess we will have until the debian maintainer includes it before 
> we proceed to move the other addresses.
> 
>>> I think it will be fine with the user if we remove the new address that
>>> was created yesterday so the conversion process has a clean starting
>>> point to work with.
>>
>> If the original and new accounts can coexist, then it might be enough 
>> to reconstruct the original one, like you did the other broken ones, 
>> to bring it back to life.  Then import any mail from the new into the 
>> original, and delete the new.
>>
>> If they have the same name, and can't coexist, then it's more 
>> complicated.  You _might_ be able to delete the new account (using 
>> cyradm or similar), then reconstruct to bring the original one back in 
>> its place from the data on disk (... but maybe deleting the new 
>> account will find the data from the original one and delete that too, 
>> I'm not sure -- tread carefully, take backups).  If that works, you'd 
>> probably need to also run cyr_expire before the reconstruct to make 
>> sure the new account is really deleted, so it doesn't try to 
>> reconstruct the ghost of the new account instead.  And then, hopefully 
>> you got a backup of the new account before you deleted it, so that you 
>> can import the new mail back into the original once it's alive again.
>>
>> At this point, you could theoretically use relocate_by_id to move the 
>> account to uuid storage if you wanted to (and if you had it) -- you 
>> should do so eventually, but it's not urgent.  Maybe the reconstruct 
>> will have already done that for you anyway, I'm not sure.
>>
>> Though thinking about it, if the two accounts have the same name, 
>> maybe reconstruct can just sort it all out itself.  I'm not sure if 
>> you tried already, and it didn't work, or if you weren't sure and are 
>> asking first.  You could try reconstructing with the "-n" option so 
>> that it doesn't change anything, just reports what it would have done, 
>> and see if what it wants to do seems right.  I think the "-f" option 
>> would be useful here too.  If reconstruct is able to sort it out 
>> itself, that seems safer and less hassle than juggling a 
>> delete/import/etc.
>>
>> Cheers,
>>
>> ellie
>>
> 
> Regarding your last paragraph, we have already tried reconstruct with 
> -rfG options, but I think it only looked at the new uuid storage since 
> the mailbox existed there by the time we ran it.
> 
> So I think we are going to try this: Anyone please feel free to comment 
> if you think we are doing something stupid. ;-)
> 
> 1. Archive the old mail dir using tar.
> 
> tar -zcvf jazzed.tar.gz 
> /var/spool/cyrus/mail/domain/B/wildcats.com/S/user/jazzed
> 
> 2. Use cyradm to remove the new [email protected] in the uuid storage.
> 
> 3. Check to make sure jazzed is still in the old storage space. If it is 
> missing, copy it back from the archive created in step 1.
> 
> 4. Run reconstruct in the hopes it will find the address in the old 
> storage space and reconstruct it in the new uuid storage space. This is 
> what I believe happened in the other addresses.
> 
> 
> Thanks in advance for any help/thoughts.
> 

Well steps 1 and 3 above went OK and step 2 appeared to work...possibly 
too well.

However, nothing happens when I run reconstruct on user/jazzed@wildcats 
and Cyrus doesn't see jazzed in either the old OR new storage space.

This is what I see with cyradm...

localhost.ironicdesign.com> lm
DELETED/user/jazzed/[email protected] (\HasNoChildren)
DELETED/user/jazzed/Drafts/[email protected] (\HasNoChildren)
DELETED/user/jazzed/Sent/[email protected] (\HasNoChildren)
DELETED/user/jazzed/Spam/[email protected] (\HasNoChildren)
DELETED/user/jazzed/Templates/[email protected] (\HasNoChildren)
DELETED/user/jazzed/Trash/[email protected] (\HasNoChildren)

So I am not sure what to try next. Cyrus doesn't see jazzed in either 
the old or new storage format.

For now I suppose we need to recreate his new mailbox so he can still 
access his current email while we try to figure out how to recover his 
2+GB of old email.

Too bad we can't just copy the old mail directories and files into the 
new uuid storage space and then run reconstruct.

-- 
Andy


------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/T9d294f89a3d1d260-M7491cde0f82a13c4864b9411
Delivery options: https://cyrus.topicbox.com/groups/info/subscription