Re: Upgrading James, how to merge maildirs

Marc Chamberlin <[email protected]> Mon, 24 Jan 2022 10:40:47 -0800
Newsgroups	gmane.comp.jakarta.james.user
Message-ID	<[email protected]>
Hi Benoit, and thank you for your response. Looking though some of the 
past postings on this mail list it appears top posting is the norm so I 
will follow suit. I have done a brief survey of the technologies you 
have mentioned and here are my questions back and responses to some of 
your observations.

1. About the Maildir implementation, I was unaware that it is being 
deprecated. There is no mention of that on the James website as far as I 
can see. Nor does the code itself inform me that MailDir is being 
deprecated when I configure James to use it. So in light of this, my 
expectation as a user was that MailDir was being supported and 
maintained. Sorry I don't read every thread on this email list so I 
missed this discussion about MailDir. I will miss MailDir, felt it was 
reliable and popular, and didn't rely on some external tool, such as a 
database server. I have found that database servers are unreliable, do 
not follow the KISS principal, and often difficult to migrate from one 
OS release to the next. But since MailDir is being deprecated, I will 
look into using the JPA/MySQL back end. (since I already use MySQL for 
other purposes, I don't want to have to learn and grok some other 
database server such as Derby)

2. While the IMAPSync tool you mentioned does seem like a possible way 
to do migrations, I have a few questions -

     Doesn't this method require two instances of the James email server 
to be running? Seems like that might be difficult to do on a single 
host. If I set up James on a different server, so that I can run two 
instances of James at the same time, it seems like I will have to move 
my database twice, once to a different system in  either the same 
database configuration, or into the new database configuration, then 
back  to the original James server with an internet facing port. Then I 
will have to repeat this process with the other version of James which 
has a database that has diverged. I don't  think this will be an easy "3 
minute process" as you mentioned!

And that brings up another concern. will the IMAPSync process handle the 
merge of my two diverged databases correctly? Will this process detect 
and not transfer identical emails from one server to the other, or will 
I have a lot of double copies of emails in my new instance of James? If 
so, I and my users will end up with an enormous number of duplicate 
emails that will have to be deleted, by hand! YUCK!!!

I will look forward to your reply and suggestions, but at this point I 
am uneasy about following your suggestion to use IMAPSync, as a means to 
accomplish the merge of my two variants of email databases.  Thanks 
again in advance!

    Marc

On 1/23/22 21:26, Benoit TELLIER wrote:
> Hello Marc,
>
> First and foremost, Maildir implementation is long unmaintained, and
> prove to be vulnerable [1].
>
> We put great efforts to provide smooth migrations for all mailbox
> implementation. According to your sayings we failed to do so. If this
> was needed this w=shows the lack of attention the MailDir implementation
> did receive.
>
> We discussed a few month before to drop MailDir implementation as part
> of 3.7.0 release (Maildir would still be supported on 3.6.x branch). [2]
> I know the MailDir word is appealing as it is the de-facto
> implementation in most email servers and on-filesystem yet it was buggy,
> brittle, unmaintained, vulnerable, and thus did not keep it up with the
> promise in its name.
>
> SOHO/HomeOperator are invited to switch to the 0-dependency alternative
> which is the JPA implementation with an embedded database like derby. I
> recommend a clean IMAPSync to do the MailDir-JPA implementation, [3]
> which can be done in a matter of minutes.
>
> Might you encounter issues on this journey I would be happy to help.
>
> [1] http://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-40525
> [2] https://www.mail-archive.com/[email protected]/msg70909.html
> [3]
> https://github.com/apache/james-project/blob/master/server/apps/distributed-app/docs/modules/ROOT/pages/operate/migrating.adoc
> (also applies for JPA <-> MailDir migration)
>
> Best regards,
>
> Benoit TELLIER
>
> On 23/01/2022 06:49, Marc Chamberlin wrote:
>> Hello,  I am a small SOHO/Home operator of a James server, running it
>> on OpenSuSE 15.2 and am in the midst of trying to upgrade it from a
>> snapshot version of 3.4 to 3.6, but before jumping that far, I
>> downloaded and installed the released version of 3.4 thinking that
>> upgrading from the snapshot version to 3.4 released version would be
>> the safest first step. It hasn't gone well and I have been jumping
>> back and forth between these two versions while discovering and fixing
>> problems as they show up. (Mostly things like missing jar files or
>> wrong versions of them, and miss-configurations)  Anywise I am now
>> noticing a divergence in the maildir databases, new folders added,
>> emails in one version but not in the other etc.
>>
>> So I am wondering if there is a tool or some other means to do a merge
>> by hand. I tried/attempted some obvious things such as simply copying
>> maildir folders across from one database to the other and updating the
>> subscription lists but that alone didn't work. So I could use some
>> guidance and/or pointers to documentation that describes how James
>> models, uses, and makes changes to it's maildir database. The James
>> website does not provide much in the way of documentation nor has
>> Google been helpful.
>>
>> Thanks in advance for taking the time to help me with this issue
>> and/or provide advice.    Marc
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>


-- 

*"The Truth is out there" - Spooky*

*_   _   .   .   .       .   .   .   _ _       .   _   _   _   _   .     
   .   .   .           _   .   .       .           .   _   _       .   _ 
       _ _   .   .   .       .   _   _   .       _   .   .   _   .   _   
_           _   _       .   _       .   _   .     _   .   _   . *

Computers: the final frontier.
These are the voyages of the user Marc.
His mission: to explore strange new hardware.
To seek out new software and new applications.
To boldly go where no Marc has gone before!

(/This email is digitally signed and the electronic signature is 
attached. If you know how, you can use my public key to prove this email 
indeed came from me and has not been modified in transit. My public key, 
which can be used for sending encrypted email to me also, can be found 
at - https://keys.openpgp.org/[email protected] or just 
ask me for it and I will send it to you as an attachment. If you don't 
understand all this geek speak, no worries, just ignore this explanation 
and ignore the signature key attached to this email (it will look like 
gibberish if you open it) and/or ask me to explain it further if you like./)
OpenPGP_0xD23D75B63BF0E8B7.asc (application/pgp-keys, 2.4 KB)
OpenPGP_signature (application/pgp-signature, 665 B) - not displayed