Re: dbmail + geo-replication
Andrea Brancatelli <[email protected]> Mon, 24 Sep 2018 10:11:48 +0200
| Newsgroups | gmane.mail.imap.dbmail |
|---|---|
| Organization | Schema31 s.r.l. |
| Message-ID | <[email protected]> |
--===============0471829211== Content-Type: multipart/alternative; boundary="=_3deb93589623cdf3b063c7d7cf67caac" --=_3deb93589623cdf3b063c7d7cf67caac Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII While it's true that dbmail didn't receive any commit for new code later, emails are a pretty old and not moving target. In general there a couple of open points and a couple of bugs here and there that would need fixing, but frankly I don't feel worried about using it. Personally I'd love to see the product restart and I'd love to personally do it, but time is very low. --- Andrea Brancatelli On 2018-09-22 21:45, Ryan Beethe wrote: > Thank you, that is all really useful information. > > The more I work through my configuration the more I think my best option > at this time is going to be to put a cloud-based HAProxy in front of my > two servers so if one goes down I can easily just switch to the other. > It is good to hear that you have had a positive experience with the > multi-master syncing. It sounds like a solid solution, and in my case a > simple one because for now I won't even need IMAP from either server. > > The other thing that worries me is that dbmail is going to fall into a > state of disrepair without any developers backing it (as was recently > mentioned on this list). What do other dbmail users say about that? > Are you thinking of switching? > > Ryan > > On Sat, Sep 22, 2018 at 03:14:50PM +0200, Andrea Brancatelli wrote: > >> We've been having a MySQL multi master replication with three geographically >> distant server for years without any particular issue. >> >> You don't need the galera-specific "distributed acknowledge" stuff that, as you >> guessed, slows down every insert. >> >> Basically everything in dbmail's database has a native unique primary key (auto >> incrementing), PLUS an hash column that is used for deduplication. That means >> that the only really clumsy problem, even in case of a split brain situation is >> that if the very same email arrives in two different location while the >> database is not in sync, is that the mail doesn't get deduped. Definitely not a >> big issue. >> >> One tricky parts, on other hand, is the IMAP side as you have to make sure that >> any IMAP session goes to the same Backend or, always in case of a split brain >> situation, the user may see mail appearing and disappearing if two different >> IMAP session go to two different, unaligned, sites. >> >> But there are very easy workaround for that too - here we usually NGINX as a >> frontend IMAP and we have a script that does some hashing of the login name to >> map it to a specific geographic location - unless that location is down. >> >> If you need some more details in all of this just ask. >> >> --- >> >> Andrea Brancatelli >> >> On 2018-09-21 05:26, Ryan Beethe wrote: >> >> I am currently operating a small mail server (postfix + dovecot) but I >> have experienced a couple of ISP issues recently and would like to >> improve the availability of my server by operating a second server at a >> separate physical site (the sites I have available are separated by a >> 60ms ping). >> >> I am new to the world of high availability, but given my resources and >> constraints, an "active/passive" configuration seems to be my best >> option, where I have a primary server that is up most of the time and I >> can use pacemaker to switch from one to the other for failovers. >> >> I think I could get away with not using a distributed file system if I >> were to switch from dovecot to dbmail. I definitely need the >> distributed database, so avoid also using a distributed file system >> seems like a way to keep my architecture as simple as possible (but >> please correct me if I am wrong). >> >> For doing geo-replication, I have read that galera supports >> georeplication. But my gut sense is that galera's synchronous protocol >> in combination with my ping time would make anything that needed to >> write to the database a lot (like dbmail) prohibitively slow. Does >> anybody know if that is the case or not? >> >> Also, I read that auto_increment columns with the galera plugin are >> guaranteed to be unique, but not guaranteed to be sequential. Would >> that break dbmail? >> >> My other option would be a master-master synchronization between the two >> databases. I know people on this mailing list have been doing that for >> a while, because I read about that setup on this mailing list as far >> back as 2006. But (noob question alert), since the master-master >> database sync is asynchronous, doesn't that run the risk of data loss or >> corruption from the very latest data if a server crashes? Has that ever >> happened to any of you with dbmail, and what is the recover process >> like? >> >> Ryan >> _______________________________________________ >> DBmail mailing list >> [email protected] >> http://lists.nfg.nl/mailman/listinfo/dbmail --=_3deb93589623cdf3b063c7d7cf67caac Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= =3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: "Trebuchet = MS",Geneva,sans-serif'> <p>While it's true that dbmail didn't receive any commit for new code later= , emails are a pretty old and not moving target.</p> <p>In general there a couple of open points and a couple of bugs here and t= here that would need fixing, but frankly I don't feel worried about using i= t.</p> <p>Personally I'd love to see the product restart and I'd love to personall= y do it, but time is very low.</p> <div>---<br /> <pre><strong>Andrea Brancatelli <span style=3D"color: #ff0000;"></span></strong></pre> </div> <p>On 2018-09-22 21:45, Ryan Beethe wrote:</p> <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ig= nored --> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= Thank you, that is all really useful information.<br /> <br /> The more I w= ork through my configuration the more I think my best option<br /> at this = time is going to be to put a cloud-based HAProxy in front of my<br /> two s= ervers so if one goes down I can easily just switch to the other.<br /> It = is good to hear that you have had a positive experience with the<br /> mult= i-master syncing. It sounds like a solid solution, and in my case a<b= r /> simple one because for now I won't even need IMAP from either server= =2E<br /> <br /> The other thing that worries me is that dbmail is going to= fall into a<br /> state of disrepair without any developers backing it (as= was recently<br /> mentioned on this list). What do other dbmail use= rs say about that?<br /> Are you thinking of switching?<br /> <br /> Ryan<b= r /> <br /> On Sat, Sep 22, 2018 at 03:14:50PM +0200, Andrea Brancatelli wr= ote: <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0">We've been having a MySQL multi master replication wit= h three geographically<br /> distant server for years without any particula= r issue.<br /> <br /> You don't need the galera-specific "distributed ackno= wledge" stuff that, as you<br /> guessed, slows down every insert.<br /> <b= r /> Basically everything in dbmail's database has a native unique primary = key (auto<br /> incrementing), PLUS an hash column that is used for dedupli= cation. That means<br /> that the only really clumsy problem, even in case = of a split brain situation is<br /> that if the very same email arrives in = two different location while the<br /> database is not in sync, is that the= mail doesn't get deduped. Definitely not a<br /> big issue.<br /> <br /> O= ne tricky parts, on other hand, is the IMAP side as you have to make sure t= hat<br /> any IMAP session goes to the same Backend or, always in case of a= split brain<br /> situation, the user may see mail appearing and disappear= ing if two different<br /> IMAP session go to two different, unaligned, sit= es.<br /> <br /> But there are very easy workaround for that too - here we = usually NGINX as a<br /> frontend IMAP and we have a script that does some = hashing of the login name to<br /> map it to a specific geographic location= - unless that location is down.<br /> <br /> If you need some more details= in all of this just ask.<br /> <br /> ---<br /> <br /> Andrea Brancatelli<= br /> <br /> <br /> <br /> On 2018-09-21 05:26, Ryan Beethe wrote:<br /> <b= r /> I am currently operating a small mail server (= postfix + dovecot) but I<br /> have experienced a c= ouple of ISP issues recently and would like to<br /> &nbs= p;improve the availability of my server by operating a second server at a<b= r /> separate physical site (the sites I have avail= able are separated by a<br /> 60ms ping).<br /> <br= /> I am new to the world of high availability, but= given my resources and<br /> constraints, an "acti= ve/passive" configuration seems to be my best<br />  = ;option, where I have a primary server that is up most of the time and I<br= /> can use pacemaker to switch from one to the oth= er for failovers.<br /> <br /> I think I could get = away with not using a distributed file system if I<br /> = were to switch from dovecot to dbmail. I definitely need the<br= /> distributed database, so avoid also using a dis= tributed file system<br /> seems like a way to keep= my architecture as simple as possible (but<br /> p= lease correct me if I am wrong).<br /> <br /> For d= oing geo-replication, I have read that galera supports<br /> &n= bsp; georeplication. But my gut sense is that galera's synchrono= us protocol<br /> in combination with my ping time = would make anything that needed to<br /> write to t= he database a lot (like dbmail) prohibitively slow. Does<br /> = anybody know if that is the case or not?<br /> <br /> &nb= sp; Also, I read that auto_increment columns with the gale= ra plugin are<br /> guaranteed to be unique, but no= t guaranteed to be sequential. Would<br /> th= at break dbmail?<br /> <br /> My other option would= be a master-master synchronization between the two<br />  = ; databases. I know people on this mailing list have been doing = that for<br /> a while, because I read about that s= etup on this mailing list as far<br /> back as 2006= =2E But (noob question alert), since the master-master<br /> &n= bsp; database sync is asynchronous, doesn't that run the risk of= data loss or<br /> corruption from the very latest= data if a server crashes? Has that ever<br /> &nbs= p;happened to any of you with dbmail, and what is the recover process<br />= like?<br /> <br /> Ryan<br= /> _______________________________________________= <br /> DBmail mailing list<br /> = <a href=3D"mailto:[email protected]">[email protected]</a><br /> &nbs= p; <a href=3D"http://lists.nfg.nl/mailman/listinfo/dbmail"= target=3D"_blank" rel=3D"noopener noreferrer">http://lists.nfg.nl/mailman/= listinfo/dbmail</a><br /> </blockquote> </div> </blockquote> </body></html> --=_3deb93589623cdf3b063c7d7cf67caac-- --===============0471829211== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ DBmail mailing list [email protected] http://lists.nfg.nl/mailman/listinfo/dbmail --===============0471829211==--