Re: dbmail + geo-replication
Andrea Brancatelli <[email protected]> Sat, 22 Sep 2018 15:14:50 +0200
| Newsgroups | gmane.mail.imap.dbmail |
|---|---|
| Organization | Schema31 s.r.l. |
| Message-ID | <[email protected]> |
--===============0845720658== Content-Type: multipart/alternative; boundary="=_9618218eff774aad23004b1ca45b9cf8" --=_9618218eff774aad23004b1ca45b9cf8 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII We've been having a MySQL multi master replication with three geographically distant server for years without any particular issue. You don't need the galera-specific "distributed acknowledge" stuff that, as you guessed, slows down every insert. Basically everything in dbmail's database has a native unique primary key (auto incrementing), PLUS an hash column that is used for deduplication. That means that the only really clumsy problem, even in case of a split brain situation is that if the very same email arrives in two different location while the database is not in sync, is that the mail doesn't get deduped. Definitely not a big issue. One tricky parts, on other hand, is the IMAP side as you have to make sure that any IMAP session goes to the same Backend or, always in case of a split brain situation, the user may see mail appearing and disappearing if two different IMAP session go to two different, unaligned, sites. But there are very easy workaround for that too - here we usually NGINX as a frontend IMAP and we have a script that does some hashing of the login name to map it to a specific geographic location - unless that location is down. If you need some more details in all of this just ask. --- Andrea Brancatelli On 2018-09-21 05:26, Ryan Beethe wrote: > I am currently operating a small mail server (postfix + dovecot) but I > have experienced a couple of ISP issues recently and would like to > improve the availability of my server by operating a second server at a > separate physical site (the sites I have available are separated by a > 60ms ping). > > I am new to the world of high availability, but given my resources and > constraints, an "active/passive" configuration seems to be my best > option, where I have a primary server that is up most of the time and I > can use pacemaker to switch from one to the other for failovers. > > I think I could get away with not using a distributed file system if I > were to switch from dovecot to dbmail. I definitely need the > distributed database, so avoid also using a distributed file system > seems like a way to keep my architecture as simple as possible (but > please correct me if I am wrong). > > For doing geo-replication, I have read that galera supports > georeplication. But my gut sense is that galera's synchronous protocol > in combination with my ping time would make anything that needed to > write to the database a lot (like dbmail) prohibitively slow. Does > anybody know if that is the case or not? > > Also, I read that auto_increment columns with the galera plugin are > guaranteed to be unique, but not guaranteed to be sequential. Would > that break dbmail? > > My other option would be a master-master synchronization between the two > databases. I know people on this mailing list have been doing that for > a while, because I read about that setup on this mailing list as far > back as 2006. But (noob question alert), since the master-master > database sync is asynchronous, doesn't that run the risk of data loss or > corruption from the very latest data if a server crashes? Has that ever > happened to any of you with dbmail, and what is the recover process > like? > > Ryan > _______________________________________________ > DBmail mailing list > [email protected] > http://lists.nfg.nl/mailman/listinfo/dbmail --=_9618218eff774aad23004b1ca45b9cf8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= =3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: "Trebuchet = MS",Geneva,sans-serif'> <p>We've been having a MySQL multi master replication with three geographic= ally distant server for years without any particular issue.</p> <p>You don't need the galera-specific "distributed acknowledge" stuff that,= as you guessed, slows down every insert.</p> <p>Basically everything in dbmail's database has a native unique primary ke= y (auto incrementing), PLUS an hash column that is used for deduplication= =2E That means that the only really clumsy problem, even in case of a split= brain situation is that if the very same email arrives in two different lo= cation while the database is not in sync, is that the mail doesn't get dedu= ped. Definitely not a big issue.</p> <p>One tricky parts, on other hand, is the IMAP side as you have to make su= re that any IMAP session goes to the same Backend or, always in case of a s= plit brain situation, the user may see mail appearing and disappearing if t= wo different IMAP session go to two different, unaligned, sites.</p> <p>But there are very easy workaround for that too - here we usually NGINX = as a frontend IMAP and we have a script that does some hashing of the login= name to map it to a specific geographic location - unless that location is= down.</p> <p>If you need some more details in all of this just ask.</p> <div>---<br /> <pre><strong>Andrea Brancatelli <br /></strong></pre> </div> <p><br /></p> <p>On 2018-09-21 05:26, Ryan Beethe wrote:</p> <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ig= nored --> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= I am currently operating a small mail server (postfix + dovecot) but I<br /= > have experienced a couple of ISP issues recently and would like to<br /> = improve the availability of my server by operating a second server at a<br = /> separate physical site (the sites I have available are separated by a<br= /> 60ms ping).<br /> <br /> I am new to the world of high availability, bu= t given my resources and<br /> constraints, an "active/passive" configurati= on seems to be my best<br /> option, where I have a primary server that is = up most of the time and I<br /> can use pacemaker to switch from one to the= other for failovers.<br /> <br /> I think I could get away with not using = a distributed file system if I<br /> were to switch from dovecot to dbmail= =2E I definitely need the<br /> distributed database, so avoid also u= sing a distributed file system<br /> seems like a way to keep my architectu= re as simple as possible (but<br /> please correct me if I am wrong).<br />= <br /> For doing geo-replication, I have read that galera supports<br /> g= eoreplication. But my gut sense is that galera's synchronous protocol= <br /> in combination with my ping time would make anything that needed to<= br /> write to the database a lot (like dbmail) prohibitively slow. D= oes<br /> anybody know if that is the case or not?<br /> <br /> Also, I rea= d that auto_increment columns with the galera plugin are<br /> guaranteed t= o be unique, but not guaranteed to be sequential. Would<br /> that br= eak dbmail?<br /> <br /> My other option would be a master-master synchroni= zation between the two<br /> databases. I know people on this mailing= list have been doing that for<br /> a while, because I read about that set= up on this mailing list as far<br /> back as 2006. But (noob question= alert), since the master-master<br /> database sync is asynchronous, doesn= 't that run the risk of data loss or<br /> corruption from the very latest = data if a server crashes? Has that ever<br /> happened to any of you = with dbmail, and what is the recover process<br /> like?<br /> <br /> Ryan<= br /> _______________________________________________<br /> DBmail mailing = list<br /> <a href=3D"mailto:[email protected]">[email protected]</a><br />= <a href=3D"http://lists.nfg.nl/mailman/listinfo/dbmail" target=3D"_blank" = rel=3D"noopener noreferrer">http://lists.nfg.nl/mailman/listinfo/dbmail</a>= <br /> </div> </blockquote> </body></html> --=_9618218eff774aad23004b1ca45b9cf8-- --===============0845720658== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ DBmail mailing list [email protected] http://lists.nfg.nl/mailman/listinfo/dbmail --===============0845720658==--