Feature: Mnesia hooks
k32 <[email protected]> Fri, 28 Jan 2022 16:26:50 +0000
| Newsgroups | gmane.comp.lang.erlang.general |
|---|---|
| Message-ID | <[email protected]> |
Hello, We've been experimenting with the ways to make large Mnesia clusters more viable. Our basic idea is to move away from full-mesh topology to a "mesh+star" topology: a small part of the cluster behaves like a regular mnesia cluster (we call it core cluster) connected in a full mesh, but the rest of the nodes are read-only; they passively and asynchronously replicate transactions from the core cluster over custom protocol, and delegate write operations to the core nodes via RPC, hence "star". So far this approach looks quite promising, we've been able to achieve decent throughput in 20+ node clusters, so this may be a possible solution to the mnesia scalability challenge. But we had to patch mnesia a bit: https://github.com/emqx/otp/pull/16/files (the initial idea to rely on the events alone proved to introduce too much overhead). Ideally, we would like to contribute this patch upstream, so we're seeking the opinion from the OTP team. Hook API may be of use for other things too. Not sure if this is the right channel, though. -- BR