Re: Mnesia files

On Wed, Apr 14, 2021 at 11:14 PM Frank Muller <[email protected]>
wrote:

> Many thanks Mikael. Each file role is clear now.
>
> But I still don’t get how the system recover from a failure. What Mnesia
> does in that case?
> I suppose it will check the transaction logs first.
> Please elaborate.
>
>
To be precise: we want the database to have ACID properties around
transactions. You can drill this even more down and ask what kind of
consistency level you want, see e.g., Kyle Kingsbury:
https://jepsen.io/consistency - so ACID is a bit hand-wavy but it'll do.

If we lose power in the middle of a transaction, we want the database to
come back either with the transaction comitted or not. We don't want a
halfway transaction. We could achieve this by writing all our intentions
down in a log, together with commit points. When the system then boots
again, we replay the log from the genesis point up until the last good
transaction commit point. Everything after that point, we discard and
regard as not having happened.

However, that log might grow very large. So we want a way to circumvent
indefinite growth. We can do this by keeping a snapshot of our tables on
disk. Then we only need to keep a global log from the last snapshot point
and forward rather than having to keep the full log available for replay.
But taking snapshots is a relatively expensive operation too. So we keep a
"local" log for each table. The current state is the snapshot + a replay of
the local log. New transactions enter the global log first, and are then
flushed out to the local logs. The next dump folds the local log into a new
snapshot, so we can delete the local log and create a new local log for
future changes.

Getting this to work is quite complicated, and it isn't made simpler by the
fact that mnesia can also run in a distributed mode, where data is
replicated among several nodes, and you have to handle several
impossibility results from distributed computing.