Re: DETS table auto_save behaviour

On Tue, May 25, 2021 at 8:43 AM Nicolas Martyanoff <[email protected]> wrote:
> I was hoping to use DETS as a local persistent buffer in case data
> cannot be written to a remote database, but it seems impossible to
> guarantee that every entry is being sync-ed to disk.

I'm not too familiar with the internals of DETS, but basically data
goes straight to/from disk while meta-data about allocated and free
areas of the file are cached in memory. I don't know if writes are
sync or not. In our experience, DETS files are somewhat fragile, plus
they have a hard 2GB size limitation which made them extremely awkward
for our use case (large mnesia tables). That's part of the reason we
migrated most of our mnesia tables to eleveldb.

If I had to have a standalone (not mnesia) local persistent store I'd
probably go with eleveldb (or one of its spinoffs) if I needed lookups
by key, or a disk_log if I just needed a FIFO buffer. disk_log allows
you to choose how sync or async your writes are. _I_ wouldn't use
DETS.