Re: DETS table auto_save behaviour

It's always tricky with open files during some abrupt crashes. OS-level
file system caching means that not all written data may have been
physically written to disk.

To detect this, dets has a flag indicating whether the file was properly
closed. As I understand it, the 'auto-save' does the same thing as when the
file is closed, except the file stays open.

BR,
Ulf W

Den ons 26 maj 2021 23:10Mikael Pettersson <[email protected]> skrev:

> On Tue, May 25, 2021 at 8:43 AM Nicolas Martyanoff <[email protected]>
> wrote:
> > I was hoping to use DETS as a local persistent buffer in case data
> > cannot be written to a remote database, but it seems impossible to
> > guarantee that every entry is being sync-ed to disk.
>
> I'm not too familiar with the internals of DETS, but basically data
> goes straight to/from disk while meta-data about allocated and free
> areas of the file are cached in memory. I don't know if writes are
> sync or not. In our experience, DETS files are somewhat fragile, plus
> they have a hard 2GB size limitation which made them extremely awkward
> for our use case (large mnesia tables). That's part of the reason we
> migrated most of our mnesia tables to eleveldb.
>
> If I had to have a standalone (not mnesia) local persistent store I'd
> probably go with eleveldb (or one of its spinoffs) if I needed lookups
> by key, or a disk_log if I just needed a FIFO buffer. disk_log allows
> you to choose how sync or async your writes are. _I_ wouldn't use
> DETS.
>