Re: apr_dbm and concurrency

Personally, I'd deprecate apr_dbm entirely, and direct all clients towards
sqlite.

The DBMs were intended for simple key/value stores 20 years ago. We have
much better tech now. Sqlite does K/V so much better, *and* it
handles processes/threads. I really don't see a reason for DBM nowadays.

Who can justify using (say:) ldbm over sqlite?

Cheers,
-g


On Thu, Nov 9, 2023 at 6:00 AM Joe Orton <[email protected]> wrote:

> On Mon, Sep 25, 2023 at 03:58:18PM +0100, Joe Orton wrote:
> > It is unspecified whether the apr_dbm.h interface is safe to use for
> > multiple processes/threads having r/w access to a single database.
> > Results appear to be:
> >
> > - sdbm, gdbm are safe
> > - bdb, ndbm are not safe
>
> I developed a better test case (event MPM + mod_dav + apr_dbm with
> parallel clients doing cycles of LOCK/UNLOCK)
>
> sdbm, gdbm and bdb all failed.
>
> gdbm and sdbm both use either fcntl- or flock-based locking, which
> doesn't work between threads. (apr_file_lock for sdbm)
>
> Unfortunately lmdb is also unsafe at least because *opening* database
> files has races. This is documented under the Caveats section of
> http://www.lmdb.tech/doc/ (which also says it relies on flock-based
> locking, though I only hit issues with it failing during open in
> testing)
>
> The only ways forward I see are either to give up (declare that apr_dbm
> is not safe to use from threads) which passes the buck to httpd, or
> implement a locking layer inside apr_dbm() and don't assume the
> underlying drivers do thread-safe locking at all.
>
>