Re: apr_dbm and concurrency

On Mon, Sep 25, 2023 at 05:37:00PM +0200, Branko Čibej wrote:
> IIRC, Berkeley DB multi-process concurrency is managed through an on-disk
> "register" file external to the actual key/value store. The key/value store
> format is not affected by the presence of this file. The DB_REGISTER
> mechanism was introduced in BDB 4.4 (now long defunct) and can be used for
> both concurrency control and automatic database recovery. The client-side
> code for this can be lifted from Subversion.
> 
> (I was involved in designing this mechanism for BDB and implementing its use
> in Subversion, but that was ages ago -- back in 2005. There may be better
> ways do do this in newer versions of Berkeley DB).
> 
> TL;DR: all upstream supported versions of BDB should have this mechanism
> available and APR can detect if it's being used without changing the API,
> and even "upgrade" existing databases with the register file on the fly
> without affecting the actual database.

Thanks a lot for the response Branko, this is helpful.

I spent a lot more time playing with this and I can get apr_dbm to use a 
DB_ENV in the way you suggest with locking. However, the path handling 
becomes very complicated/impossible - with a DB_ENV the database is 
assumed to be inside the "db_home" directory but we want to load/save a 
database by absolute path name.

Using BDB in this way also appears incompatible with apr_dbm since the 
_usednames API assumes that at most two state files are used and with a 
full BDB environment the state is a whole directory. (mod_dav relies on 
this feature working correctly)

So I don't want to appear like I'm shrugging this off but *at best* it 
might be possible to fix BDB for some versions (possibly, in theory, 
with compat concerns), and that leaves older BDB releases plus NDBM 
still broken.

As an httpd developer I really don't care about database internals, I 
just want this to work. I also know that Berkeley DB is deprecated and 
will be removed from my Linux distribution of choice, so I don't want to 
invest too much time in it. :(

Regards, Joe