Re: using cdb for 'update' without race condition?

[email protected] wrote:

> I'm interested in checking out cdb for a database that will be updated
> programmatically.  Since the database is 'constant', 'update' is a bit
> of a contradiction in terms, but I believe the effect of an atomic
> update for a cdb database works like this:
> 
>   1. My app reads a database and extracts all the key-value pairs.
> 
>   2. My app produces a new set of key-value pairs.

You ought to have the source file around already, which would obviate
these two steps.  You can have your program edit the source file, which
is easily machine-parseable, and regenerate it as desired.

>   3. I atomically make a new database. I cannot do this through a C
>      API but only via the cdbmake program.  Atomicity is guaranteed by
>      the way cdbmake uses the filesystem.

The way cdbmake "uses the filesystem" is, it generates a temporary file
and renames it into place using the rename(2) call, just as mv(1) does
if the target is on the same filesystem.  The operation isn't _exactly_
atomic, but the kernel guarantees that the target filename will always
resolve to either the old file's inode or the new file's inode, which is
the only part anyone generally cares about.

> It appears to me that if I want to avoid losing an update via race
> conditions that my cdb file needs to be protected by some kind of
> lock, and it's up to my app to use liblockfile or something similar.

You should never need to lock the database file.  That's the whole
point.  You might want to lock your source file while editing it.  Or
lock your output file if you must dump the database every time -- in
which case, you'd lock the output file before opening the database, and
unlock it after completing the update.  And frankly, for locking in this
case, you may as well simply use open(...,O_CREAT|O_EXCL) for the output
file, coupled with rename(2) or unlink(2)... so long as the file doesn't
reside on an nfs-mounted filesystem, in which case symlink(2) is a
little better than the aforementioned open(2) call (not to mention most
file locking alternatives).  If you'd rather use shell scripts for any
of this, you can use the 'noclobber' option, `mv` or `rm`, and `ln -s`,
respectively.

Or you can still use liblockfile if you feel like it.

ari