Re: Feedback requested on an external lockfile module...

> All this business about safely writing pickles got me to wondering
> about locking files.  Rather than implement something which was
> SpamBayes-specific, I scouted around a little bit then implemented
> something which hopefully
> 
>   * has a reasonable API (I think)
>   * is cross-platform
>   * allows multiple different implementations (currently one each based
> on
>     os.link and os.mkdir and one which uses a SQLite database to
> maintain
>     lock info)
>   * might someday be good enough for inclusion in Python proper as the
> One
>     True Way to do advisory file locking.
> 
> The result is the lockfile module, which you can find at your local
> neighborhood (virtual) cheese shop:
> 
>     http://pypi.python.org/pypi/lockfile/
> 
> I've done a fair amount of testing on my Mac.  (It has a fair number of
> doctests.  Thank you again Uncle Timmy.)  Scott Dial (from the python-
> dev mailing list) ran it through its paces on Windows a bit and sent me
> some revisions.  Other than Scott's inputs I've received no direct
> feedback on the module, though when I first raised the idea of one-
> lock-mechanism-to- rule-them-all it generated a few comments.  I
> suspect Py3k and an impending
> 2.5.2 release are probably sucking up all available Python developer
> round tuits.
> 
> Today I added lockfile usage into Dave Abrahams' safe_pickle function,
> in the process renaming it to safe_pickle_write and adding a
> corresponding safe_pickle_read.  I then replaced all pickle.{dump,load}
> calls with the relevant safe_pickle_{write,read} functions.
> 
> This code has not been checked in yet.  I've attached a diff against
> the current rev 3168.  I'm interested in how people think I should
> proceed.
> Some possibilities:
> 
>     * throw it out - we don't need dependencies on external crap
>     * keep it but pull lockfile into the spambayes package for ease of
>       distribution
>     * check the changes in on head
>     * create a branch and check the changes in there
>     * get 1.1 out the door, damn it!  leave this for 1.2 (around 2011)
>     * keep the safe_pickle_{read,write} functions but take out the
> lockfile
>       stuff.

IMO, the cost of not having locks is greater than the cost of managing such
a module.  I've no real opinion on how to best manage the code itself, but
I'm personally willing to jump through a few small hoops (eg, downloading an
external package) to help make this work.  On the other hand, expecting
*all* users running from source-code (ie, most of the non-Windows world) to
download extra packages is error prone - so I'd be happy with CVS users
needing to install an external module, but for the source distribution to
include it (assuming that isn't too much of a PITA)

Cheers,

Mark