Re: createrepo: external locking?

On Wed, Aug 25, 2010 at 09:48, Oliver Hookins <[email protected]> wrote:
> On Wed, Aug 25, 2010 at 03:26:20PM +0200, ext seth vidal wrote:
>> On Wed, 2010-08-25 at 14:56 +0200, Oliver Hookins wrote:
>> > Hi all,
>> >
>> > I've hit an interesting problem on some of our repositories we use for
>> > populating with our own applications. We have build servers generating RPMs
>> > automatically (sometimes on every scm commit) and these are then dumped into the
>> > yum repository directories and createrepo run on the directory to generate the
>> > metadata.
>> >
>> > At the moment, this means that every job in the build system that generates RPMs
>> > must have its own directory (and now there are quite a few) as multiple
>> > createrepo instances running against the same directory fails. We can't just run
>> > it from cron periodically as there are some triggered jobs which pull down the
>> > most recent metadata immediately afterwards to do some testing of the new
>> > packages.
>> >
>> > So, is there some way to get createrepo to lock a particular directory so that
>> > multiple processes can update the metadata at least sequentially? Or is this
>> > kind of thing best done with a wrapper around createrepo (or not at all)?
>> >
>>
>> I don't see how createrepo locking the dir would help unless all other
>> apps honored the locks quasi-sanely.
>>
>> Is there a reason you cannot just make the program(s) which need to
>> access the metadata run in sequence and not in parallel?
>
> Not easily, and it's not desirable. The build process is often very long but the
> RPM generation and subsequent metadata generation is relatively short. I don't
> think we can put locks in our build system for just this part of the build,
> sadly.
>
> It seems like we might have to just keep maintaining one directory per build
> process...

You can use this, like I do when I run cron:
http://code.google.com/p/withlock/

later, Robert Xu