Re: lockfilebootstrapper clears flags even if it causes build to abort

On 20 November 2011 12:09, Bill Somerville
<[email protected]>wrote:

> Hi Julian,****
>
> ** **
>
> I am refactoring a build system that builds many projects that share a
> codebase. The current implementation uses many copies of the same workspace
> (Perforce SCM) and currently builds them sequentially. I want to reduce it
> to all builds running concurrently (at least as far as one per available
> CPU) and all building from the same workarea.
>

*nod* - IMHO I think you're going against the grain here. What I would do
is focus on seperate work areas to ensure that there was no collision. If
you're concerned about duplication there are strategies to reduce this - I
used to be fond of generating project-specific config files so that they
were consistent.

I hear what you're saying, but there's unlikely to be anybody scratching
this itch, unless it's you :)

Best

J.


> ****
>
> ** **
>
> I have this working nicely with a much simpler CC config.xml file by using
> a new “dummy” project that is responsible for sync’ing the workarea and it
> uses lockfiles (one for each real project that shares the workarea) to
> ensure that it only runs when no other project is using the workarea. The
> individual projects have a veto if they are out of date /w respect to the
> sync project so they are using the buildstatus files already. Using
> pre-registration to create a “template” project makes this a very simple
> and clean config file with trivially small project definitions.****
>
> ** **
>
> As an aside, it would seem to be less error prone to reduce the
> lockfilebootstrapper/lockfilelistener to a simple RAII type object as a
> bootstrapper that automatically released the lock (an implicit
> lockfilelistener with the same ‘projectname’ as the bootstrapper) when the
> project goes idle. I can’t think of a single safe use case where releasing
> the lock is not required when the project goes idle.****
>
> ** **
>
> The problem I am having is that when I start CC the sync project can build
> first, it acquires all the locks while it runs and the other projects get
> vetoed which would be fine if their initial force build status didn’t get
> consumed during the veto.****
>
> ** **
>
> I believe this isn’t just a startup issue, if I add a new project while CC
> is running, it may happen that its first run happens to coincide with the
> sync project building and its initial forced build (from having no .ser
> file) is consumed leaving the project waiting for a change that may not
> happen for a long time.****
>
> ** **
>
> This seems to be a classic MT race condition.****
>
> ** **
>
> It seems surprising to me that bootstrappers can cause exceptions that
> actually represent unexceptional behaviour. Maybe there should be a
> mechanism for a bootstrapper to veto a build completely – perhaps running
> the bootstrappers earlier in the process and catching exceptions that
> represent expected failures to bootstrap (like the lockfile bootstrapper
> failing what is merely a “try lock” on the lockfile) then returning a no-go
> status to the scheduler for that project timeslot rather than having the
> temporary failure to acquire a lock change the project status.****
>
> ** **
>
> HTH****
>
> Bill.****
>
> ** **
>
> *From:* Julian Simpson [mailto:[email protected]]
> *Sent:* 20 November 2011 09:52
> *To:* [email protected]
> *Subject:* Re: [Cruisecontrol-user] lockfilebootstrapper clears flags
> even if it causes build to abort****
>
> ** **
>
> ** **
>
> On 20 November 2011 00:09, Bill Somerville <
> [email protected]> wrote:****
>
> Hi,****
>
>  ****
>
> I am having problems getting CC to run builds reliably when I use lock
> files to express dependencies.****
>
>
> If project depend on each other, have you considered using this:
> http://cruisecontrol.sourceforge.net/main/configxml.html#buildstatus?
> IMHO you should use the lockfiles only to prevent builds from conflicting
> (if they use the same port numbers, etc)
>
> J. ****
>
>  ****
>
> If a project has never been built (no .ser file) or is set to build after
> failure and uses a lock file bootstrapper to serialize /w respect to other
> projects I get a race condition where the forced build flag etc  gets
> consumed by the first build attempt even if the bootstrapper exception gets
> thrown aborting the build. This means that a project may never build until
> a modification happens which is undesirable as it defeats the object of
> these flags.****
>
>  ****
>
> TIA****
>
> Bill.****
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Cruisecontrol-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user****
>
>
>
>
> -- ****
>
> Julian Simpson****
>
> The Build Doctor Ltd.****
>
> http://www.build-doctor.com****
>
> [email protected]****
>
> (+44) 207 183 0323****
>
> ** **
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Cruisecontrol-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user
>
>


-- 
Julian Simpson
The Build Doctor Ltd.
http://www.build-doctor.com
[email protected]
(+44) 207 183 0323

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d

_______________________________________________
Cruisecontrol-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user