Re: lockfilebootstrapper clears flags even if it causes build to abort
Bill Somerville <[email protected]> Sun, 20 Nov 2011 12:37:53 +0000
| Newsgroups | gmane.comp.java.cruise-control.user |
|---|---|
| Message-ID | <[email protected]> |
Hi Julian, Thanks for the fast response. OK, I understand what you are saying and would be interested in what strategy you might suggest to reduce duplication of workspaces. In my case they are large and potentially there may be large numbers of them due to many cross compiles (code base includes C/C++) multiplied by SCM branches too. At this stage I am not able to make changes to the very complex Ant scripts, fortunately each project can be tailored to use its own build area so the shared source area seems a no-brainer simplification. The lockfilebootstrapper appears to broken too me, can you give a use case that won't suffer from the problem I have described? Regards Bill. From: Julian Simpson [mailto:[email protected]] Sent: 20 November 2011 12:22 To: [email protected] Subject: Re: [Cruisecontrol-user] lockfilebootstrapper clears flags even if it causes build to abort On 20 November 2011 12:09, Bill Somerville <[email protected]> wrote: Hi Julian, I am refactoring a build system that builds many projects that share a codebase. The current implementation uses many copies of the same workspace (Perforce SCM) and currently builds them sequentially. I want to reduce it to all builds running concurrently (at least as far as one per available CPU) and all building from the same workarea. *nod* - IMHO I think you're going against the grain here. What I would do is focus on seperate work areas to ensure that there was no collision. If you're concerned about duplication there are strategies to reduce this - I used to be fond of generating project-specific config files so that they were consistent. I hear what you're saying, but there's unlikely to be anybody scratching this itch, unless it's you :) Best J. I have this working nicely with a much simpler CC config.xml file by using a new "dummy" project that is responsible for sync'ing the workarea and it uses lockfiles (one for each real project that shares the workarea) to ensure that it only runs when no other project is using the workarea. The individual projects have a veto if they are out of date /w respect to the sync project so they are using the buildstatus files already. Using pre-registration to create a "template" project makes this a very simple and clean config file with trivially small project definitions. As an aside, it would seem to be less error prone to reduce the lockfilebootstrapper/lockfilelistener to a simple RAII type object as a bootstrapper that automatically released the lock (an implicit lockfilelistener with the same 'projectname' as the bootstrapper) when the project goes idle. I can't think of a single safe use case where releasing the lock is not required when the project goes idle. The problem I am having is that when I start CC the sync project can build first, it acquires all the locks while it runs and the other projects get vetoed which would be fine if their initial force build status didn't get consumed during the veto. I believe this isn't just a startup issue, if I add a new project while CC is running, it may happen that its first run happens to coincide with the sync project building and its initial forced build (from having no .ser file) is consumed leaving the project waiting for a change that may not happen for a long time. This seems to be a classic MT race condition. It seems surprising to me that bootstrappers can cause exceptions that actually represent unexceptional behaviour. Maybe there should be a mechanism for a bootstrapper to veto a build completely - perhaps running the bootstrappers earlier in the process and catching exceptions that represent expected failures to bootstrap (like the lockfile bootstrapper failing what is merely a "try lock" on the lockfile) then returning a no-go status to the scheduler for that project timeslot rather than having the temporary failure to acquire a lock change the project status. HTH Bill. From: Julian Simpson [mailto:[email protected]] Sent: 20 November 2011 09:52 To: [email protected] Subject: Re: [Cruisecontrol-user] lockfilebootstrapper clears flags even if it causes build to abort On 20 November 2011 00:09, Bill Somerville <[email protected]> wrote: Hi, I am having problems getting CC to run builds reliably when I use lock files to express dependencies. If project depend on each other, have you considered using this: http://cruisecontrol.sourceforge.net/main/configxml.html#buildstatus? IMHO you should use the lockfiles only to prevent builds from conflicting (if they use the same port numbers, etc) J. If a project has never been built (no .ser file) or is set to build after failure and uses a lock file bootstrapper to serialize /w respect to other projects I get a race condition where the forced build flag etc gets consumed by the first build attempt even if the bootstrapper exception gets thrown aborting the build. This means that a project may never build until a modification happens which is undesirable as it defeats the object of these flags. TIA Bill. ------------------------------------------------------------------------ ------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Cruisecontrol-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user -- Julian Simpson The Build Doctor Ltd. http://www.build-doctor.com [email protected] (+44) 207 183 0323 <tel:%28%2B44%29%20207%20183%200323> ------------------------------------------------------------------------ ------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Cruisecontrol-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user -- Julian Simpson The Build Doctor Ltd. http://www.build-doctor.com [email protected] (+44) 207 183 0323 ------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Cruisecontrol-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/cruisecontrol-user