Re: Build profile results - Guess the Top 3 time sinks!

Hi Kai,

thanks for analyzing the build process ... your results basically 
reflect what I expected ;-).

I have not yet taken a look at Jam, but I am playing with build 
processes (non-)recursive makes etc. by my own for a while (some years). 
I created an experimental build system, which basically works, if I 
understand correctly what you said, like Jam. This build system is based 
on GNU make and reads in all dependencies into _one_ process, basically 
'stat'ing every file once only and generating one dependency tree. It 
actually works like charm :-) for my home projects (several hundreds to 
a thousand files), admitting that I have not done any scaling tests or 
some such, though I am sure that this is the right approach.

Despite improving the build performance, another advantage of having a 
single process building would be to take inter module dependencies into 
account while building, which currently are just ignored.

Anyway, if you need some help, drop me note.

Kay


Kai Backman wrote:
>  Hi everyone,
>  
>  I put up some initial build profile results on the Wiki. The top
> three time sinks are: 33% C/C++ compile, 27% dmake/build.pl,
> 22% dependencies (dpcc). So about 50% of our time is spent
> on build or build related activities. Build time on a fast 4GB
> machine is 17h. More details here:
> http://wiki.services.openoffice.org/wiki/BuildSpeedup 
> <http://www.google.com/url?sa=D&q=http%3A%2F%2Fwiki.services.openoffice.org%2Fwiki%2FBuildSpeedup>
>  
>  Looking at the available data the build times can probably be
> decreased by a factor of 4 without source code changes. With small
> source code changes the improvement could be more significant.
>  
>  I'm currently doing some Jam based build tests in transex3, and
> I'll tackle other parts next. Jam is a build tool written by Christopher
> Seiwald and released by Perforce. They use it internally to build all
> Perforce platform versions and it is used by other FOSS projects like
> Boost, Freetype and Haiku. There is an old but interesting article on
> transitioning from make to Jam here:
> http://www.perforce.com/jam/doc/scm7.html 
> <http://www.google.com/url?sa=D&q=http%3A%2F%2Fwww.perforce.com%2Fjam%2Fdoc%2Fscm7.html> 
> 
>  
>  Jam works by doing a single start up pass where it reads all the build
> files and stats all sources and targets. Once the dependency graph is
> ready the requested number of processes is spawned in parallel.
> The source code is 12k lines of C (+2k lines in a bison parser).
>  
>  Another alternative would be to change the build system to use
> non-recursive dmake. The effort is probably at par with rewriting the build
> system from scratch in jam. The build.pl/dmake source is 39k lines of C
> and Perl. My very rough estimate for the available speedup would be 1.5-3x.
>  
>  In any case, IMHO the best strategy would be to roll out an improved
> build without replacing the existing build.pl/dmake system. Developers
> can then switch once the new build system performs measurably better.
>  
>  Ideas and suggestions most welcome. :-)
>  
>  Take care,
>  
>  Kai
> 
> -- 
> Kai Backman, Software Engineer, [email protected] <mailto:[email protected]>