Re: Build profile results - Guess the Top 3 time sinks!

 Hi,

 I would love to see a single makefile dmake and compare it against the
performance of  Jam based setup. It would also be great to see how
compile batching could be implemented using dmake, it's an easy
optimization that gives nice speed boosts.

 You are right about dmake not being recursive, but the combined build/dmake
behaviour is very similar to the classical recursive make. Again, it would
be great to see this reworked, but I don't really have the dmake skills to
do that,
so it would have to be you or some other build guru who hacked that part ..
:-)

 I'm looking forward to hearing from mh/blauwal/ause and the other build
experts
on this as well (they can probably give hints with milestone portability).
I'm
currently working against m155 and I'll freeze on that until I have
something
more substantial to show.

 Take care,

 Kai

On 2/16/06, Volker Quetschke <[email protected]> wrote:
>
> Hi!
>
> Kai Backman wrote:
> >  Hi everyone,
> >
> >  I put up some initial build profile results on the Wiki. The top
> > three time sinks are: 33% C/C++ compile, 27% dmake/build.pl,
> > 22% dependencies (dpcc). So about 50% of our time is spent
> > on build or build related activities. Build time on a fast 4GB
> > machine is 17h.
> Depends what you call fast, my slow AMD 1700XP, 512MB, reasonably
> fast harddisk, needs 17h to build. ( I might be off by an hour or
> so, I didn't use it lately. )
>
> > More details here:
> > http://wiki.services.openoffice.org/wiki/BuildSpeedup
> > <
> http://www.google.com/url?sa=D&q=http%3A%2F%2Fwiki.services.openoffice.org%2Fwiki%2FBuildSpeedup
> >
> >
> >  Looking at the available data the build times can probably be
> > decreased by a factor of 4 without source code changes. With small
> > source code changes the improvement could be more significant.
> >
> >  I'm currently doing some Jam based build tests in transex3, and
> > I'll tackle other parts next. Jam is a build tool written by Christopher
> > Seiwald and released by Perforce. They use it internally to build all
> > Perforce platform versions and it is used by other FOSS projects like
> > Boost, Freetype and Haiku. There is an old but interesting article on
> > transitioning from make to Jam here:
> > http://www.perforce.com/jam/doc/scm7.html
> > <
> http://www.google.com/url?sa=D&q=http%3A%2F%2Fwww.perforce.com%2Fjam%2Fdoc%2Fscm7.html
> >
> >
> >
> >  Jam works by doing a single start up pass where it reads all the build
> > files and stats all sources and targets. Once the dependency graph is
> > ready the requested number of processes is spawned in parallel.
> > The source code is 12k lines of C (+2k lines in a bison parser).
> >
> >  Another alternative would be to change the build system to use
> > non-recursive dmake. The effort is probably at par with rewriting the
> build
> > system from scratch in jam. The build.pl/dmake source is 39k lines of C
> > and Perl. My very rough estimate for the available speedup would be
> 1.5-3x.
> I have problems with the term "recursive dmake". Beside for dependency
> generation
> there are no dmake's calling dmakes. But IMHO the main issue is that
> build.pl calls dmake for each "sub-module". For example for sw/ that are
> 76
> dmake processes that are started by build.pl and therefore 76 dmakes that
> have to source the ~1 MB of makefiles in solenv/inc.
>
> That's not recursive, but still bad.
>
> If you would create a toplevel makefile for each module to take over the
> dependency handling that build.pl currently does you are more or less
> in the same situation as with jam. dmake reads the makefiles once, creates
> a dependency graph and then creates the targets according to the rules.
>
> >  In any case, IMHO the best strategy would be to roll out an improved
> > build without replacing the existing build.pl/dmake system. Developers
> > can then switch once the new build system performs measurably better.
> First of all: I like the idea, secondly: Nearly every new milestone breaks
> the build because of some changes that worked in the Hamburg environment
> but break the OOo configure based setup (aka non-official builds).
> (I'm exaggerating to emphasize the problem.)
>
> Without the support and help of the Hamburg people this will become a
> maintainance nightmare.
>
> Volker
>
> --
> If you like my work consider:  http://www.scytek.de/donations.html
> PGP/GPG key  (ID: 0x9F8A785D)  available  from  wwwkeys.de.pgp.net
> key-fingerprint 550D F17E B082 A3E9 F913  9E53 3D35 C9BA 9F8A 785D
>
>
>


--
Kai Backman, Software Engineer, [email protected]