Re: visualize build parallelism, duration of build steps?

I like this idea. I've previously worked on measuring parallelism at a more
macro level (compiling, linking, static analysis, unit testing, integration
testing, etc) with a floating bar graph visualization.generated in Python.
I am not sure how best to make it work with SCons and potentially tens of
thousands (or more) individual items, but I'm happy to be a part of that
conversation.

I've had a suspicion for a few years that there might be a task scheduling
bug where at certain points a super task seems to come along and stop any
other tasks from being scheduled until it is done. I have no
reproducibility instructions other than to stare at your build using
process explorer / htop / your tool of choice and to watch. It would be
great to have a database that could provide evidence.

-Brian



On Tue, Oct 19, 2021, 06:13 Stijn De Ruyck <[email protected]> wrote:

> FWIW, during my own testing, I noticed a performance benefit for GCC
> compilations up to the number of physical cores +2. So for a hyperthreaded
> 12 core system with enough memory available, I would see gains up to -j14.
> The other 10 virtual cores wouldn’t help much. Anything higher than -j14
> would not result in decreased compilation time, only increased memory
> consumption. You also need to take into account memory & disk bandwidth
> bottlenecks when using high parallelism on only a single system.
>
>
>
> I’m not aware of turn-keyish integration for distributed builds, but we
> have used DistCC with ease in the past, there’s not much setup involved,
> and now use Scons with LSF instead.
>
>
>
> Stijn
>
>
>
> *From:* Scons-users <[email protected]> * On Behalf Of *Gabe
> Black
> *Sent:* Tuesday, October 19, 2021 11:56
> *To:* SCons users mailing list <[email protected]>
> *Subject:* Re: [Scons-users] visualize build parallelism, duration of
> build steps?
>
>
>
> [External Email]: This email arrived from an external source - Please
> exercise caution when opening any attachments or clicking on links.
>
> To follow up with some measurements, I found that my build performance
> scales logarithmically with -j, up to the number of threads in my system. I
> have a 12 core CPU with 2 threads per core, so that is up to 24 threads.
> After that point, performance stops improving and actually degrades
> slightly. By logarithmically, I mean that the graph of the log of the -j
> value tracks the decrease in run time of a clean build reasonably well. Or
> in other words, I would expect going from -j4 to -j16 to improve build time
> by a factor of 2x, and not a factor of 4x.
>
>
>
> Is this expected? Is there a way to diagnose what keeping our build from
> scaling roughly linearly? We have many individual object files which should
> be buildable in parallel, so I'm not sure what the bottleneck is. There
> curve seems to be fairly smooth and not a hard drop off, so I don't think
> we're hitting a specific limit in memory bandwidth or capacity, overflowing
> the disk cache, etc.
>
>
>
> Gabe
>
>
>
> On Mon, Oct 18, 2021 at 3:45 PM Gabe Black <[email protected]> wrote:
>
> Hi folks, sorry if this is a really obvious question, but is there a
> command line flag or tool or something to visualize how parallel a SCons
> build is, if there are any bottlenecks, if there are abnormally long
> running build steps, etc?
>
>
>
> I don't have an immediate pressing need for a tool like that, but I think
> it might be helpful identifying places our build could be tightened up. It
> could and also help get a rough idea of how well the build would scale to a
> very parallel build machine, ie. what might build time be with 32 cores? 64
> cores? 128 cores? etc. How much memory would you need for 128 cores to
> avoid memory bottlenecks?
>
>
>
> Somewhat on that topic, are there any relatively easy to set up
> distributed build mechanisms for SCons? I'm sure you could plug distcc into
> it for instance, but is there anything which has some nice, turn key(ish?)
> integration?
>
>
>
> Thanks!
>
> Gabe
>
> _______________________________________________
> Scons-users mailing list
> [email protected]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users