Re: Not able to make full use of CPUs for processing speed
Don Baldwin <[email protected]>
| Newsgroups | gmane.comp.programming.tools.scons.user |
|---|---|
| Message-ID | <BY5PR02MB613190B7123072458D820446F5DC9@BY5PR02MB6131.namprd02.prod.outlook.com> |
Our value of N is much less than 100. We’ll look into the MongoDB solution. Thanks. -Don From: Scons-users <[email protected]> On Behalf Of Bill Deegan Sent: Wednesday, September 15, 2021 4:09 PM To: SCons users mailing list <[email protected]> Subject: Re: [Scons-users] Not able to make full use of CPUs for processing speed WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros. Don, How big is your N? For reasonable N's SCons can keep up, but for N's in the 100s it' has some issues. The folks a MongoDB are using icecc and have built with N's > 250.. and SCons has issues. They've floated a PR to address a shortcoming in the scheduler which helped their build, but it's still marked a WIP so not merged. see: https://github.com/SCons/scons/pull/3386 You could try making those changes locally and see if that helps. -Bill On Wed, Sep 15, 2021 at 3:45 PM Don Baldwin <[email protected]<mailto:[email protected]>> wrote: Hi, We have a build machine with, say, N CPUs, and we try increasing num_jobs to a higher value, but we’re not seeing a decrease in build times. For example, say we have 10 CPUs. We would expect that as num_jobs approaches 10, we would continue to see better and better performance during the compile phase. However, what we’re seeing is that performance improves until num_jobs hits about 5, at which point further increases to num_jobs isn’t having an effect. My suspicion is that SCons is doing a lot of processing on it’s own trying to figure out which jobs to queue next, e.g. scanning .c files for new implicit dependencies, and that there’s a theoretical limit on how many CPUs it’s able to keep up with. Could something like that be going on? Thanks, Don _______________________________________________ Scons-users mailing list [email protected]<mailto:[email protected]> https://pairlist4.pair.net/mailman/listinfo/scons-users _______________________________________________ Scons-users mailing list [email protected] https://pairlist4.pair.net/mailman/listinfo/scons-users