Re: SCons Python Builder not building in parallel?

So you have a builder which is implemented in python and does non-trivial
amount of work in the builder logic itself?
If so, then yes move the logic to a script and have your builder call the
script with the appropriate arguments.

When scons is running your builders python logic the GIL comes into play.

-Bill

On Sat, Sep 22, 2018 at 1:16 PM Hua Yanghao <[email protected]> wrote:

> Dear Scons Users,
> I recently noticed that with -jN my CPU utilization is less than 100%
> (4 cores) during some phases (e.g. when executing a python builder)
> while reaches 400% when compiling C codes. I have a python builder
> that generates quite some .h files, which all the C files depend on.
> Those .h files has no relationship/dependencies among themselves, and
> running with --debug=explain gives below information:
>
> ./run configs/qemu_arm_vexpress.py -j8 --debug=explain
> Running : ./tools/scons -Q config=configs/qemu_arm_vexpress.py -j8
> --debug=explain
> USW INFO: usw_config completed in 1.60 seconds.
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/bios_bin/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/bios_bin/config/config.h
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/bios_elf/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/bios_elf/config/config.h
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/core0/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/core0/config/config.h
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/core1/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/core1/config/config.h
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/core2/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/core2/config/config.h
> scons: rebuilding
> `build/configs/qemu_arm_vexpress/core3/config/config.h' because
> AlwaysBuild() is specified
> __CONFIG build/configs/qemu_arm_vexpress/core3/config/config.h
>
> Any hints why these are not building in parallel? Is it because scons
> builds python with multithreads where the GIL eats all the CPU
> intensive workloads performance? Or should I re-implement the builder
> in such a way that it is not using scons tools but rather running as a
> dedicated executable and then wrapped into scons tools?
>
> Thanks,
> Yanghao Hua
> _______________________________________________
> Scons-users mailing list
> [email protected]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users