Re: managing dependencies

We don’t “have to” do this, but we do it to speed up multi-hour builds. If we don’t do it (the real life project is 1000x more complex naturally), we end up with 150 products finishing roughly at the same time, which will max out the bandwidth available for uploading the finished artifacts to some server for example.

Actually, I think it is rather nice that you can do this in SCons, in a rather straight forward and clean way.

Cheers,
Arvid


From: Scons-users <[email protected]> on behalf of Bill Deegan <[email protected]>
Reply to: SCons users mailing list <[email protected]>
Date: Monday, 9 December 2019 at 16:55
To: SCons users mailing list <[email protected]>
Subject: Re: [Scons-users] managing dependencies

That said if you have to do this, you're likely doing something wrong.

If you have a custom builder, perhaps you're not adding all the output files in the emitter or in the emitter call.
SCons can't order dependencies it's not aware of.

-Bill

On Mon, Dec 9, 2019 at 3:26 AM Arvid Rosén <[email protected]<mailto:[email protected]>> wrote:
Well,

In SCons, there is no need to use an extra file for this purpose. You can declare dependencies without having files too.

I mean something like this:

A = Program(“a.cpp”)
B = Program(“b.cpp”)
C = Program(“c.cpp”)
D = Program(“d.cpp”)

Depends(C, A)
Depends(D, [A, B])

Builds of A and B starts immediately. C starts when A is finished. D starts when A and B have finished.

Being a python script, this can be made more general using for loops and stuff, and we typically have a variable somewhere to control how many products that can be in-flight simultaneously. Btw, this is a typical use case of “Requires” rather than “Depends” I guess, but I chose the latter for simplicity here.

Cheers,
Arvid



From: daggs <[email protected]<mailto:[email protected]>>
Date: Monday, 9 December 2019 at 09:04
To: Arvid Rosén <[email protected]<mailto:[email protected]>>
Cc: SCons users mailing list <[email protected]<mailto:[email protected]>>
Subject: Re: [Scons-users] managing dependencies

Greetings,

something like for all short deps, depend them on a file which I can use touch to create?
will the same file for all of them do the job?

Thank,

Dagg.


Sent: Monday, December 09, 2019 at 9:38 AM
From: "Arvid Rosén" <[email protected]<mailto:[email protected]>>
To: "SCons users mailing list" <[email protected]<mailto:[email protected]>>
Subject: Re: [Scons-users] managing dependencies
Hi,

Maybe, but I certainly wouldn’t rely on that.

We do very massive builds with hundreds of targets in parallel. To avoid all targets finishing almost at the same time (competing with each other for the same limited resources), we add “fake” dependencies so to schedule the builds so that only a couple of targets can be worked on at the same time, but still 20 threads or so can be used. This is very easy to do in SCons.

Cheers,
Arvid

From: Scons-users <[email protected]<mailto:[email protected]>> on behalf of daggs <[email protected]<mailto:[email protected]>>
Reply to: SCons users mailing list <[email protected]<mailto:[email protected]>>
Date: Monday, 9 December 2019 at 08:06
To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>
Cc: SCons users mailing list <[email protected]<mailto:[email protected]>>
Subject: Re: [Scons-users] managing dependencies

Greetings,

does the order of dependecy declaration affects the order?

Dagg.


Sent: Monday, December 09, 2019 at 1:06 AM
From: "Bill Deegan" <[email protected]<mailto:[email protected]>>
To: "SCons users mailing list" <[email protected]<mailto:[email protected]>>
Subject: Re: [Scons-users] managing dependencies
Nope.
You specify the dependencies, SCons figures out the order.
If you need one thing to happen before another then that's a dependency you must explicitly list.

-Bill

On Sun, Dec 8, 2019 at 4:49 AM daggs <[email protected]<mailto:[email protected]>> wrote:
Greetings,

I wanted to ask about scons dependencies management, I have a target with large number of dependencies, all the dependencies don't have dependencies of their own.
some of the dependencies take more time to generate than the other ones.
I've noticed that not all the dependencies are handled at the start (I have a print at the start of each dependency handling).
is it possible to instruct scons to give an higher priority to a specific dependency?

Thanks,

Dagg.
_______________________________________________
Scons-users mailing list
[email protected]<mailto:[email protected]>
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________ Scons-users mailing list [email protected]<mailto:[email protected]> https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________ Scons-users mailing list [email protected]<mailto:[email protected]> https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
[email protected]<mailto:[email protected]>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users