Re: Fwd: Dependency tree for test framework
Bill Deegan <[email protected]> Thu, 3 Apr 2025 14:37:37 -0700
| Newsgroups | gmane.comp.programming.tools.scons.user |
|---|---|
| Message-ID | <CAEyG4CE8CKs34G6mUJ2pzOXKaBfUvuD8LHhAQQOCM43=RucSyw@mail.gmail.com> |
--===============3216465973071117664== Content-Type: multipart/alternative; boundary="000000000000bfdbcc0631e6941b" --000000000000bfdbcc0631e6941b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable You can verify the dependencies match what you expect via: scons --tree=3Dall -n On Thu, Apr 3, 2025 at 2:36=E2=80=AFPM Bill Deegan <[email protected]= om> wrote: > BTW in SCons compilers and linkers are not considered immutable and are > usually content bashed and in the dependency tree. > > So something like this should work: > > DefaultEnvironment(tools=3D[]) > env=3DEnvironment(tools=3D[]) > env.Command(['aNew/b.out','aNew/b.log],['a.ini','b.dat'],'./aNew.exe') > env.Command(['aOld/b.out','aOld/b.log],['a.ini','b.dat'],'./aOld.exe') > > env.Command(['aNew/b.out.tmp','aOld/b.out.tmp'],['aNew/b.out','aOld/b.out= '],'./PreProcess.py') > env.Command('b.log.diff',['aNew/b.log','aOld/b.log'],'./LogDiff.exe') > > env.Command('b.out.diff',['aNew/b.out.tmp','aOld/b.out.tmp'],'./OutDiff.p= y') > > > This assumes you don't want SCons to also build aNew.exe, aOld.exe, > LogDiff.exe. > If you do, then you'd remove tools=3D[] from Environment, and add some > env.Program()'s above. > -Bill > > > > On Thu, Apr 3, 2025 at 2:26=E2=80=AFPM Bill Deegan <bill@baddogconsulting= .com> > wrote: > >> Ahh so the arrows are the opposite of how one normally diagrams a build >> DAG.. >> >> So PreProcess.py doesn't depend on a{New,Old}/b.out, it just read them, >> in reality a{New,Old}/b.out.tmp depends on PreProcess.py and respectivel= y >> a{New,Old}b.out ? >> >> -Bill >> >> On Thu, Apr 3, 2025 at 2:06=E2=80=AFPM Oliver Koch via Scons-users < >> [email protected]> wrote: >> >>> Dear all, >>> >>> I have seen that my mail was received and I also got some answers. Than= k >>> you for that! Appreciated. >>> >>> However, I do not know, how to answer, so I write another mail and hope >>> that the subject will assign it to my thread. >>> >>> @Mats Wichmann: >>> Thank you for the hints to the two chapters. >>> I will look into it. >>> >>> @Keith Prussing: >>> Thank you for the hints to ToolsForFools. >>> I will look into it. >>> >>> @Bill Deegan: >>> >>> > wow that's about the most complicated way to share your dependencies = I've >>> > seen so far.. >>> >>> I am sorry. I thought the tree is clearer than a long verbal explanatio= n. >>> >>> > I see an arc from aOld\b.out.tmp -> aOld\b.out which goes around >>> > PreProcess.py (same for aNew\b.out.tmp). >>> > How does b.out "use" the .tmp file directly? >>> > Or is that a mistake/error in your representation? >>> >>> Good point, but no, it is not a "mistake" (see below: 3 types of arrows= ). >>> >>> PreProcess.py reads aNew\b.out and writes aNew\b.out.tmp. >>> >>> If either PreProcess.py changes (e.g. because I fix a bug in the Python >>> script) or the input file aNew\b.out changes aNew\b.out.tmp must be >>> re-created. SCons should recognize this a re-create aNew\b.out.tmp >>> >>> The arrows should be read as "depends on" in one or the other way. >>> This is maybe, what causes confusion. I see 3 types of dependencies: >>> 1.) "reads" (from app to input file) >>> aNew.exe reads a.ini >>> aNew.exe reads b.dat >>> PreProcess.py reads aNew\b.out >>> and so on >>> 2.) "is created by" (from output file to app) >>> aNew\b.out is created by aNew.exe >>> aNew\b.out.tmp is created by PreProcess.py >>> b.log.diff is created by LogDiff.exe >>> and so on >>> 3.) "(logically) depends on" (from input to output file) >>> aNew\b.out.tmp depends on aNew\b.out >>> aOld\b.out.tmp depends on aOld\b.out >>> (I skipped other arrows, e.g. from aNew\b.out to b.dat, for clarity= . >>> For the above two it was necessary to avoid the ambiguity that >>> aNew\b.out.tmp depends only on aNew\b.out and >>> aOld\b.out.tmp depends only on aOld\b.out.) >>> >>> > Also question how a.ini and b.dat can accept outputs from both aNew.e= xe and >>> > aOld.exe? >>> See above. Arrows show dependency (I guess this is UML compliant.). >>> The data flow goes in the oppsosite direction. >>> >>> PreProcess.py, LogDiff.exe, and OutDiff.py will not change very often, = but I will not exclude it. >>> aNew.exe will change, since this is the app I am developing. >>> aOld.exe is the old app, so it should usually not change. >>> >>> So, all boxes should be regarded as "mutable". >>> >>> Usually, in a normal build system, the compiler and linker are regarded >>> as immutable. My situation is comparable to the scenario that I am >>> developing the compiler and linker (.exe/.py) and I want to check, whet= her the >>> build targets (.diff) are still the same, although the source files (.d= at) or config (.ini) have not >>> changed. >>> >>> Kind regards, >>> >>> Oliver >>> >>> Am 29.03.2025 um 06:30 schrieb Oliver Koch: >>> >>> Dear Scons developers and users, >>> >>> I would like to set up a rather complex test framework to compare the >>> changes between different versions (old vs. new) of an executable. >>> The workflow also includes different steps (tools) to process resp. >>> compare the output files. >>> >>> See the attached HTML file (resp. the according Markdown file). >>> Any green box should be regarded as an independent component the can be >>> changed by me. Any white box is a dependent file that is generated from= the >>> independents. >>> >>> The task is to create a dependency tree in SCons that re-generates the >>> (necessary) final output files (b.out.diff resp.b.log.diff), if any of = the >>> dependent green boxes have been changed. >>> >>> I am struggeling with your documentation to conclude, how a SCons scrip= t >>> should look like for the above purpose, since I am not using any classi= cal >>> compiler or linker in this workflow. >>> Could you indicate me, how the shown dependency tree must be implemente= d >>> with a SCons script? >>> (And maybe explain a little bit why?) >>> >>> Regards, >>> >>> Oliver >>> >>> >>> >>> >>> <https://www.avast.com/sig-email?utm_medium=3Demail&utm_source=3Dlink&u= tm_campaign=3Dsig-email&utm_content=3Demailclient> >>> Virenfrei.www.avast.com >>> <https://www.avast.com/sig-email?utm_medium=3Demail&utm_source=3Dlink&u= tm_campaign=3Dsig-email&utm_content=3Demailclient> >>> <#m_3322483925395728487_m_-6009214567214357308_m_-178949517691114710_DA= B4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >>> _______________________________________________ >>> Scons-users mailing list >>> [email protected] >>> https://pairlist4.pair.net/mailman/listinfo/scons-users >>> >> --000000000000bfdbcc0631e6941b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>You can verify the dependencies match what you expect= via:</div><div><br></div><div>scons --tree=3Dall -n</div><div><br></div><d= iv><br></div></div><br><div class=3D"gmail_quote gmail_quote_container"><di= v dir=3D"ltr" class=3D"gmail_attr">On Thu, Apr 3, 2025 at 2:36=E2=80=AFPM B= ill Deegan <<a href=3D"mailto:[email protected]">bill@baddogcons= ulting.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><div dir=3D"ltr"><div>BTW in SCons compilers and linkers are not= considered immutable and are usually content bashed and in the dependency = tree.</div><div><br></div><div>So something like this should work:</div><di= v><br></div><div>DefaultEnvironment(tools=3D[])</div><div>env=3DEnvironment= (tools=3D[])</div><div>env.Command(['aNew/b.out','aNew/b.log],[= 'a.ini','b.dat'],'./aNew.exe')</div><div><div>env.C= ommand(['aOld/b.out','aOld/b.log],['a.ini','b.dat&#= 39;],'./aOld.exe')</div><div>env.Command(['aNew/b.out.tmp',= 'aOld/b.out.tmp'],['aNew/b.out','aOld/b.out'],'= ./PreProcess.py')</div><div>env.Command('b.log.diff',['aNew= /b.log','aOld/b.log'],'./LogDiff.exe')</div><div>env.Co= mmand('b.out.diff',['aNew/b.out.tmp','aOld/b.out.tmp= 9;],'./OutDiff.py')</div><div><br></div><div><br></div><div>This as= sumes you don't want SCons to also build aNew.exe, aOld.exe, LogDiff.ex= e.</div><div>If you do, then you'd remove tools=3D[] from Environment, = and add some env.Program()'s above.</div><div>-Bill</div><br></div><div= ><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g= mail_attr">On Thu, Apr 3, 2025 at 2:26=E2=80=AFPM Bill Deegan <<a href= =3D"mailto:[email protected]" target=3D"_blank">bill@baddogconsulti= ng.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m= argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left= :1ex"><div dir=3D"ltr"><div>Ahh so the arrows are the opposite of how one n= ormally diagrams a build DAG..</div><div><br></div><div>So PreProcess.py do= esn't depend on a{New,Old}/b.out, it just read them, in reality a{New,O= ld}/b.out.tmp depends on PreProcess.py and respectively a{New,Old}b.out ?</= div><div><br></div><div>-Bill</div></div><br><div class=3D"gmail_quote"><di= v dir=3D"ltr" class=3D"gmail_attr">On Thu, Apr 3, 2025 at 2:06=E2=80=AFPM O= liver Koch via Scons-users <<a href=3D"mailto:[email protected]" tar= get=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid= rgb(204,204,204);padding-left:1ex"><u></u> =20 =20 =20 <div> <div>Dear all,<br> <br> I have seen that my mail was received and I also got some answers. Thank you for that! Appreciated.<br> <br> However, I do not know, how to answer, so I write another mail and hope that the subject will assign it to my thread.<br> <br> @Mats Wichmann:<br> Thank you for the hints to the two chapters.<br> I will look into it.<br> <br> @Keith Prussing:<br> Thank you for the hints to ToolsForFools.<br> I will look into it.<br> <br> @Bill Deegan:<br> <pre>> wow that's about the most complicated way to share your= dependencies I've > seen so far.. I am sorry. I thought the tree is clearer than a long verbal explanation. > I see an arc from aOld\b.out.tmp -> aOld\b.out which goes around > PreProcess.py (same for aNew\b.out.tmp). > How does b.out "use" the .tmp file directly? > Or is that a mistake/error in your representation? Good point, but no, it is not a "mistake" (see below: 3 types of = arrows). PreProcess.py reads aNew\b.out and writes aNew\b.out.tmp. If either PreProcess.py changes (e.g. because I fix a bug in the Python script) or the input file aNew\b.out changes aNew\b.out.tmp must be re-created. SCons should recognize this a re-create aNew\b.out.tmp The arrows should be read as "depends on" in one or the other way= . This is maybe, what causes confusion. I see 3 types of dependencies: 1.) "reads" (from app to input file) aNew.exe reads a.ini aNew.exe reads b.dat PreProcess.py reads aNew\b.out and so on 2.) "is created by" (from output file to app) aNew\b.out is created by aNew.exe aNew\b.out.tmp is created by PreProcess.py b.log.diff is created by LogDiff.exe and so on 3.) "(logically) depends on" (from input to output file) aNew\b.out.tmp depends on aNew\b.out aOld\b.out.tmp depends on aOld\b.out (I skipped other arrows, e.g. from aNew\b.out to b.dat, for clarity. For the above two it was necessary to avoid the ambiguity that aNew\b.out.tmp depends only on aNew\b.out and aOld\b.out.tmp depends only on aOld\b.out.) > Also question how a.ini and b.dat can accept outputs from both aNew.ex= e and > aOld.exe? See above. Arrows show dependency (I guess this is UML compliant.). The data flow goes in the oppsosite direction. PreProcess.py, LogDiff.exe, and OutDiff.py will not change very often, but = I will not exclude it. aNew.exe will change, since this is the app I am developing. aOld.exe is the old app, so it should usually not change. So, all boxes should be regarded as "mutable". Usually, in a normal build system, the compiler and linker are regarded as immutable. My situation is comparable to the scenario that I am developing the compiler and linker (.exe/.py) and I want to check, whether = the build targets (.diff) are still the same, although the source files (.dat) = or config (.ini) have not changed. </pre> Kind regards,<br> <br> Oliver <br> <br> Am 29.03.2025 um 06:30 schrieb Oliver Koch:<br> </div> <blockquote type=3D"cite">Dear Scons developers and users, <br> <br> I would like to set up a rather complex test framework to compare the changes between different versions (old vs. new) of an executable. <br> The workflow also includes different steps (tools) to process resp. compare the output files. <br> <br> See the attached HTML file (resp. the according Markdown file). <br> Any green box should be regarded as an independent component the can be changed by me. Any white box is a dependent file that is generated from the independents. <br> <br> The task is to create a dependency tree in SCons that re-generates the (necessary) final output files (b.out.diff resp.b.log.diff), if any of the dependent green boxes have been changed. <br> <br> I am struggeling with your documentation to conclude, how a SCons script should look like for the above purpose, since I am not using any classical compiler or linker in this workflow. <br> Could you indicate me, how the shown dependency tree must be implemented with a SCons script? <br> (And maybe explain a little bit why?) <br> <br> Regards, <br> <br> Oliver<br> </blockquote> <br> <div id=3D"m_3322483925395728487m_-6009214567214357308m_-1789495176911147= 10DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br><table style=3D"border-top:1px = solid rgb(211,212,222)"><tbody><tr><td style=3D"width:55px;padding-top:13px= "><a href=3D"https://www.avast.com/sig-email?utm_medium=3Demail&utm_sou= rce=3Dlink&utm_campaign=3Dsig-email&utm_content=3Demailclient" targ= et=3D"_blank"><img src=3D"https://s-install.avcdn.net/ipm/preview/icons/ico= n-envelope-tick-round-orange-animated-no-repeat-v1.gif" alt=3D"" width=3D"4= 6" height=3D"29" style=3D"width: 46px; height: 29px;"></a></td><td style=3D= "width:470px;padding-top:12px;color:rgb(65,66,78);font-size:13px;font-famil= y:Arial,Helvetica,sans-serif;line-height:18px">Virenfrei.<a href=3D"https:/= /www.avast.com/sig-email?utm_medium=3Demail&utm_source=3Dlink&utm_c= ampaign=3Dsig-email&utm_content=3Demailclient" style=3D"color:rgb(68,83= ,234)" target=3D"_blank">www.avast.com</a></td></tr></tbody></table><a href= =3D"#m_3322483925395728487_m_-6009214567214357308_m_-178949517691114710_DAB= 4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width=3D"1" height=3D"1"> </a></div></di= v> _______________________________________________<br> Scons-users mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon= s.org</a><br> <a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"= noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s= cons-users</a><br> </blockquote></div> </blockquote></div> </blockquote></div> --000000000000bfdbcc0631e6941b-- --===============3216465973071117664== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Scons-users mailing list [email protected] https://pairlist4.pair.net/mailman/listinfo/scons-users --===============3216465973071117664==--