Re: SCons misses a dependency which results in incorrect output.
Tal Dayan <[email protected]> Wed, 16 Jul 2025 22:37:15 -0700
| Newsgroups | gmane.comp.programming.tools.scons.user |
|---|---|
| Message-ID | <CAM+zu_6_rFErBmpVph+S8V_9+RpvNOHmBW4U81f-VCCyRLWtiA@mail.gmail.com> |
--===============7563494951548632737==
Content-Type: multipart/alternative; boundary="000000000000d87cc8063a1967ed"
--000000000000d87cc8063a1967ed
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Yes, this work. Here is the log: https://pastebin.com/4GW5wWqQ
I wonder, will it also work deleting file2 from the database instead of
deleting the file itself? This may be less intrusive in case the
erroneous file may help diagnosing the issue.
Thanks for your help.
On Wed, Jul 16, 2025 at 9:13=E2=80=AFPM Bill Deegan <bill@baddogconsulting.=
com>
wrote:
> If you patch your SCons sources like this, it should remove any failed
> actions targets.
> (This is just a quick hack, working on a proper fix with option to contro=
l
> it)
>
> diff --git a/SCons/Script/Main.py b/SCons/Script/Main.py
> index 98f1f345a..f0152a383 100644
> --- a/SCons/Script/Main.py
> +++ b/SCons/Script/Main.py
> @@ -317,6 +317,10 @@ class BuildTask(SCons.Taskmaster.OutOfDateTask):
> node =3D [node]
> nodename =3D ', '.join(map(str, node))
>
> + for n in node:
> + print(f"removing {n}")
> + n.remove()
> +
> errfmt =3D "scons: *** [%s] %s\n"
> sys.stderr.write(errfmt % (nodename, buildError))
>
>
>
>
> On Wed, Jul 16, 2025 at 5:15=E2=80=AFPM Tal Dayan <[email protected]> wrote:
>
>> Our system is based on Python plugin classes that provide scons Builders
>> so I wonder if we can just define our own subclass of Builder that
>> automatically adds the deletion on errors functionality.
>>
>>
>> https://github.com/FPGAwars/apio/blob/008b090e92246414353776ef5df4c0dd3e=
274fab/apio/scons/plugin_ice40.py#L62C16-L62C23
>>
>> On Wed, Jul 16, 2025 at 5:02=E2=80=AFPM Bill Deegan <bill@baddogconsulti=
ng.com>
>> wrote:
>>
>>> Tai,
>>>
>>> If you have a program you're running as the action in a command which i=
s
>>> erroring out but writing the file, you can work around this issue for t=
he
>>> time being by wrapping that program with a script which checks the exit
>>> value of your program, and removing the target files if it exits with a=
n
>>> error status.
>>>
>>> That should be sufficient to get you going again.
>>>
>>> -Bill
>>>
>>> On Wed, Jul 16, 2025 at 3:18=E2=80=AFPM Bill Deegan <bill@baddogconsult=
ing.com>
>>> wrote:
>>>
>>>> Did you read my explanation and how to fix it in your environment
>>>> assuming command.sh is similar to what's really happening in your buil=
d?
>>>>
>>>>
>>>> On Wed, Jul 16, 2025 at 3:16=E2=80=AFPM Bill Deegan <bill@baddogconsul=
ting.com>
>>>> wrote:
>>>>
>>>>> It's an enhancement request.
>>>>>
>>>>> The bug is in your command.sh script.
>>>>> I'll change the description and text.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 16, 2025 at 12:56=E2=80=AFPM Tal Dayan <[email protected]> wr=
ote:
>>>>>
>>>>>> Hi Bill, I will file an issue.
>>>>>>
>>>>>> I don't think it's safe for SCons to assume that once a program
>>>>>> writes an output file it can't crash.
>>>>>>
>>>>>> On Wed, Jul 16, 2025 at 12:45=E2=80=AFPM Bill Deegan <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Arguably your command.sh is incorrectly written.
>>>>>>>
>>>>>>> It copies the file and then checks it's contents and exits with
>>>>>>> error status if there improper.
>>>>>>>
>>>>>>> So a quick fix would be to check the input before copying the file.
>>>>>>>
>>>>>>> From the URL you posted in other email here's the command.sh
>>>>>>> #!/bin/bash
>>>>>>> # Usage: ./command input > output
>>>>>>>
>>>>>>> # Copy source to destination
>>>>>>> cp "$1" "$2"
>>>>>>>
>>>>>>> # If the input file stats with 'bad' inject an error AFTER creating
>>>>>>> the output file.
>>>>>>>
>>>>>>> first_line=3D$(head -n 1 "$1")
>>>>>>>
>>>>>>> if [[ "$first_line" =3D=3D bad* ]]; then
>>>>>>> exit 1
>>>>>>> fi
>>>>>>>
>>>>>>> exit 0
>>>>>>>
>>>>>>> Change it to this:
>>>>>>> #!/bin/bash
>>>>>>> # Usage: ./command input > output
>>>>>>>
>>>>>>> # If the input file stats with 'bad' inject an error AFTER creating
>>>>>>> the output file.
>>>>>>>
>>>>>>> first_line=3D$(head -n 1 "$1")
>>>>>>>
>>>>>>> if [[ "$first_line" =3D=3D bad* ]]; then
>>>>>>> exit 1
>>>>>>> fi
>>>>>>>
>>>>>>> # Copy source to destination
>>>>>>> cp "$1" "$2"
>>>>>>>
>>>>>>> exit 0
>>>>>>>
>>>>>>> Results:
>>>>>>>
>>>>>>> % ./run.sh
>>>>>>>
>>>>>>> ----- Iteration 1: file1 =3D 'good' -----
>>>>>>>
>>>>>>> scons: Reading SConscript files ...
>>>>>>> scons: done reading SConscript files.
>>>>>>> scons: Building targets ...
>>>>>>> scons: building `file2' because it doesn't exist
>>>>>>> ./command.sh file1 file2
>>>>>>> +-file2
>>>>>>> +-file1
>>>>>>> scons: done building targets.
>>>>>>>
>>>>>>> File1
>>>>>>> MD5 (file1) =3D d7f986677d9f563bd1794b09d82206a3
>>>>>>> 1 good
>>>>>>>
>>>>>>> File2
>>>>>>> MD5 (file2) =3D d7f986677d9f563bd1794b09d82206a3
>>>>>>> 1 good
>>>>>>>
>>>>>>> DBlite:
>>>>>>> =3D=3D=3D .:
>>>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>>> 2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE
>>>>>>> $TARGET]
>>>>>>>
>>>>>>> ----- Iteration 2: file1 =3D 'bad' -----
>>>>>>>
>>>>>>> scons: Reading SConscript files ...
>>>>>>> scons: done reading SConscript files.
>>>>>>> scons: Building targets ...
>>>>>>> scons: rebuilding `file2' because `file1' changed
>>>>>>> ./command.sh file1 file2
>>>>>>> scons: *** [file2] Error 1
>>>>>>> +-file2
>>>>>>> +-file1
>>>>>>> scons: building terminated because of errors.
>>>>>>>
>>>>>>> File1
>>>>>>> MD5 (file1) =3D df207dc9143c6fabf60b69b9c3035103
>>>>>>> 1 bad
>>>>>>>
>>>>>>> File2
>>>>>>> md5: file2: No such file or directory
>>>>>>> cat: file2: No such file or directory
>>>>>>>
>>>>>>> DBlite:
>>>>>>> =3D=3D=3D .:
>>>>>>> file1: df207dc9143c6fabf60b69b9c3035103 1752694898 4
>>>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>>> 2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE
>>>>>>> $TARGET]
>>>>>>>
>>>>>>> ----- Iteration 3: file1 =3D 'good' -----
>>>>>>>
>>>>>>> scons: Reading SConscript files ...
>>>>>>> scons: done reading SConscript files.
>>>>>>> scons: Building targets ...
>>>>>>> scons: building `file2' because it doesn't exist
>>>>>>> ./command.sh file1 file2
>>>>>>> +-file2
>>>>>>> +-file1
>>>>>>> scons: done building targets.
>>>>>>>
>>>>>>> File1
>>>>>>> MD5 (file1) =3D d7f986677d9f563bd1794b09d82206a3
>>>>>>> 1 good
>>>>>>>
>>>>>>> File2
>>>>>>> MD5 (file2) =3D d7f986677d9f563bd1794b09d82206a3
>>>>>>> 1 good
>>>>>>>
>>>>>>> DBlite:
>>>>>>> =3D=3D=3D .:
>>>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>>>> 2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE
>>>>>>> $TARGET]
>>>>>>>
>>>>>>> There's not presently logic in SCons to delete target files if the
>>>>>>> associated action yields an error.
>>>>>>>
>>>>>>> I think what you want is equivalent to makes .DELETE_ON_FAILURE,
>>>>>>> there's actually a SO question on this:
>>>>>>>
>>>>>>> https://stackoverflow.com/questions/29546276/scons-delete-target-on=
-failure-of-any-action
>>>>>>>
>>>>>>> Please go ahead and file an enhancement request to add equivalent
>>>>>>> to DELETE_ON_FAILURE, please include your reproducer scripts.
>>>>>>>
>>>>>>> -Bill
>>>>>>>
>>>>>>> On Wed, Jul 16, 2025 at 10:44=E2=80=AFAM Tal Dayan <[email protected]> =
wrote:
>>>>>>>
>>>>>>>> Looking at the end state of scons after invocation #3, the actual
>>>>>>>> md5 of file2 doesn't match its md5 in the dblite.
>>>>>>>>
>>>>>>>> https://i.imgur.com/NGco3yQ.png
>>>>>>>>
>>>>>>>> On Wed, Jul 16, 2025 at 10:33=E2=80=AFAM Tal Dayan <[email protected]>=
wrote:
>>>>>>>>
>>>>>>>>> Hi Keith, I updated the example files here
>>>>>>>>> https://github.com/FPGAwars/apio/issues/676
>>>>>>>>>
>>>>>>>>> They now include the md5 of the files and a dump of
>>>>>>>>> .sconsign.dblite
>>>>>>>>>
>>>>>>>>> On Wed, Jul 16, 2025 at 10:08=E2=80=AFAM Keith Prussing <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> I suspect it's because `file1` has the same hash in the
>>>>>>>>>> .sconsign.dblite as the last "good" build (i.e. the first one).
>>>>>>>>>> Thus
>>>>>>>>>> you get the line "scons: `file2' is up to date." on the third ru=
n.
>>>>>>>>>> However, I am not an expert in the specifics of SCons' hashing
>>>>>>>>>> methods.
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 16, 2025 at 12:38=E2=80=AFPM Tal Dayan <[email protected]=
m> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Hi all,
>>>>>>>>>> >
>>>>>>>>>> > We encountered this problem with the nextpnr tool and created
>>>>>>>>>> here a small and independent example that demonstrates it.
>>>>>>>>>> >
>>>>>>>>>> > In the example below, a shell script 'command.sh' reads the
>>>>>>>>>> source file 'file1' and writes it to the target file 'file2'. Ho=
wever, if
>>>>>>>>>> the
>>>>>>>>>> > source file starts with 'bad' it exits with an error code,
>>>>>>>>>> *after* creating the target file.
>>>>>>>>>> >
>>>>>>>>>> > The script `run.sh', runs scons three times with these values
>>>>>>>>>> of the source file file1 'good', 'bad', and 'good'. The expecta=
tion is
>>>>>>>>>> that after the third scons run, file2 should contain the value '=
good' but
>>>>>>>>>> it contains the value 'bad'.
>>>>>>>>>> >
>>>>>>>>>> > Do we miss anything or is it simply a bug?
>>>>>>>>>> >
>>>>>>>>>> > SConstruct:
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> > # SCons environment
>>>>>>>>>> >
>>>>>>>>>> > env =3D Environment()
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > # Copy file1 =E2=86=92 file2 using command.sh
>>>>>>>>>> >
>>>>>>>>>> > # Inject an error if file1 starts with 'bad"
>>>>>>>>>> >
>>>>>>>>>> > file2 =3D env.Command(
>>>>>>>>>> >
>>>>>>>>>> > target=3D'file2',
>>>>>>>>>> >
>>>>>>>>>> > source=3D'file1',
>>>>>>>>>> >
>>>>>>>>>> > action=3D'./command.sh $SOURCE > $TARGET'
>>>>>>>>>> >
>>>>>>>>>> > )
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > # Make 'file2' the default target
>>>>>>>>>> >
>>>>>>>>>> > Default(file2)
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > command.sh:
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> > #!/bin/bash
>>>>>>>>>> >
>>>>>>>>>> > # Usage: ./command input > output
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > # Read from the first argument and copy to stdout
>>>>>>>>>> >
>>>>>>>>>> > cat "$1"
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > # If the input file starts with 'bad', inject an error AFTER
>>>>>>>>>> creating the output file.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > first_line=3D$(head -n 1 "$1")
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > if [[ "$first_line" =3D=3D bad* ]]; then
>>>>>>>>>> >
>>>>>>>>>> > exit 1
>>>>>>>>>> >
>>>>>>>>>> > fi
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > exit 0
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > run.sh
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> > #!/bin/bash
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > # Clean up.
>>>>>>>>>> >
>>>>>>>>>> > rm -f .sconsign.dblite
>>>>>>>>>> >
>>>>>>>>>> > rm -f file[12]
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> > echo "---- Iteration 1: file1 =3D 'good'"
>>>>>>>>>> >
>>>>>>>>>> > echo "good" > file1
>>>>>>>>>> >
>>>>>>>>>> > scons
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File1"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file1
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File2"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file2
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> > echo "---- Iteration 2: file1 =3D 'bad'"
>>>>>>>>>> >
>>>>>>>>>> > echo "bad" > file1
>>>>>>>>>> >
>>>>>>>>>> > cat -n file1
>>>>>>>>>> >
>>>>>>>>>> > scons
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File1"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file1
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File2"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file2
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> > echo "---- Iteration 3: file1 =3D 'good'"
>>>>>>>>>> >
>>>>>>>>>> > echo "good" > file1
>>>>>>>>>> >
>>>>>>>>>> > cat -n file1
>>>>>>>>>> >
>>>>>>>>>> > scons
>>>>>>>>>> >
>>>>>>>>>> > echo
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File1"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file1
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > echo "File2"
>>>>>>>>>> >
>>>>>>>>>> > cat -n file2
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > Run log:
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> > $ ./run.sh
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > ---- Iteration 1: file1 =3D 'good'
>>>>>>>>>> >
>>>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>>>> >
>>>>>>>>>> > scons: done reading SConscript files.
>>>>>>>>>> >
>>>>>>>>>> > scons: Building targets ...
>>>>>>>>>> >
>>>>>>>>>> > ./command.sh file1 > file2
>>>>>>>>>> >
>>>>>>>>>> > scons: done building targets.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > File1
>>>>>>>>>> >
>>>>>>>>>> > 1 good
>>>>>>>>>> >
>>>>>>>>>> > File2
>>>>>>>>>> >
>>>>>>>>>> > 1 good
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > ---- Iteration 2: file1 =3D 'bad'
>>>>>>>>>> >
>>>>>>>>>> > 1 bad
>>>>>>>>>> >
>>>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>>>> >
>>>>>>>>>> > scons: done reading SConscript files.
>>>>>>>>>> >
>>>>>>>>>> > scons: Building targets ...
>>>>>>>>>> >
>>>>>>>>>> > ./command.sh file1 > file2
>>>>>>>>>> >
>>>>>>>>>> > scons: *** [file2] Error 1
>>>>>>>>>> >
>>>>>>>>>> > scons: building terminated because of errors.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > File1
>>>>>>>>>> >
>>>>>>>>>> > 1 bad
>>>>>>>>>> >
>>>>>>>>>> > File2
>>>>>>>>>> >
>>>>>>>>>> > 1 bad
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > ---- Iteration 1: file1 =3D 'good'
>>>>>>>>>> >
>>>>>>>>>> > 1 good
>>>>>>>>>> >
>>>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>>>> >
>>>>>>>>>> > scons: done reading SConscript files.
>>>>>>>>>> >
>>>>>>>>>> > scons: Building targets ...
>>>>>>>>>> >
>>>>>>>>>> > scons: `file2' is up to date.
>>>>>>>>>> >
>>>>>>>>>> > scons: done building targets.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > File1
>>>>>>>>>> >
>>>>>>>>>> > 1 good
>>>>>>>>>> >
>>>>>>>>>> > File2
>>>>>>>>>> >
>>>>>>>>>> > 1 bad
>>>>>>>>>> >
>>>>>>>>>> > ----------------------------------------------
>>>>>>>>>> >
>>>>>>>>>> > _______________________________________________
>>>>>>>>>> > Scons-users mailing list
>>>>>>>>>> > [email protected]
>>>>>>>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Keith Prussing
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-users mailing list
>>>>>>>>>> [email protected]
>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>> Scons-users mailing list
>>>>>>>> [email protected]
>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Scons-users mailing list
>>>>>>> [email protected]
>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> [email protected]
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>> Scons-users mailing list
>>> [email protected]
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> [email protected]
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> [email protected]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
--000000000000d87cc8063a1967ed
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Yes, this work.=C2=A0 Here is the log:=C2=A0<a href=3D"htt=
ps://pastebin.com/4GW5wWqQ">https://pastebin.com/4GW5wWqQ</a><div><br></div=
><div>I wonder, will it also work deleting file2 from the database instead =
of deleting the file itself? This may be less intrusive in case the erroneo=
us=C2=A0file may help diagnosing the issue.</div><div><br></div><div>Thanks=
for your help.</div><div><br></div><div><br></div></div><br><div class=3D"=
gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On=
Wed, Jul 16, 2025 at 9:13=E2=80=AFPM Bill Deegan <<a href=3D"mailto:bil=
[email protected]">[email protected]</a>> wrote:<br></div><=
blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l=
eft:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>If y=
ou patch your SCons sources like this, it should remove any failed actions =
targets.</div><div>(This is just a quick hack, working on a proper fix with=
option to control it)</div><div><br></div><div>diff --git a/SCons/Script/M=
ain.py b/SCons/Script/Main.py<br>index 98f1f345a..f0152a383 100644<br>--- a=
/SCons/Script/Main.py<br>+++ b/SCons/Script/Main.py<br>@@ -317,6 +317,10 @@=
class BuildTask(SCons.Taskmaster.OutOfDateTask):<br>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0node =3D [node]<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0nodename =3D ', '.join(map(str, node))<br><br>+ =C2=A0 =C2=A0=
=C2=A0 =C2=A0for n in node:<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
print(f"removing {n}")<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0n.remove()<br>+<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0errfmt =3D "=
;scons: *** [%s] %s\n"<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sys.stderr=
.write(errfmt % (nodename, buildError))</div><div><br></div><div><br></div>=
<div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=
=3D"gmail_attr">On Wed, Jul 16, 2025 at 5:15=E2=80=AFPM Tal Dayan <<a hr=
ef=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>> wrote:<=
br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"=
>Our system is based on Python plugin classes that provide scons Builders s=
o I wonder if we can just define our own subclass of Builder that automatic=
ally adds=C2=A0the=C2=A0deletion on errors functionality.<div><br></div><di=
v><a href=3D"https://github.com/FPGAwars/apio/blob/008b090e92246414353776ef=
5df4c0dd3e274fab/apio/scons/plugin_ice40.py#L62C16-L62C23" target=3D"_blank=
">https://github.com/FPGAwars/apio/blob/008b090e92246414353776ef5df4c0dd3e2=
74fab/apio/scons/plugin_ice40.py#L62C16-L62C23</a></div></div><br><div clas=
s=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jul 16, 202=
5 at 5:02=E2=80=AFPM Bill Deegan <<a href=3D"mailto:bill@baddogconsultin=
g.com" target=3D"_blank">[email protected]</a>> wrote:<br></div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Tai=
,</div><div><br></div><div>If you have a program you're running as the =
action in a command which is erroring out but writing the file, you can wor=
k around this issue for the time being by wrapping that program with a scri=
pt which checks the exit value of your program, and removing the target fil=
es if it exits with an error status.</div><div><br></div><div>That should b=
e sufficient to get you going again.</div><div><br></div><div>-Bill</div></=
div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On=
Wed, Jul 16, 2025 at 3:18=E2=80=AFPM Bill Deegan <<a href=3D"mailto:bil=
[email protected]" target=3D"_blank">[email protected]</a>>=
wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=
=3D"ltr"><div>Did you read my explanation and how to fix it in your environ=
ment assuming command.sh is similar to what's really happening in your =
build?</div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D=
"ltr" class=3D"gmail_attr">On Wed, Jul 16, 2025 at 3:16=E2=80=AFPM Bill Dee=
gan <<a href=3D"mailto:[email protected]" target=3D"_blank">bill=
@baddogconsulting.com</a>> wrote:<br></div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex"><div dir=3D"ltr"><div>It's an enhancement request.=
</div><div><br></div><div>The bug is in your=C2=A0command.sh script.</div><=
div>I'll change the description and text.</div><div><br></div><div><br>=
</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_=
attr">On Wed, Jul 16, 2025 at 12:56=E2=80=AFPM Tal Dayan <<a href=3D"mai=
lto:[email protected]" target=3D"_blank">[email protected]</a>> wrote:<br></div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Hi Bill,=
I will file an issue.<div><br></div><div>I don't think it's safe f=
or SCons to assume that once a program writes an output file it can't c=
rash.=C2=A0</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" clas=
s=3D"gmail_attr">On Wed, Jul 16, 2025 at 12:45=E2=80=AFPM Bill Deegan <<=
a href=3D"mailto:[email protected]" target=3D"_blank">bill@baddogco=
nsulting.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin=
g-left:1ex"><div dir=3D"ltr"><div>Arguably your command.sh is incorrectly w=
ritten.</div><div><br></div><div>=C2=A0It copies the file and then checks i=
t's contents and exits with error status if there improper.</div><div><=
br></div><div>So a quick fix would be to check the input before copying the=
file.</div><div><br></div><div>From the URL you posted in other email here=
's the command.sh</div><div><span style=3D"font-family:monospace">#!/bi=
n/bash<br># Usage: ./command input > output<br><br># Copy source to dest=
ination<br>cp "$1" "$2"<br><br># If the input file stat=
s with 'bad' inject an error AFTER creating the output file.<br><br=
>first_line=3D$(head -n 1 "$1")<br><br>if [[ "$first_line&qu=
ot; =3D=3D bad* ]]; then<br>=C2=A0 exit 1<br>fi<br><br>exit 0</span></div><=
div><span style=3D"font-family:monospace"><br></span></div><div><span style=
=3D"font-family:monospace">Change it to this:</span></div><div><span style=
=3D"font-family:monospace">#!/bin/bash<br># Usage: ./command input > out=
put<br><br># If the input file stats with 'bad' inject an error AFT=
ER creating the output file.<br><br>first_line=3D$(head -n 1 "$1"=
)<br><br>if [[ "$first_line" =3D=3D bad* ]]; then<br>=C2=A0 exit =
1<br>fi<br><br># Copy source to destination<br>cp "$1" "$2&q=
uot;<br><br>exit 0</span><br><br></div><div>Results:</div><div><br></div><d=
iv><span style=3D"font-family:monospace">% ./run.sh<br><br>----- Iteration =
1: file1 =3D 'good' -----<br><br>scons: Reading SConscript files ..=
.<br>scons: done reading SConscript files.<br>scons: Building targets ...<b=
r>scons: building `file2' because it doesn't exist<br>./command.sh =
file1 file2<br>+-file2<br>=C2=A0 +-file1<br>scons: done building targets.<b=
r><br>File1<br>MD5 (file1) =3D d7f986677d9f563bd1794b09d82206a3<br>=C2=A0 =
=C2=A0 =C2=A01 good<br><br>File2<br>MD5 (file2) =3D d7f986677d9f563bd1794b0=
9d82206a3<br>=C2=A0 =C2=A0 =C2=A01 good<br><br>DBlite:<br>=3D=3D=3D .:<br>f=
ile1: d7f986677d9f563bd1794b09d82206a3 1752694898 5<br>file2: d7f986677d9f5=
63bd1794b09d82206a3 1752694898 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 file1: d7f9=
86677d9f563bd1794b09d82206a3 1752694898 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 2d=
bc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]<br><br>----- =
Iteration 2: file1 =3D 'bad' -----<br><br>scons: Reading SConscript=
files ...<br>scons: done reading SConscript files.<br>scons: Building targ=
ets ...<br>scons: rebuilding `file2' because `file1' changed<br>./c=
ommand.sh file1 file2<br>scons: *** [file2] Error 1<br>+-file2<br>=C2=A0 +-=
file1<br>scons: building terminated because of errors.<br><br>File1<br>MD5 =
(file1) =3D df207dc9143c6fabf60b69b9c3035103<br>=C2=A0 =C2=A0 =C2=A01 bad<b=
r><br>File2<br>md5: file2: No such file or directory<br>cat: file2: No such=
file or directory<br><br>DBlite:<br>=3D=3D=3D .:<br>file1: df207dc9143c6fa=
bf60b69b9c3035103 1752694898 4<br>file2: d7f986677d9f563bd1794b09d82206a3 1=
752694898 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 file1: d7f986677d9f563bd1794b09d=
82206a3 1752694898 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 2dbc2dce125a753309a27b7=
d5157aaaa [./command.sh $SOURCE $TARGET]<br><br>----- Iteration 3: file1 =
=3D 'good' -----<br><br>scons: Reading SConscript files ...<br>scon=
s: done reading SConscript files.<br>scons: Building targets ...<br>scons: =
building `file2' because it doesn't exist<br>./command.sh file1 fil=
e2<br>+-file2<br>=C2=A0 +-file1<br>scons: done building targets.<br><br>Fil=
e1<br>MD5 (file1) =3D d7f986677d9f563bd1794b09d82206a3<br>=C2=A0 =C2=A0 =C2=
=A01 good<br><br>File2<br>MD5 (file2) =3D d7f986677d9f563bd1794b09d82206a3<=
br>=C2=A0 =C2=A0 =C2=A01 good<br><br>DBlite:<br>=3D=3D=3D .:<br>file1: d7f9=
86677d9f563bd1794b09d82206a3 1752694899 5<br>file2: d7f986677d9f563bd1794b0=
9d82206a3 1752694899 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 file1: d7f986677d9f56=
3bd1794b09d82206a3 1752694899 5<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 2dbc2dce125a=
753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]</span></div><div><br></=
div><div>There's not presently logic in SCons to delete target files if=
the associated action yields an error.</div><div><br></div><div>I think wh=
at you want is equivalent to makes .DELETE_ON_FAILURE, there's actually=
a SO question on this:</div><div><a href=3D"https://stackoverflow.com/ques=
tions/29546276/scons-delete-target-on-failure-of-any-action" target=3D"_bla=
nk">https://stackoverflow.com/questions/29546276/scons-delete-target-on-fai=
lure-of-any-action</a></div><div><br></div><div>Please go ahead and file an=
enhancement request to add equivalent to=C2=A0DELETE_ON_FAILURE, please in=
clude your reproducer scripts.</div><div><br></div><div>-Bill</div></div><b=
r><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, =
Jul 16, 2025 at 10:44=E2=80=AFAM Tal Dayan <<a href=3D"mailto:tal@zapta.=
com" target=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Looking at the end sta=
te of scons after invocation #3, the actual md5 of file2 doesn't match =
its md5 in the dblite.<div><br></div><div><a href=3D"https://i.imgur.com/NG=
co3yQ.png" target=3D"_blank">https://i.imgur.com/NGco3yQ.png</a></div></div=
><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On We=
d, Jul 16, 2025 at 10:33=E2=80=AFAM Tal Dayan <<a href=3D"mailto:tal@zap=
ta.com" target=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote=
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so=
lid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Hi Keith, I updated=
the example files here=C2=A0<a href=3D"https://github.com/FPGAwars/apio/is=
sues/676" target=3D"_blank">https://github.com/FPGAwars/apio/issues/676</a>=
<div><br></div><div>They now include the md5 of the files and a dump of=C2=
=A0<span style=3D"font-variant-ligatures:no-common-ligatures;color:rgb(0,0,=
0);font-family:Menlo;font-size:12px">.sconsign.dblite</span></div>
</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">=
On Wed, Jul 16, 2025 at 10:08=E2=80=AFAM Keith Prussing <<a href=3D"mail=
to:[email protected]" target=3D"_blank">[email protected]</a>> w=
rote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p=
x 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I suspect =
it's because `file1` has the same hash in the<br>
.sconsign.dblite as the last "good" build (i.e. the first one). T=
hus<br>
you get the line "scons: `file2' is up to date." on the third=
run.<br>
However, I am not an expert in the specifics of SCons' hashing<br>
methods.<br>
<br>
On Wed, Jul 16, 2025 at 12:38=E2=80=AFPM Tal Dayan <<a href=3D"mailto:ta=
[email protected]" target=3D"_blank">[email protected]</a>> wrote:<br>
><br>
> Hi all,<br>
><br>
> We encountered this problem with the nextpnr tool and created here a s=
mall and independent example that demonstrates it.<br>
><br>
> In the example below, a shell script 'command.sh' reads the so=
urce file 'file1' and writes it to the target file 'file2'.=
However, if the<br>
> source file starts with 'bad' it exits with an error code, *af=
ter* creating the target file.<br>
><br>
> The script `run.sh', runs scons three times with these values of t=
he source file file1 'good', 'bad', and 'good'.=C2=
=A0 The expectation is that after the third scons run, file2 should contain=
the value 'good' but it contains the value 'bad'.<br>
><br>
> Do we miss anything or is it simply a bug?<br>
><br>
> SConstruct:<br>
><br>
> ----------------------------------------------<br>
><br>
> # SCons environment<br>
><br>
> env =3D Environment()<br>
><br>
><br>
> # Copy file1 =E2=86=92 file2 using command.sh<br>
><br>
> # Inject an error if file1 starts with 'bad"<br>
><br>
> file2 =3D env.Command(<br>
><br>
>=C2=A0 =C2=A0 =C2=A0target=3D'file2',<br>
><br>
>=C2=A0 =C2=A0 =C2=A0source=3D'file1',<br>
><br>
>=C2=A0 =C2=A0 =C2=A0action=3D'./command.sh $SOURCE > $TARGET'=
;<br>
><br>
> )<br>
><br>
><br>
> # Make 'file2' the default target<br>
><br>
> Default(file2)<br>
><br>
> ----------------------------------------------<br>
><br>
><br>
> command.sh:<br>
><br>
> ----------------------------------------------<br>
><br>
> #!/bin/bash<br>
><br>
> # Usage: ./command input > output<br>
><br>
><br>
> # Read from the first argument and copy to stdout<br>
><br>
> cat "$1"<br>
><br>
><br>
> # If the input file starts with 'bad', inject an error AFTER c=
reating the output file.<br>
><br>
><br>
> first_line=3D$(head -n 1 "$1")<br>
><br>
><br>
> if [[ "$first_line" =3D=3D bad* ]]; then<br>
><br>
>=C2=A0 =C2=A0exit 1<br>
><br>
> fi<br>
><br>
><br>
> exit 0<br>
><br>
> ----------------------------------------------<br>
><br>
><br>
><br>
> run.sh<br>
><br>
> ----------------------------------------------<br>
><br>
> #!/bin/bash<br>
><br>
><br>
> # Clean up.<br>
><br>
> rm -f .sconsign.dblite<br>
><br>
> rm -f file[12]<br>
><br>
><br>
> echo<br>
><br>
> echo "---- Iteration 1: file1 =3D 'good'"<br>
><br>
> echo "good" > file1<br>
><br>
> scons<br>
><br>
> echo<br>
><br>
><br>
> echo "File1"<br>
><br>
> cat -n file1<br>
><br>
><br>
> echo "File2"<br>
><br>
> cat -n file2<br>
><br>
><br>
><br>
> echo<br>
><br>
> echo "---- Iteration 2: file1 =3D 'bad'"<br>
><br>
> echo "bad" > file1<br>
><br>
> cat -n file1<br>
><br>
> scons<br>
><br>
> echo<br>
><br>
><br>
> echo "File1"<br>
><br>
> cat -n file1<br>
><br>
><br>
> echo "File2"<br>
><br>
> cat -n file2<br>
><br>
><br>
> echo<br>
><br>
> echo "---- Iteration 3: file1 =3D 'good'"<br>
><br>
> echo "good" > file1<br>
><br>
> cat -n file1<br>
><br>
> scons<br>
><br>
> echo<br>
><br>
><br>
> echo "File1"<br>
><br>
> cat -n file1<br>
><br>
><br>
> echo "File2"<br>
><br>
> cat -n file2<br>
><br>
> ----------------------------------------------<br>
><br>
><br>
><br>
> Run log:<br>
><br>
> ----------------------------------------------<br>
><br>
> $ ./run.sh<br>
><br>
><br>
> ---- Iteration 1: file1 =3D 'good'<br>
><br>
> scons: Reading SConscript files ...<br>
><br>
> scons: done reading SConscript files.<br>
><br>
> scons: Building targets ...<br>
><br>
> ./command.sh file1 > file2<br>
><br>
> scons: done building targets.<br>
><br>
><br>
> File1<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 good<br>
><br>
> File2<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 good<br>
><br>
><br>
> ---- Iteration 2: file1 =3D 'bad'<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 bad<br>
><br>
> scons: Reading SConscript files ...<br>
><br>
> scons: done reading SConscript files.<br>
><br>
> scons: Building targets ...<br>
><br>
> ./command.sh file1 > file2<br>
><br>
> scons: *** [file2] Error 1<br>
><br>
> scons: building terminated because of errors.<br>
><br>
><br>
> File1<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 bad<br>
><br>
> File2<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 bad<br>
><br>
><br>
> ---- Iteration 1: file1 =3D 'good'<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 good<br>
><br>
> scons: Reading SConscript files ...<br>
><br>
> scons: done reading SConscript files.<br>
><br>
> scons: Building targets ...<br>
><br>
> scons: `file2' is up to date.<br>
><br>
> scons: done building targets.<br>
><br>
><br>
> File1<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 good<br>
><br>
> File2<br>
><br>
>=C2=A0 =C2=A0 =C2=A0 1 bad<br>
><br>
> ----------------------------------------------<br>
><br>
> _______________________________________________<br>
> Scons-users mailing list<br>
> <a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users=
@scons.org</a><br>
> <a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" re=
l=3D"noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listi=
nfo/scons-users</a><br>
<br>
<br>
<br>
-- <br>
Keith Prussing<br>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
_______________________________________________<br>
Scons-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Scons-users@scon=
s.org</a><br>
<a href=3D"https://pairlist4.pair.net/mailman/listinfo/scons-users" rel=3D"=
noreferrer" target=3D"_blank">https://pairlist4.pair.net/mailman/listinfo/s=
cons-users</a><br>
</blockquote></div>
--000000000000d87cc8063a1967ed--
--===============7563494951548632737==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users
--===============7563494951548632737==--