Re: Scons none-deterministic behavior for incremental build

Does this happen if you use python 2.7?

On Tue, Dec 4, 2018 at 6:00 PM Hua Yanghao <[email protected]> wrote:

> Apparently the MD5 signature is taking *turns* some how.
>
> You can see below the old md5 and new md5 checksum is somehow
> alternating between two fixed value ...
> How is scons calculating the md5 value? is it only depends on the file
> content?
> The md5sum of the "arch_init.c" is:
> md5sum firmware/arch/armv7/arch_init.c
> db624fbb365812d7ba58c87d49f44b6f  firmware/arch/armv7/arch_init.c
>
> First build:
> e494890ea1b6e36acf275f3698a3f3f9
> c706a016770a7c85ac42f16f79456797
> Returning True
> in self.changed ... 2
> return not r: False
> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> <SCons.Script.Main.BuildTask object at 0x7fdc934c7fd0>
> ______CC
> build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
>
> Second build:
> c706a016770a7c85ac42f16f79456797
> e494890ea1b6e36acf275f3698a3f3f9
> Returning True
> in self.changed ... 2
> return not r: False
> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> <SCons.Script.Main.BuildTask object at 0x7f3fca5e4fd0>
> ______CC
> build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
> On Tue, Dec 4, 2018 at 11:54 PM Hua Yanghao <[email protected]> wrote:
> >
> > That is generated by sconstruct, not from shell. Any potential issue
> > here? (It was working for a very long time before and only breaks
> > recently when I added some new features).
> >
> > In the meantime I have isolated the issue to be in Node.Node.changed()
> > call, where scons old & new signature for the C files seems not
> > matching (print("###### 3.1") is executed in error case):
> > 1463         if self.has_builder():
> >                                                           |3
> > 1464             print("####### 3")
> >                                                           |3
> > 1465             import SCons.Util
> >                                                           |3
> > 1466             newsig = SCons.Util.MD5signature(contents)
> >                                                           |3
> > 1467             if bi.bactsig != newsig:
> >                                                           |3
> > 1468                 if t: Trace(': bactsig %s != newsig %s' %
> > (bi.bactsig, newsig))                                             |3
> > 1469                 print("####### 3.1")
> >                                                           |3
> > 1470                 result = True
> > On Tue, Dec 4, 2018 at 11:24 PM Bill Deegan <[email protected]>
> wrote:
> > >
> > > Generated by logic in your sconstruct/sconscripts? or do you shell out
> and run a script/program to generate it.
> > >
> > >
> > > On Tue, Dec 4, 2018 at 4:37 PM Hua Yanghao <[email protected]>
> wrote:
> > >>
> > >> Hi Bill,
> > >> config.h is generated file, not checked in, it is generated in the
> > >> build folder only.
> > >> Well at least I see scons is very odering-sensitive. For example, if
> > >> the order of SConscript processing changes (in the case of a set()),
> > >> even if there is no dependency actually changed, scons thinks the
> > >> target need to be rebuilt.
> > >>
> > >> any progress on these similar issues already? :)
> > >>
> > >> Best Regards,
> > >> Yanghao
> > >> On Tue, Dec 4, 2018 at 10:15 PM Bill Deegan <
> [email protected]> wrote:
> > >> >
> > >> > Is "build/configs/qemu_arm_vexpress/arm3/config/config.h" a
> generated file? Or checked in?
> > >> >
> > >> > It's very unlikely to be set ordering. set's should only be used
> for membership checks not iteration in the SCons codebase.
> > >> >
> > >> > We're actively debugging similar issue.
> > >> >
> > >> > -Bill
> > >> >
> > >> > On Tue, Dec 4, 2018 at 3:23 PM Hua Yanghao <[email protected]>
> wrote:
> > >> >>
> > >> >> I does that a lot ... but a quick try to make sure no
> > >> >> LIBS/LIBPATH/CPPPATH etc. is touched after Object() is being
> called,
> > >> >> still shows the same 50~70% rebuild of the entire project.
> > >> >> I am not using any parallel build, and somehow scons is having a
> very
> > >> >> none-deterministic behavior.
> > >> >>
> > >> >> Actually I fixed partially the issue (e.g. in simpler build the
> > >> >> incremental builds now works fine), due to python set() is having a
> > >> >> random order for every run. I am thinking maybe scons is somehow
> also
> > >> >> depends on the ordering of a set()? And when it happens to be the
> same
> > >> >> order then no rebuild?
> > >> >> On Tue, Dec 4, 2018 at 9:00 PM Oscar Anson <[email protected]>
> wrote:
> > >> >> >
> > >> >> > Hi,
> > >> >> >
> > >> >> > Not sure if this could be related, but a similar issue happened
> to me where I get a relink of a shared library without changing anything in
> the project. It happened that I was setting the environment LIBS and
> LIBPATH after calling the builder of the library.
> > >> >> >
> > >> >> > El mar., 4 dic. 2018 20:55, Hua Yanghao <[email protected]>
> escribió:
> > >> >> >>
> > >> >> >> Dear Scons Experts,
> > >> >> >> I recently hit on a very strange problem where scons incremental
> > >> >> >> builds sometimes OK, sometimes the entire project is re-built.
> > >> >> >>
> > >> >> >> scons: 3.0.1
> > >> >> >> python: 3.6.7
> > >> >> >>
> > >> >> >> --debug=explain gives below example trace:
> > >> >> >> scons: rebuilding
> > >> >> >> `build/configs/qemu_arm_vexpress/arm3/config/config.h' because
> > >> >> >> AlwaysBuild() is specified
> > >> >> >> __CONFIG build/configs/qemu_arm_vexpress/arm3/config/config.h
> > >> >> >> scons: rebuilding
> > >> >> >>
> `build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o'
> > >> >> >> because:
> > >> >> >>            `firmware/arch/armv7/arch_init.c' is no longer a
> dependency
> > >> >> >>            `firmware/arch/armv7/arch_init.c' is a new dependency
> > >> >> >>
> > >> >> >> It seems some how the C file itself is removed and re-added as
> the
> > >> >> >> Object() dependency, and causes rebuild. However this is *NOT*
> > >> >> >> *always* the case. Sometime the rebuild doesn't happen.
> > >> >> >>
> > >> >> >> I failed to create a minimal example that can show this issue
> so far,
> > >> >> >> but in the production environment it is about 50% full-rebuild
> and 50%
> > >> >> >> incremental re-build.
> > >> >> >>
> > >> >> >> Any thoughts or hints?
> > >> >> >>
> > >> >> >> Thanks,
> > >> >> >> Yanghao
> > >> >> >> _______________________________________________
> > >> >> >> Scons-users mailing list
> > >> >> >> [email protected]
> > >> >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >> >
> > >> >> > _______________________________________________
> > >> >> > Scons-users mailing list
> > >> >> > [email protected]
> > >> >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >> _______________________________________________
> > >> >> Scons-users mailing list
> > >> >> [email protected]
> > >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >
> > >> > _______________________________________________
> > >> > Scons-users mailing list
> > >> > [email protected]
> > >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> _______________________________________________
> > >> Scons-users mailing list
> > >> [email protected]
> > >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >
> > > _______________________________________________
> > > Scons-users mailing list
> > > [email protected]
> > > https://pairlist4.pair.net/mailman/listinfo/scons-users
> _______________________________________________
> Scons-users mailing list
> [email protected]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users