Re: Scons Implicity Dependency not found issue

Replied to the wrong threads .... re-attach it here.

Any thoughts? This seems really like when you move around some header
files scons is using some cached knowledge from last built which
caused the incremental built to fail.
Not sure if the trace file is useful enough already, I could try to
create a minimal suite of files that actually reproduce this issue too
if needed.

On Wed, Oct 17, 2018 at 10:37 PM Hua Yanghao <[email protected]> wrote:

> However, if on commit Y you do a clean and re-build, everything passes.
> Seems some cached state is in play here?
>
> On Wed, Oct 17, 2018 at 10:35 PM Hua Yanghao <[email protected]> wrote:
>
>> It looks like this time even though I reproduced the issue it is not
>> really sticky and always passes on a second run.
>> I have re-run with the trace: https://pastebin.com/2ziqM4ri
>>
>> The part that is important is there is one file, e.g. included <io.h>,
>> and in one version io.h is in folder A/io.h. And then there is a
>> restructure of file and io.h moved to folder B/io.h.
>> Folder A is in search path at commit X, build pass. Folder B is in search
>> path at commit Y, build pass too.
>>
>> However if you first build on commit X (pass) and then switch to commit Y
>> and build again, it fails like above.
>>
>> @Bill Deegan <[email protected]>  please help to take a look the
>> trace file and let me know if more experiment I can do. At least now I can
>> reproduce it reliably on-demand.
>>
>> On Wed, Oct 17, 2018 at 10:06 PM Hua Yanghao <[email protected]>
>> wrote:
>>
>>> I finally reproduced this issue, however this
>>> --taskmastertrace=trace.log the compilation passes instead of failing.
>>> Thoughts?
>>>
>>> hua@huyangha-mobl:~/git/usw-abc $ ./run configs/pangu.py
>>> Running : ./tools/scons -Q config=configs/pangu.py
>>> INFO: Processing target usw ... Done.
>>> INFO: usw_config completed in 0.19 seconds.
>>> __CONFIG build/configs/pangu/usw/config/config.h
>>> ____SHCC build/configs/pangu/usw/firmware/arch/weak.os
>>> ______AR build/configs/pangu/usw/firmware/arch/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/arch/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/arch/pangu/lib.os
>>> ____SHCC build/configs/pangu/usw/firmware/arch/pangu/main.os
>>> scons: *** [build/configs/pangu/usw/firmware/arch/pangu/test.os]
>>> Implicit dependency
>>> `build/configs/pangu/usw/firmware/arch/pangu/include/io.h' not found,
>>> needed by target `build/configs/pangu/usw/firmware/arch/pangu/test.os'.
>>> hua@huyangha-mobl:~/git/usw-abc $ ./run configs/pangu.py
>>> --taskmastertrace=trace.log
>>> Running : ./tools/scons -Q config=configs/pangu.py
>>> --taskmastertrace=trace.log
>>> INFO: Processing target usw ... Done.
>>> INFO: usw_config completed in 0.21 seconds.
>>> __CONFIG build/configs/pangu/usw/config/config.h
>>> ____SHCC build/configs/pangu/usw/firmware/arch/pangu/test.os
>>> ____SHCC build/configs/pangu/usw/firmware/boards/weak.os
>>> ______AR build/configs/pangu/usw/firmware/boards/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/boards/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/boards/pangu/board.os
>>> ____SHCC build/configs/pangu/usw/firmware/common/cpu.os
>>> ____SHCC build/configs/pangu/usw/firmware/common/weak.os
>>> ____SHCC build/configs/pangu/usw/firmware/common/log.os
>>> ______AR build/configs/pangu/usw/firmware/common/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/common/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/common/main.os
>>> ____SHCC build/configs/pangu/usw/firmware/drivers/pcie/dwc_ep.os
>>> ______AR build/configs/pangu/usw/firmware/drivers/pcie/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/drivers/pcie/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/lib/cmd/cmd.os
>>> ____SHCC build/configs/pangu/usw/firmware/lib/cmd/test.os
>>> ____SHCC build/configs/pangu/usw/firmware/lib/fio/cmd.os
>>> ____SHCC build/configs/pangu/usw/firmware/lib/fio/usw_device.os
>>> ____SHCC build/configs/pangu/usw/firmware/lib/fio/linux64_device.os
>>> ______AR build/configs/pangu/usw/firmware/lib/fio/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/lib/fio/libdefault.o
>>> ____SHCC
>>> build/configs/pangu/usw/firmware/lib/simple_console/simple_console.os
>>> ______AR build/configs/pangu/usw/firmware/lib/simple_console/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/lib/simple_console/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/lib/simple_pipe/connect.os
>>> ____SHCC build/configs/pangu/usw/firmware/lib/simple_pipe/pipe.os
>>> ______AR build/configs/pangu/usw/firmware/lib/simple_pipe/libdefault.o
>>> __RANLIB build/configs/pangu/usw/firmware/lib/simple_pipe/libdefault.o
>>> ____SHCC build/configs/pangu/usw/firmware/test/console/hello.os
>>> ____SHCC build/configs/pangu/usw/firmware/test/console/test.os
>>> ____SHCC build/configs/pangu/usw/firmware/test/example/test.os
>>> ____SHLD build/configs/pangu/usw/libusw.so
>>> _USWHASH build/configs/pangu/usw/libusw.so
>>> _USWHASH build/configs/pangu/usw/libusw.so --> Changed
>>> hua@huyangha-mobl:~/git/usw-abc $
>>>
>>> On Tue, Oct 16, 2018 at 11:10 PM Hua Yanghao <[email protected]>
>>> wrote:
>>>
>>>> Config.h does not include any other headers. And the problem is it is
>>>> not easily reproducible. However once it happens it is sticky. I will try
>>>> harder tomorrow and get back to you. Thanks.
>>>>
>>>> On Tue 16. Oct 2018 at 23:06, Bill Deegan <[email protected]>
>>>> wrote:
>>>>
>>>>> Try running with
>>>>>
>>>>> --taskmastertrace=trace.log
>>>>>
>>>>> You'll get a pretty verbose log there, and search for io.h it should
>>>>> show more information about what's happening.
>>>>>
>>>>> Is io.h included via "-include config.h" ?
>>>>>
>>>>> -Bill
>>>>>
>>>>> On Tue, Oct 16, 2018 at 5:01 PM Hua Yanghao <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi Bill,
>>>>>> Sorry I missed that one.
>>>>>>
>>>>>> The full command line when hit the failure:
>>>>>> gcc -o build/configs/pangu/usw/firmware/arch/pangu/test.os -c -g
>>>>>> -Wall -Werror -include config.h -Wno-error=main -Wno-main
>>>>>> -Wno-format-security -Wno-format -std=gnu99 -D_USW_FLAGS_TEST_0
>>>>>> -D_USW_FLAGS_TEST_1 -D_USW_FLAGS_TEST_2 -fPIC
>>>>>> -Ibuild/configs/pangu/usw/config
>>>>>> -Ibuild/configs/pangu/usw/firmware/arch/pangu/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/arch/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/common/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/cmd/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/simple_pipe/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/fio/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/simple_console/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/test/example/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/test/console/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/boards/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/boards/pangu/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/drivers/pcie/include
>>>>>> -Ibuild/configs/pangu/usw/config
>>>>>> -Ibuild/configs/pangu/usw/firmware/arch/pangu/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/arch/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/common/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/cmd/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/simple_pipe/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/fio/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/lib/simple_console/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/test/example/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/test/console/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/boards/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/boards/pangu/include
>>>>>> -Ibuild/configs/pangu/usw/firmware/drivers/pcie/include
>>>>>> build/configs/pangu/usw/firmware/arch/pangu/test.c
>>>>>>
>>>>>> And my CPPPATH seems also matching the command line:
>>>>>> ['/home/hua/git/usw/build/configs/pangu/usw/config',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/arch/pangu/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/arch/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/common/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/cmd/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/simple_pipe/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/fio/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/simple_console/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/test/example/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/test/console/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/boards/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/boards/pangu/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/drivers/pcie/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/config',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/arch/pangu/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/arch/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/common/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/cmd/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/simple_pipe/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/fio/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/lib/simple_console/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/test/example/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/test/console/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/boards/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/boards/pangu/include',
>>>>>> '/home/hua/git/usw/build/configs/pangu/usw/firmware/drivers/pcie/include']
>>>>>>
>>>>>> Note the firmware/include is in both command line and CPPPATH, that's
>>>>>> where my io.h is located.
>>>>>>
>>>>>> Best Regards,
>>>>>> Yanghao
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 16, 2018 at 8:07 PM Bill Deegan <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> You've still not replied with the compile command line.
>>>>>>> Also what's your CPPPATH?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 16, 2018 at 1:25 PM Hua Yanghao <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> it is the correct path and io.h is actually duplicated from
>>>>>>>> firmware/include/io.h
>>>>>>>>
>>>>>>>> On Tue 16. Oct 2018 at 17:34, Bill Deegan <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Is `build/configs/pangu/usw/firmware/arch/pangu/include/io.h' the
>>>>>>>>> correct path to io.h?
>>>>>>>>> Are you using duplicate=0, or not specifying in you SConscript()?
>>>>>>>>> Are you using chdir somewhere in your SCons logic?
>>>>>>>>>
>>>>>>>>> Also, please paste the compile line for the compile which should
>>>>>>>>> find io.h when it's passing?
>>>>>>>>>
>>>>>>>>> On Tue, Oct 16, 2018 at 11:14 AM Hua Yanghao <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> no it is static file.
>>>>>>>>>>
>>>>>>>>>> On Tue 16. Oct 2018 at 17:08, Bill Deegan <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Is io.h a generated file?
>>>>>>>>>>> Paste the compile line for the compile which should find io.h
>>>>>>>>>>> when it's passing?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Oct 16, 2018 at 8:54 AM Hua Yanghao <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Bill,
>>>>>>>>>>>> Yes io.h is on my CPPPATH. And I am using variant_dir="build"
>>>>>>>>>>>> for
>>>>>>>>>>>> SConscript files. So normally all used header files will end up
>>>>>>>>>>>> in the
>>>>>>>>>>>> build folder too. But occasionally scons seems missed it. I am
>>>>>>>>>>>> struggling to reproduce it ...
>>>>>>>>>>>>
>>>>>>>>>>>> BR, Yanghao
>>>>>>>>>>>> On Tue, Oct 16, 2018 at 2:51 PM Bill Deegan <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Yes. Headers are scanned recursively.
>>>>>>>>>>>> > Implicit cache should only be enable by command line flag or
>>>>>>>>>>>> SetOption()
>>>>>>>>>>>> >
>>>>>>>>>>>> > Where is io.h located? is it in your CPPPATH?
>>>>>>>>>>>> >
>>>>>>>>>>>> > On Tue, Oct 16, 2018 at 6:43 AM Hua Yanghao <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> By the way, is scons scanner scans C file headers
>>>>>>>>>>>> recursively? Thanks.
>>>>>>>>>>>> >> On Tue, Oct 16, 2018 at 12:40 PM Hua Yanghao <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>> >> >
>>>>>>>>>>>> >> > scons: ***
>>>>>>>>>>>> [build/configs/pangu/usw/firmware/arch/pangu/test.os]
>>>>>>>>>>>> >> > Implicit dependency
>>>>>>>>>>>> >> > `build/configs/pangu/usw/firmware/arch/pangu/include/io.h'
>>>>>>>>>>>> not found,
>>>>>>>>>>>> >> > needed by target
>>>>>>>>>>>> `build/configs/pangu/usw/firmware/arch/pangu/test.os'
>>>>>>>>>>>> >> >
>>>>>>>>>>>> >> > I recently run into this odd problem, and it is very
>>>>>>>>>>>> difficult to
>>>>>>>>>>>> >> > reproduce. I did not explicitly enable the
>>>>>>>>>>>> "implicit_cache" option
>>>>>>>>>>>> >> > anywhere in my code. If I simply remove the build folder
>>>>>>>>>>>> and then
>>>>>>>>>>>> >> > re-build everything works as normal. the problem happens
>>>>>>>>>>>> when there is
>>>>>>>>>>>> >> > a old build, and then I pulled in some new code where this
>>>>>>>>>>>> "test.c"
>>>>>>>>>>>> >> > has included an additional header io.h and scons seems not
>>>>>>>>>>>> re-scanning
>>>>>>>>>>>> >> > the file by default.
>>>>>>>>>>>> >> >
>>>>>>>>>>>> >> > I am using scons 3.0.1, is the implict_cache by default
>>>>>>>>>>>> enabled there
>>>>>>>>>>>> >> > and I have to disable it manually?
>>>>>>>>>>>> >> >
>>>>>>>>>>>> >> > Thanks,
>>>>>>>>>>>> >> > Yanghao Hua
>>>>>>>>>>>> >> _______________________________________________
>>>>>>>>>>>> >> Scons-users mailing list
>>>>>>>>>>>> >> [email protected]
>>>>>>>>>>>> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>> >
>>>>>>>>>>>> > _______________________________________________
>>>>>>>>>>>> > Scons-users mailing list
>>>>>>>>>>>> > [email protected]
>>>>>>>>>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>>> [email protected]
>>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>> [email protected]
>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-users mailing list
>>>>>>>>>> [email protected]
>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Scons-users mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Scons-users mailing list
>>>>>>>> [email protected]
>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Scons-users mailing list
>>>>>>> [email protected]
>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> [email protected]
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> [email protected]
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users