Re: 回复： 回复： 回复： 回复： 回复： 回复： Ninja compil ation fails after using include path file instead of " -I" option

Daniel Moody <[email protected]>
Newsgroups gmane.comp.programming.tools.scons.user
Message-ID <CANws3JSOnjqKtZBGXpsrbGenvi+YH3FZX0NtkNLhWzuMxFnk=A@mail.gmail.com> 
Hello liruncong,

I have a test branch for the issues you described where re-running scons is
not re-compiling, could you please try it out?:
https://github.com/dmoody256/scons/tree/ninja_integration_branch

On Sun, Apr 17, 2022 at 2:29 AM liruncong2018 <[email protected]> wrote:

> If execute the command "scons -j%NUMBER_OF_PROCESSORS% generate-ninja
> --verbose --experimental=ninja", it will be completely recompiled.
> If execute the command again, then it will be completely recompiled.
> Each time is equivalent to the first compilation.
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Daniel Moody" <[email protected]>;
> *发送时间:* 2022年4月17日(星期天) 中午1:52
> *收件人:* "liruncong2018"<[email protected]>;
> *抄送:* "Bill Deegan"<[email protected]>;"SCons users mailing list"<
> [email protected]>;
> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
> after using include path file instead of "-I" option
>
> If you use the command line target 'generate-ninja' for the scons command,
> does it still fail to build and execute ninja on the second run?
>
>
> On Sat, Apr 16, 2022 at 10:36 PM liruncong2018 <[email protected]>
> wrote:
>
>> Great, looking forward to your PR!
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Daniel Moody" <[email protected]>;
>> *发送时间:* 2022年4月17日(星期天) 中午11:31
>> *收件人:* "liruncong2018"<[email protected]>;
>> *抄送:* "Bill Deegan"<[email protected]>;"SCons users mailing
>> list"<[email protected]>;
>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>> after using include path file instead of "-I" option
>>
>> Okay that indicates that scons is not deciding to build and execute the
>> ninja file, probably because nothing has changed in the build scripts when
>> you only change the source file. This is a bug, even if the ninja file is
>> not regened, scons should execute it for you. I will make a PR for this
>> issue. For now you will need to use the run_ninja_env.bat.
>>
>>
>> On Sat, Apr 16, 2022 at 9:54 PM liruncong2018 <[email protected]>
>> wrote:
>>
>>> Hi,
>>> When I just execute run_ninja_env.bat, it shows compile error. It seems
>>> that there is no problem with ninja, the problem should be that scons does
>>> not call run_ninja_env.bat to execute. Now how do I diagnose this problem
>>> with scons?
>>> ------------------ 原始邮件 ------------------
>>> *发件人:* "Daniel Moody" <[email protected]>;
>>> *发送时间:* 2022年4月17日(星期天) 上午10:28
>>> *收件人:* "liruncong2018"<[email protected]>;
>>> *抄送:* "Bill Deegan"<[email protected]>;"SCons users mailing
>>> list"<[email protected]>;
>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>> after using include path file instead of "-I" option
>>>
>>> When I test with the repo https://github.com/liruncong/NinJaTest for
>>> bsp/beaglebone build using the changes from here:
>>> https://github.com/SCons/scons/pull/4133, and I put an 'a' in the
>>> src/irq.c file, I see the file error when I rerun the build.
>>>
>>> Could you specify what file you changed?
>>>
>>> Also on subsequent runs, you can just run_ninja_env.bat, it will skip
>>> scons regeneration, but if you want scons to regen it should work also.
>>>
>>> On Sat, Apr 16, 2022 at 8:46 PM liruncong2018 <[email protected]>
>>> wrote:
>>>
>>>> Hi,
>>>> After the first compilation is successful, modify the file, expect the
>>>> compilation to fail, but compile again without error message.
>>>> The following is the log after the command("scons
>>>> -j%NUMBER_OF_PROCESSORS% --verbose --experimental=ninja") is executed:
>>>> ______________________________________
>>>> scons: Reading SConscript files ...
>>>> scons: done reading SConscript files.
>>>> scons: Building targets ...
>>>> scons: done building targets.
>>>> ______________________________________
>>>> I have attached the compressed "build.ninja" file in the attachment.
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> *发件人:* "Daniel Moody" <[email protected]>;
>>>> *发送时间:* 2022年4月17日(星期天) 凌晨1:53
>>>> *收件人:* "Bill Deegan"<[email protected]>;
>>>> *抄送:* "SCons users mailing list"<[email protected]
>>>> >;"liruncong2018"<[email protected]>;
>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>> fails after using include path file instead of "-I" option
>>>>
>>>> Also i realized that link_objects.txt is probably not meant for the
>>>> object builders. I assume this file is intended for linking the libraries
>>>> and programs? In that case you will want a scanner for those builders
>>>> instead. Do you have error messages you can show, it should help to see
>>>> what your issue is.
>>>>
>>>> On Sat, Apr 16, 2022, 10:58 AM Bill Deegan <[email protected]>
>>>> wrote:
>>>>
>>>>> try bzip'ing the ninja.build file. That should make it considerably
>>>>> smaller.
>>>>>
>>>>> On Sat, Apr 16, 2022 at 4:37 PM liruncong2018 via Scons-users <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Still the same problem. I want to send "build.ninja" as an
>>>>>> attachment, but the email always bounces, what information can I print so I
>>>>>> can find the problem?
>>>>>>
>>>>>> ------------------ 原始邮件 ------------------
>>>>>> *发件人:* "Daniel Moody" <[email protected]>;
>>>>>> *发送时间:* 2022年4月16日(星期六) 凌晨0:29
>>>>>> *收件人:* "liruncong2018"<[email protected]>;
>>>>>> *抄送:* "SCons users mailing list"<[email protected]>;
>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>>>>> after using include path file instead of "-I" option
>>>>>>
>>>>>> Here's the same code but better color scheme:
>>>>>>
>>>>>> incPathsFile = env.Textfile(
>>>>>>     "build/__cpp_path.txt",
>>>>>>     cppPathStr
>>>>>> )
>>>>>>
>>>>>> defineOptsFile = env.Textfile(
>>>>>>     "build/__define_options.txt",
>>>>>>     "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>> )
>>>>>>
>>>>>> linkObjsFile = env.Textfile(
>>>>>>     "build/__link_objects.txt",
>>>>>>     "\n".join(objsList)
>>>>>> )
>>>>>>
>>>>>> def _add_scanner(builder):
>>>>>>     def new_scanner(node, env, path):
>>>>>>         return incPathsFile + defineOptsFile + linkObjsFile
>>>>>>
>>>>>>     builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>         function=new_scanner,
>>>>>>         path_function=[],
>>>>>>     )
>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>     _add_scanner(object_builder)
>>>>>>
>>>>>> On Fri, Apr 15, 2022 at 10:50 AM Daniel Moody <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> hello liruncong,
>>>>>>>
>>>>>>> If you want to add multiple @file dependencies via the scanner do
>>>>>>> them all in the same scanner. Like I said you can only apply one scanner,
>>>>>>> so in this case you are overwriting each previous scanner. The scanner
>>>>>>> should return a list of files in which to set dependencies for the current
>>>>>>> node.
>>>>>>>
>>>>>>> The path function is not used in this case, and I just left the
>>>>>>> CPPPATH from the default SourceFileScanner I copied the scanner code from.
>>>>>>> It does have to be something besides None I believe though, or scons won't
>>>>>>> scan. The correct way this should be used in this case is you should place
>>>>>>> the @file in a known list of directories or just a single directory, and
>>>>>>> then pass a list of directories or a list containing the single directory,
>>>>>>> and the scanner should find the files by extension in the list of
>>>>>>> directories. This is the fundamental working of scanners if you want to
>>>>>>> read more:
>>>>>>> https://scons.org/doc/production/HTML/scons-user/ch20.html, however
>>>>>>> you could also use a dummy list if there is no dynamic component here.
>>>>>>>
>>>>>>> Here is an updated example based on the new info you provided:
>>>>>>>
>>>>>>> incPathsFile = env.Textfile(
>>>>>>>     "build/__cpp_path.txt",
>>>>>>>     cppPathStr
>>>>>>> )
>>>>>>>
>>>>>>> defineOptsFile = env.Textfile(
>>>>>>>     "build/__define_options.txt",
>>>>>>>     "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>>> )
>>>>>>>
>>>>>>> linkObjsFile = env.Textfile(
>>>>>>>     "build/__link_objects.txt",
>>>>>>>     "\n".join(objsList)
>>>>>>> )
>>>>>>>
>>>>>>> def _add_scanner(builder):
>>>>>>>     def new_scanner(node, env, path):
>>>>>>>         return incPathsFile + defineOptsFile + linkObjsFile
>>>>>>>
>>>>>>>     builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>         function=new_scanner,
>>>>>>>         path_function=[],
>>>>>>>     )
>>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>     _add_scanner(object_builder)
>>>>>>>
>>>>>>> On Fri, Apr 15, 2022 at 9:58 AM liruncong2018 <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Can you give me more information? What source file? Does the
>>>>>>>> compilation fail the first time? Does it recompile successfully with only
>>>>>>>> scons building? Can you push your latest code so I can see it in your repo?
>>>>>>>> -- I don't know how to reproduce this problem in the git test
>>>>>>>> project, I will try to see if it can be reproduced.
>>>>>>>> Also each builder can only hold one target or source scanner, but
>>>>>>>> you can chain the scanner by calling other scanners from within the single
>>>>>>>> scanner. Are you assigning multiple scanners to the same builder?
>>>>>>>> -- The following are some scanners, I don't quite understand the
>>>>>>>> path_function, especially the second parameter of the last scanner
>>>>>>>> (linkObjsFile).
>>>>>>>> ————————————————————————————————————————
>>>>>>>>     incPathsFile = env.Textfile(
>>>>>>>>         "build/__cpp_path.txt",
>>>>>>>>         cppPathStr
>>>>>>>>     )
>>>>>>>>     def _add_scanner(builder):
>>>>>>>>         def new_scanner(node, env, path):
>>>>>>>>             return incPathsFile
>>>>>>>>
>>>>>>>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>>             function=new_scanner,
>>>>>>>>             path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>>>>>>>         )
>>>>>>>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>>         _add_scanner(object_builder)
>>>>>>>>
>>>>>>>>     defineOptsFile = env.Textfile(
>>>>>>>>         "build/__define_options.txt",
>>>>>>>>         "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>>>>     )
>>>>>>>>
>>>>>>>>     def _add_scanner(builder):
>>>>>>>>         def new_scanner(node, env, path):
>>>>>>>>             return defineOptsFile
>>>>>>>>
>>>>>>>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>>             function=new_scanner,
>>>>>>>>             path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>>>>>>>         )
>>>>>>>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>>         _add_scanner(object_builder)
>>>>>>>>
>>>>>>>>     linkObjsFile = env.Textfile(
>>>>>>>>         "build/__link_objects.txt",
>>>>>>>>         "\n".join(objsList)
>>>>>>>>     )
>>>>>>>>
>>>>>>>>     def _add_scanner(builder):
>>>>>>>>         def new_scanner(node, env, path):
>>>>>>>>             return linkObjsFile
>>>>>>>>
>>>>>>>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>>             function=new_scanner,
>>>>>>>>             path_function=SCons.Script.FindPathDirs('LIBPATH'),
>>>>>>>>         )
>>>>>>>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>>         _add_scanner(object_builder)
>>>>>>>> ————————————————————————————————————————————————
>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>> *发件人:* "Daniel Moody" <[email protected]>;
>>>>>>>> *发送时间:* 2022年4月14日(星期四) 晚上11:36
>>>>>>>> *收件人:* "liruncong2018"<[email protected]>;
>>>>>>>> *抄送:* "SCons users mailing list"<[email protected]>;
>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>
>>>>>>>> Can you give me more information? What source file? Does the
>>>>>>>> compilation fail the first time? Does it recompile successfully with only
>>>>>>>> scons building? Can you push your latest code so I can see it in your repo?
>>>>>>>>
>>>>>>>>  Also each builder can only hold one target or source scanner, but
>>>>>>>> you can chain the scanner by calling other scanners from within the single
>>>>>>>> scanner. Are you assigning multiple scanners to the same builder?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Apr 14, 2022 at 10:17 AM liruncong2018 <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Hello Daniel Moody,
>>>>>>>>> Thanks a lot for the example you gave. According to your example,
>>>>>>>>> I added 4 scanners, which can indeed be compiled and linked.
>>>>>>>>> But when I edit the source file, I deliberately put "a" at the end
>>>>>>>>> of the .cpp file and expect the compilation to fail. But when I compile
>>>>>>>>> again, nothing compiles.
>>>>>>>>> How to diagnose this problem?
>>>>>>>>>
>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>> *发件人:* "Daniel Moody" <[email protected]>;
>>>>>>>>> *发送时间:* 2022年4月14日(星期四) 凌晨2:27
>>>>>>>>> *收件人:* "liruncong2018"<[email protected]>;
>>>>>>>>> *抄送:* "SCons users mailing list"<[email protected]>;
>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>>
>>>>>>>>> I added an example how to hook up the dependencies:
>>>>>>>>>
>>>>>>>>> https://github.com/liruncong/NinJaTest/pull/1/files
>>>>>>>>>
>>>>>>>>> On Wed, Apr 13, 2022 at 10:01 AM liruncong2018 <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I generate some option files via env.Textfile
>>>>>>>>>> ("build/__cpp_path.txt", "build/__define_options.txt",
>>>>>>>>>> "build/__link_objects.txt", "build/__exe_cpp_path.txt") and make objs
>>>>>>>>>> depend on corresponding files ,for example:
>>>>>>>>>> env.Depends(objs, ["build/__cpp_path.txt",
>>>>>>>>>> "build/__define_options.txt" ])
>>>>>>>>>> env.Depends(target, "build/__link_objects.txt")
>>>>>>>>>> But after enabling ninja, these manually added dependencies are
>>>>>>>>>> ignored, which causes ninja to fail to compile.
>>>>>>>>>> How to deal with this?
>>>>>>>>>>
>>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>>> *发件人:* "Daniel Moody" <[email protected]>;
>>>>>>>>>> *发送时间:* 2022年4月13日(星期三) 凌晨0:02
>>>>>>>>>> *收件人:* "SCons users mailing list"<[email protected]>;
>>>>>>>>>> *抄送:* "liruncong2018"<[email protected]>;
>>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>>>
>>>>>>>>>> Hello, that PR is meant to support shared @file with ninja, not
>>>>>>>>>> intended for any improvements to TEMPFILE. Please test without TEMPFILE.
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 12, 2022 at 10:52 AM liruncong2018 via Scons-users <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>> I tested "https://github.com/SCons/scons/pull/4133", after
>>>>>>>>>>> enabling ningja, except for the link failure, .c/.cpp/.S are successfully
>>>>>>>>>>> compiled into .o, the total time is 6:48 .
>>>>>>>>>>> This time is much greater than the previous time of 3:45 when
>>>>>>>>>>> ninja was not enabled and the option file was shared using @file.
>>>>>>>>>>> After using TEMPFILE instead of @file, the compilation time is
>>>>>>>>>>> basically doubled whether ninja is enabled or not.
>>>>>>>>>>> The TEMPFILE function should be improved to use shared files to
>>>>>>>>>>> prevent the creation of a large number of temporary files.
>>>>>>>>>>>
>>>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>>>> *发件人:* "SCons users mailing list" <[email protected]>;
>>>>>>>>>>> *发送时间:* 2022年4月11日(星期一) 晚上11:43
>>>>>>>>>>> *收件人:* "SCons users mailing list"<[email protected]>;
>>>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： Ninja compilation fails
>>>>>>>>>>> after using include path file instead of "-I" option
>>>>>>>>>>>
>>>>>>>>>>> I have a potential fix to the ninja tool so that it can decide
>>>>>>>>>>> to use response files or not. This should allow you to use your own shared
>>>>>>>>>>> response file.
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/SCons/scons/pull/4133
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 11, 2022 at 10:01 AM Mats Wichmann <[email protected]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 4/11/22 08:57, liruncong2018 via Scons-users wrote:
>>>>>>>>>>>> > Hi,
>>>>>>>>>>>> > Even using msys2 does not solve all problems. The 32K limit
>>>>>>>>>>>> solves my
>>>>>>>>>>>> > compilation problems (.cpp -> .o) but not linking (.o ->
>>>>>>>>>>>> target) because
>>>>>>>>>>>> > the link command line has around 137K (about 2500 .o files).
>>>>>>>>>>>> So
>>>>>>>>>>>> > compilers such as gcc/clang/armclang will support "@file".
>>>>>>>>>>>> > Currently, scons needs to set up Depends when using @file.
>>>>>>>>>>>> This
>>>>>>>>>>>> > dependency suggestion is still directly supported by scons,
>>>>>>>>>>>> and users do
>>>>>>>>>>>> > not need to write dependencies. It should also be supported
>>>>>>>>>>>> when ninja
>>>>>>>>>>>> > is enabled.
>>>>>>>>>>>> > When LINKCOM uses "TEMPFILE", I get a link failure because
>>>>>>>>>>>> armlink.exe
>>>>>>>>>>>> > checks if --cpu is specified on the command line, so it is not
>>>>>>>>>>>> > reasonable for "TEMPFILE" to put all options to a temporary
>>>>>>>>>>>> file.
>>>>>>>>>>>>
>>>>>>>>>>>> Right... thus my suggestion - well, let's call it "musing"
>>>>>>>>>>>> rather than
>>>>>>>>>>>> "suggestion" because this might not work at all:
>>>>>>>>>>>>
>>>>>>>>>>>> > For ninja it currently looks like this:
>>>>>>>>>>>> >
>>>>>>>>>>>> > env["LINKCOM"] = '${TEMPFILE("$LINK $LINKFLAGS
>>>>>>>>>>>> /OUT:$TARGET.windows
>>>>>>>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $SOURCES.windows",
>>>>>>>>>>>> "$LINKCOMSTR")}'
>>>>>>>>>>>> >
>>>>>>>>>>>> > env["SHLINKCOM"] = '${TEMPFILE("$SHLINK $SHLINKFLAGS
>>>>>>>>>>>> $_SHLINK_TARGETS
>>>>>>>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $_SHLINK_SOURCES",
>>>>>>>>>>>> "$SHLINKCOMSTR")}'
>>>>>>>>>>>> >
>>>>>>>>>>>> > So I'm curious if moving $LINK $LINKFLAGS to the left of
>>>>>>>>>>>> ${TEMPFILE
>>>>>>>>>>>> > would help...
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>>> [email protected]
>>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>> [email protected]
>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> [email protected]
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.