Re: Has anyone seen anything like this

To answer several questions:

SCons version is 4.2.0.  It's the distro version on Fedora 35.  Python 3.10

I am using VariantDir(build_prefix, '.', duplicate=0)

build_prefix by default expants to build/dev/gcc

One can run our build locally on various platforms.  I use Fedora so use the el8 install script to setup the node.  It takes about 10 minutes to do a full parallel build with all of the prerequisites.   To build you should be able to do this.   You do need to have http network access for the downloads of prerequisites.
Run one of the install scripts in utils/scripts/install-*.sh.   This just installs stuff like SCons among other things.  You can also just look at it and figure out what you want to install that way.
clone daos and do git submodule update --init
scons --build-deps=yes install -j24

We also have several Docker scripts if that's preferrable (utils/docker)

Once you've done a build, make a simple change to daos_errno.h (add some struct or change some text in one of the FOREACH macros) and run scons install.   You'll see a few things that always build because we haven't yet done the work to prevent go build from doing its thing when nothing has changed.  But nothing that depends on daos_errno.h (e.g. src/gurt/errno.c) will rebuild.

I tried adding a clear() line.  In fact, I tried moving the configure checks to be the first thing we do and then cleared every header file as installed and daos_errno.h still doesn't trigger a rebuild of the dependencies.

There are a few places where Glob('*.c') is used.  I can try cleaning those up next as I've tried to discourage the use of Glob in general.

-Jeff

________________________________
From: Scons-users <[email protected]> on behalf of Bill Deegan <[email protected]>
Sent: Tuesday, May 24, 2022 2:34 PM
To: SCons users mailing list <[email protected]>
Subject: Re: [Scons-users] Has anyone seen anything like this

Generally when you see something like no longer a dependency/is a new it can be because the ordering of the sources has changed. (this can happen (among other ways) when the source list is generated via Glob("*.c") for example).
Though in your case is it possible the CPPPATH is not the same for each run (possibly generated via Glob())?

Which version of SCons are you using?


On Tue, May 24, 2022 at 12:04 PM Olivier, Jeffrey V <[email protected]<mailto:[email protected]>> wrote:
I have tried such things in the past but haven't found them very helpful.  This is the only output related to daos_errno.h

scons: rebuilding `install/include/daos_errno.h' because:
           `src/include/daos_errno.h' is no longer a dependency
           `src/include/daos_errno.h' is a new dependency
Install file: "src/include/daos_errno.h" as "install/include/daos_errno.h"

There is nothing related to any other file that depends on this.   As for your other question, no, it's not a generated header file, just a source file.

-Jeff
________________________________
From: Scons-users <[email protected]<mailto:[email protected]>> on behalf of Bill Deegan <[email protected]<mailto:[email protected]>>
Sent: Tuesday, May 24, 2022 12:00 PM
To: SCons users mailing list <[email protected]<mailto:[email protected]>>
Subject: Re: [Scons-users] Has anyone seen anything like this

Did you try using --debug=explain first?
That's the builtin logic to help figure out they why's..
(though it's not perfect)

Is that header file generated? or just static in the filesystem?

On Tue, May 24, 2022 at 10:20 AM Olivier, Jeffrey V <[email protected]<mailto:[email protected]>> wrote:
I am building the DAOS project from github.com/daos-stack/daos<http://github.com/daos-stack/daos> and hitting the following issue

And I've noticed that when I change a header file (src/include/daos_errno.h), dependent targets do not get rebuilt (e.g. src/gurt/errno.c).  The only thing that does get built is the Install builder for the file.  But the other targets which add src/include to the CPPPATH do not get built.  I've been trying to track down why it doesn't get rebuilt so I narrowed it down a bit by adding some of my own debug prints down in the SCons code.  I found that the csig for the Include case for the same file differs from the csig for the various cases of implicit dependencies returned by the scanner.

To illustrate this, I've added these prints in SCons/Node/FS.py

3327     def changed_content(self, target, prev_ni, repo_node=None):
3328         cur_csig = self.get_csig()
3329         check = str(self) == 'src/include/daos_errno.h'
3330         if check:
3331             print("\nChecking %s" % str(self))
3332         try:
3333             if check:
3334                 print(" new_sig=%s old_sig=%s changed=%s" % (cur_csig, prev_ni.csig, str(cur_csig != prev_ni.csig)))
3335             return cur_csig != prev_ni.csig
3336         except AttributeError:
3337             return 1

When I do this, I see this in the output for all of the checks for implicit dependencies:

Checking src/include/daos_errno.h
 new_sig=0ffda7577e533f682945b5d4c508cdd6 old_sig=0ffda7577e533f682945b5d4c508cdd6 changed=False

But the sequence surrounding the Install builder looks like this:
Checking src/include/daos_errno.h
 new_sig=b2b4b801f3f4ee78151223fa2b2d3e98 old_sig=4baa3b48b7e1cc339227b11676299434 changed=True
Checking src/include/daos_errno.h
 new_sig=b2b4b801f3f4ee78151223fa2b2d3e98 old_sig=4baa3b48b7e1cc339227b11676299434 changed=Tru
Install file: "src/include/daos_errno.h" as "install/include/daos_errno.h"

As you can see, the signature for the same file differs between these two cases.  Does anyone here know what might be happening here?

A few other details:

  *   The DAOS build uses VariantDir with duplicate=0
  *   CPPPATH includes src/include

I'm at a loss as to why this would happen and any help would be appreciated.  I've not been able to reproduce with a simpler build.  Admittedly, the DAOS build is fairly complicated as we have some optional options to prebuild a bunch of external dependencies that may not be installed.

-Jeff




______________________
Jeff Olivier, DAOS engineer
ASE/SCP/AXG/Intel
+1 720-487-7145
_______________________________________________
Scons-users mailing list
[email protected]<mailto:[email protected]>
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
[email protected]<mailto:[email protected]>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users

diff --git a/src/SConscript b/src/SConscript
index b24cb0d22..829094efa 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -12,6 +12,8 @@ HEADERS_GURT = ['dlog.h', 'debug.h', 'common.h', 'hash.h', 'list.h',
                 'heap.h', 'fault_inject.h', 'debug_setup.h',
                 'types.h', 'atomic.h', 'slab.h',
                 'telemetry_consumer.h', 'telemetry_common.h']
+HEADERS_CART = ['api.h', 'iv.h', 'types.h', 'swim.h']
+
 
 # Keep versioned libs for now to avoid any conflict with 1.0
 CART_VERSION = "4.9.0"
@@ -51,15 +53,46 @@ def scons():
     base_env.AppendUnique(CPPPATH=[Dir('include').srcnode()])
     base_env.AppendUnique(CPPPATH=[Dir('include')])
     generate_daos_version_header(env)
+
+    if not env.GetOption('clean') and not env.GetOption('help'):
+
+        print("Checking local headers can be included")
+        broken_includes = False
+        config_env = env.Clone()
+        config = Configure(config_env)
+        for header in HEADERS_GURT:
+            if not config.CheckHeader(os.path.join('gurt', header)):
+                broken_includes = True
+        for header in HEADERS_CART:
+            if not config.CheckHeader(os.path.join('cart', header)):
+                broken_includes = True
+        config.Finish()
+
+        if broken_includes:
+            print("Broken local header files, cannot continue")
+            Exit(2)
+
     for header in HEADERS:
-        env.Install(os.path.join('$PREFIX', 'include'), 'include/%s' % header)
+        header_node = env.File("#src/install/%s" % header)
+        base_env.File("#src/install/%s" % header).clear()
+        header_node.clear()
+        env.Install(os.path.join('$PREFIX', 'include'), header_node)
     if prereqs.server_requested():
         for header in HEADERS_SRV:
-            env.Install(os.path.join('$PREFIX', 'include', 'daos_srv'),
-                        'include/daos_srv/%s' % header)
+            header_node = env.File("#src/install/daos_srv/%s" % header)
+            base_env.File("#src/install/daos_srv/%s" % header).clear()
+            header_node.clear()
+            env.Install(os.path.join('$PREFIX', 'include', 'daos_srv'), header_node)
     for header in HEADERS_GURT:
-        env.Install(os.path.join('$PREFIX', 'include', 'gurt'),
-                    'include/gurt/%s' % header)
+        header_node = env.File("#src/install/gurt/%s" % header)
+        base_env.File("#src/install/gurt/%s" % header).clear()
+        header_node.clear()
+        env.Install(os.path.join('$PREFIX', 'include', 'gurt'), header_node)
+    for header in HEADERS_CART:
+        header_node = env.File("#src/install/cart/%s" % header)
+        base_env.File("#src/install/cart/%s" % header).clear()
+        header_node.clear()
+        env.Install(os.path.join('$PREFIX', 'include', 'cart'), header_node)
 
     env.Append(CCFLAGS=['-DCART_VERSION=\\"' + CART_VERSION + '\\"'])
     env.AppendUnique(CPPPATH=[Dir('cart/test/utest').srcnode()])
@@ -106,20 +139,5 @@ def scons():
     # Build test
     SConscript('tests/SConscript')
 
-    if not env.GetOption('clean') and not env.GetOption('help'):
-
-        print("Checking local headers can be included")
-        broken_includes = False
-        config_env = env.Clone()
-        config = Configure(config_env)
-        for header in HEADERS_GURT:
-            if not config.CheckHeader(os.path.join('gurt', header)):
-                broken_includes = True
-        config.Finish()
-
-        if broken_includes:
-            print("Broken local header files, cannot continue")
-            Exit(2)
-
 if __name__ == "SCons.Script":
     scons()
diff --git a/src/cart/SConscript b/src/cart/SConscript
index 260901a56..578368729 100644
--- a/src/cart/SConscript
+++ b/src/cart/SConscript
@@ -13,8 +13,6 @@ import daos_build
 # pylint: disable=ungrouped-imports
 import SCons.Action
 
-HEADERS = ['api.h', 'iv.h', 'types.h', 'swim.h']
-
 SRC = ['crt_bulk.c', 'crt_context.c', 'crt_corpc.c',
        'crt_ctl.c', 'crt_debug.c', 'crt_group.c', 'crt_hg.c', 'crt_hg_proc.c',
        'crt_init.c', 'crt_iv.c', 'crt_register.c',
@@ -73,9 +71,6 @@ def scons():
     env.AppendUnique(LIBPATH=[Dir('.')])
     env.AppendUnique(CPPPATH=[Dir('.').srcnode()])
     env.AppendUnique(CPPPATH=[Dir('include').srcnode()])
-    for header in HEADERS:
-        env.Install(os.path.join('$PREFIX', 'include', 'cart'),
-                    os.path.join('../include', 'cart', header))
 
     prereqs.require(env, 'mercury', headers_only=True)
 
@@ -103,22 +98,6 @@ def scons():
 
     Default(cart_lib)
     Export('cart_lib', 'cart_targets')
-
-    if not env.GetOption('clean') and not env.GetOption('help'):
-
-        print("Checking local headers can be included")
-        broken_includes = False
-        config_env = env.Clone()
-        config = Configure(config_env)
-        for header in HEADERS:
-            if not config.CheckHeader(os.path.join('cart', header)):
-                broken_includes = True
-        config.Finish()
-
-        if broken_includes:
-            print("Broken local header files, cannot continue")
-            Exit(2)
-
     Export('env', 'prereqs')
 
     env.Install(conf_dir, ['utils/memcheck-cart.supp',
diff --git a/src/include/daos_errno.h b/src/include/daos_errno.h
index e4a666033..35b2ef8c9 100644
--- a/src/include/daos_errno.h
+++ b/src/include/daos_errno.h
@@ -158,7 +158,7 @@ extern "C" {
 	/** Invalid user/group permissions.*/				\
 	ACTION(DER_SHMEM_PERMS,         (DER_ERR_GURT_BASE + 44),	\
 	       Unable to access shared memory segment due to		\
-	       incompatible user or group permissions)
+	       ipncompatible user or group permissions)
 	/** TODO: add more error numbers */
 
 /** Preprocessor macro defining DAOS errno values and internal definition of d_errstr */