Re: [poky] Problem creating spdx #poky #bitbake #spdx

On Thu, Mar 20, 2025 at 6:27 AM Marko, Peter <[email protected]> wrote:
>
> I have sent a patch for spdx-3.
>
>
>
> Spdx-2.2 was broken by https://git.openembedded.org/openembedded-core/commit/?id=eb597bf61cbcb0a4d43149404c93eec0894fb4c7

I can update the SPDX 2.2 code to use the deps

>
> This commit seems to have more things wrong:
>
> * not adapting spdx-2.2 codebase
>
> * importing spdx30_tasks in init even if spdx-2.2 (or no spdx) is used

This is needed so that bitbake can properly track the variable
dependencies in the Python code used by SPDX 3. It shouldn't affect
any output, regardless of whether SPDX 3, SPDX 2.2, or no SPDX is
used.

>
> Unless there is an easy fix, it should be probably reverted.

It should be an easy fix.

>
>
>
> Problem with spdx-2.2 is that it’s not enabled so also not tested.
>
> I think it should be either added to tests or removed completely as it will get broken a lot when extending spdx-3 code.

It has tests, see meta/lib/oeqa/selftest/cases/spdx.py. I'm not sure
why the test case didn't catch this though; I'll take a look at that
also.

>
>
>
> Peter
>
>
>
>
>
> From: [email protected] <[email protected]> On Behalf Of Thomas Kilian via lists.yoctoproject.org
> Sent: Thursday, March 20, 2025 12:24
> To: [email protected]
> Subject: [poky] Problem creating spdx #poky #bitbake #spdx
>
>
>
> Hi,
>
>
>
> On Poky master when I have a root filesystem containing symlinks i get the following spdx creation error:
>
>
>
> ERROR: image-debug-1.0-r0 do_create_rootfs_spdx: Error executing a python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:do_create_rootfs_spdx(d)
>      0003:
> File: '/workspaces/build/../sources/poky/meta/classes-recipe/create-spdx-image-3.0.bbclass', lineno: 31, function: do_create_rootfs_spdx
>      0027:ROOTFS_POSTUNINSTALL_COMMAND =+ "spdx_collect_rootfs_packages"
>      0028:
>      0029:python do_create_rootfs_spdx() {
>      0030:    import oe.spdx30_tasks
>  *** 0031:    oe.spdx30_tasks.create_rootfs_spdx(d)
>      0032:}
>      0033:addtask do_create_rootfs_spdx after do_rootfs before do_image
>      0034:SSTATETASKS += "do_create_rootfs_spdx"
>      0035:do_create_rootfs_spdx[sstate-inputdirs] = "${SPDXROOTFSDEPLOY}"
> File: '/workspaces/build/../sources/poky/meta/lib/oe/spdx30_tasks.py', lineno: 1071, function: create_rootfs_spdx
>      1067:        dirnames.sort()
>      1068:        filenames.sort()
>      1069:        for fn in filenames:
>      1070:            fpath = Path(dirpath) / fn
>  *** 1071:            if not fpath.is_file() or fpath.is_symlink():
>      1072:                continue
>      1073:
>      1074:            relpath = str(fpath.relative_to(image_rootfs))
>      1075:            h = bb.utils.sha256_file(fpath)
> File: '/usr/lib/python3.12/pathlib.py', lineno: 894, function: is_file
>      0890:        Whether this path is a regular file (also True for symlinks pointing
>      0891:        to regular files).
>      0892:        """
>      0893:        try:
>  *** 0894:            return S_ISREG(self.stat().st_mode)
>      0895:        except OSError as e:
>      0896:            if not _ignore_error(e):
>      0897:                raise
>      0898:            # Path doesn't exist or is a broken symlink
> File: '/usr/lib/python3.12/pathlib.py', lineno: 842, function: stat
>      0838:        """
>      0839:        Return the result of the stat() system call on this path, like
>      0840:        os.stat() does.
>      0841:        """
>  *** 0842:        return os.stat(self, follow_symlinks=follow_symlinks)
>      0843:
>      0844:    def lstat(self):
>      0845:        """
>      0846:        Like stat(), except if the path points to a symlink, the symlink's
> Exception: PermissionError: [Errno 13] Permission denied: '/workspaces/build/tmp/work/poky-linux/image-debug/1.0/rootfs/root/.vs-debugger/vs2019'
>
>
>
> This could be fixed if the check order in spdx30_tasks.py is switched to
>
> 1071:            if not fpath.is_symlink() or fpath.is_file():
>
>
>
> If I use "create-spdx-2.2.bbclass" I get the following error:
>
>
>
> ERROR: m4-native-1.4.19-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:do_create_spdx(d)
>      0003:
> File: '/workspaces/build/../sources/poky/meta/classes/create-spdx-2.2.bbclass', lineno: 509, function: do_create_spdx
>      0505:
>      0506:            if archive is not None:
>      0507:                recipe.packageFileName = str(recipe_archive.name)
>      0508:
>  *** 0509:    dep_recipes = collect_dep_recipes(d, doc, recipe)
>      0510:
>      0511:    doc_sha1 = oe.sbom.write_doc(d, doc, pkg_arch, "recipes", indent=get_json_indent(d))
>      0512:    dep_recipes.append(oe.sbom.DepRecipe(doc, doc_sha1, recipe))
>      0513:
> File: '/workspaces/build/../sources/poky/meta/classes/create-spdx-2.2.bbclass', lineno: 282, function: collect_dep_recipes
>      0278:    dep_recipes = []
>      0279:
>      0280:    deps = oe.spdx_common.get_spdx_deps(d)
>      0281:
>  *** 0282:    for dep_pn, dep_hashfn, in_taskhash in deps:
>      0283:        # If this dependency is not calculated in the taskhash skip it.
>      0284:        # Otherwise, it can result in broken links since this task won't
>      0285:        # rebuild and see the new SPDX ID if the dependency changes
>      0286:        if not in_taskhash:
> Exception: TypeError: cannot unpack non-iterable Dep object
>
> ERROR: Logfile of failure stored in: /workspaces/build/tmp/work/x86_64-linux/m4-native/1.4.19/temp/log.do_create_spdx.213270
> ERROR: Task (/workspaces/build/../sources/poky/meta/recipes-devtools/m4/m4-native_1.4.19.bb:do_create_spdx) failed with exit code '1'
>
> What can I do to get the spdx creation working again?
>
>
>
>
>
> Greetings,
>
> Thomas Kilian