Re: [yocto-patches] [yocto-autobuilder-helper][PATCH v2 05/10] scripts: add vcontainer-tarball setup, integration, and publishing

On Mon, 2026-06-01 at 16:18 -0700, Tim Orling via lists.yoctoproject.org wrote:
> From: Tim Orling <[email protected]>
> 
> Introduce the vcontainer-tarball SDK plumbing used by container build
> jobs. The vcontainer-tarball is a meta-virtualization-derived SDK
> (modelled after buildtools-tarball) that ships the container build
> toolchain so worker jobs do not need to rebuild it for every step.
> 
>   * scripts/utils.py: add setup_vcontainer_tarball(), and add an
>     env_glob keyword argument to setup_tools_tarball() and
>     enable_tools_tarball() so the vcontainer-tarball can source its
>     specific environment-setup-ci file rather than the universal
>     glob.
>   * scripts/run-config: source the vcontainer-tarball environment
>     for build-targets / cmds / test-targets / plain-cmds steps,
>     gated by a new NOVCONTAINER step variable so individual steps
>     (such as the dashboard indexing step) can opt out independently
>     of NOBUILDTOOLS.
>   * scripts/shared-repo-unpack: invoke setup_vcontainer_tarball so
>     workers extract the SDK during unpack.
>   * scripts/publish-artefacts: publish the vcontainer-tarball
>     artefact so downstream test jobs can fetch a stable SDK.
> 
> AI-Generated: Claude Cowork Opus 4.7
> Signed-off-by: Tim Orling <[email protected]>
> ---
>  config.json                |  1 +
>  scripts/publish-artefacts  |  5 +++++
>  scripts/run-config         | 19 +++++++++++++++++++
>  scripts/shared-repo-unpack |  1 +
>  scripts/utils.py           | 22 ++++++++++++++++++----
>  5 files changed, 44 insertions(+), 4 deletions(-)
> 
> diff --git a/config.json b/config.json
> index 7df4271..16d8a04 100644
> --- a/config.json
> +++ b/config.json
> @@ -1433,6 +1433,7 @@
>              "step3" : {
>                  "shortname" : "Populate/update dashboard site",
>                  "NOBUILDTOOLS" : true,
> +                "NOVCONTAINER" : true,
>                  "EXTRACMDS" : ["${SCRIPTSDIR}/run-dashboard-index ${HELPERBUILDDIR}/../"]
>              }
>          },
> diff --git a/scripts/publish-artefacts b/scripts/publish-artefacts
> index e56e131..0e820e9 100755
> --- a/scripts/publish-artefacts
> +++ b/scripts/publish-artefacts
> @@ -146,5 +146,10 @@ case "$target" in
>          sha256sums $TMPDIR/deploy/images/qemux86-64
>          cp -R --no-dereference --preserve=links $TMPDIR/deploy/images/qemux86-64/*qemux86* $DEST/patchtest
>          ;;
> +    "vcontainer-tarball")
> +        mkdir -p $DEST/vcontainer-tarball
> +        sha256sums $TMPDIR/deploy/sdk
> +        cp -R --no-dereference --preserve=links $TMPDIR/deploy/sdk/*vcontainer* $DEST/vcontainer-tarball
> +        ;;
>  esac
>  
> diff --git a/scripts/run-config b/scripts/run-config
> index e896234..0f5a26a 100755
> --- a/scripts/run-config
> +++ b/scripts/run-config
> @@ -153,6 +153,25 @@ else:
>      if args.phase == "init" and args.stepname == "buildtools":
>          sys.exit(0)
>  
> +if jcfg:
> +    vcontainer = utils.setup_vcontainer_tarball(ourconfig, args.workername, None, checkonly=True)
> +    if vcontainer:
> +        addentry("vcontainer", "Setup vcontainer tarball", "init")
> +else:
> +    # If we're executing a specific step, check whether vcontainer is disabled for it
> +    vcontainer = True
> +    if args.stepname in ("build-targets", "cmds", "test-targets", "plain-cmds"):
> +        try:
> +            vcontainer = not utils.getconfigvar("NOVCONTAINER", ourconfig, args.target, int(args.phase))
> +        except ValueError:
> +            # Not an integer step phase
> +            pass

I suspect this logic also needs to be in the if jcfg block above, else
it will add the task to all jobs on the autobuilder, then just do
nothing in the task. If there isn't anything to do, we may as well just
not add it all?

Cheers,

Richard