Re: [poky] [PATCH] sstate: remove corrupted artifacts from local mirror

On Fri, Oct 4, 2024 at 5:16 PM Yu, Max via lists.yoctoproject.org
<[email protected]> wrote:
>
> We observe sstate cache corruptions sometimes which cause rebuilds. That is not
> a fatal error as the package has to be rebuilt and updated artifact needs to be
> pushed to remote sstate cache mirror. Currently, Yocto does not handle
> corruptions properly, where the corrupted artifact is not deleted or
> renamed. Later, after the package is built the same corrupted artifact is pushed
> to remote mirror and the same procedure is circled again and again.
>
> This change verifies the outcome of the unpacking action and renames the
> artifact if a fatal error occurred ("tar" tool returns error 2). In such case we
> rename the artifact what causes that a proper one is created and uploaded
> overwriting the exisiting one - the corrupted one - in the remote mirror. That
> way we break the loop of uploading corrupted file again and again.
>

Thanks for contributing and trying to make the yocto project better.
I think the patch seems ok to me. However, please send it to
openembedded-core mailing list
[email protected] since this is part of oe-core
which poky uses.
I would also suggest to peruse through -
https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html

> Suggested-by: Przemyslaw Sobon <[email protected]>
> Signed-off-by: Max Yu <[email protected]>
> ---
>  meta/classes-global/sstate.bbclass | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
> index 11bb892a42..5a7ce35341 100644
> --- a/meta/classes-global/sstate.bbclass
> +++ b/meta/classes-global/sstate.bbclass
> @@ -937,7 +937,12 @@ sstate_unpack_package () {
>                 ZSTD="pzstd -p ${ZSTD_THREADS}"
>         fi
>
> -       tar -I "$ZSTD" -xvpf ${SSTATE_PKG}
> +       if ! tar -I "$ZSTD" -xvpf ${SSTATE_PKG}; then
> +               echo "Fatal error extracting sstate cache artifacts, file might be corrupted or truncated, renaming"
> +               mv ${SSTATE_PKG} ${SSTATE_PKG}.unpack_error
> +               exit 2
> +       fi
> +
>         # update .siginfo atime on local/NFS mirror if it is a symbolic link
>         [ ! -h ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 2>/dev/null || true
>         # update each symbolic link instead of any referenced file
> --
> 2.40.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#13449): https://lists.yoctoproject.org/g/poky/message/13449
> Mute This Topic: https://lists.yoctoproject.org/mt/108827961/1997914
> Group Owner: [email protected]
> Unsubscribe: https://lists.yoctoproject.org/g/poky/unsub [[email protected]]
> -=-=-=-=-=-=-=-=-=-=-=-
>