Re: [PATCH v4] localedata: Avoid concurrently written locales in gen-locale.sh

Florian Weimer <[email protected]> writes:

> There is no cross-directory exclusion of concurrent $(gen-locales)
> usage.  Parallel localedef calls can clobber locale data as it is
> being loaded by tests.
>
> With the separate staging areas, there are no peer directories,
> so hard-linking no longer happens.  The touch command is therefore
> unnecessary.

Thanks.

Reviewed-by: Sam James <[email protected]>

>
> ---
> v4: Drop --no-hard-links option  because it is no longer required.
>  localedata/gen-locale.sh | 22 +++++++++++++++-------
>  1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/localedata/gen-locale.sh b/localedata/gen-locale.sh
> index 4762c04284..c449efe125 100644
> --- a/localedata/gen-locale.sh
> +++ b/localedata/gen-locale.sh
> @@ -32,17 +32,25 @@ generate_locale ()
>    out=$3
>    flags=$4
>    ret=0
> +
> +  # Use a staging area to avoid writing to locales concurrently.
> +  # While this process is running, $$ is sufficiently unique.
> +  stage="${common_objpfx}localedata/gen-locale.$$.tmp"
> +  mkdir "$stage" 2>/dev/null || true
> +
>    ${localedef_before_env} ${run_program_env} I18NPATH=../localedata \
>  	${localedef_after_env} $flags -f $charmap -i $input \
> -	${common_objpfx}localedata/$out || ret=$?
> -  if [ $ret -eq 0 ]; then
> -    # The makefile checks the timestamp of the LC_CTYPE file,
> -    # but localedef won't have touched it if it was able to
> -    # hard-link it to an existing file.
> -    touch ${common_objpfx}localedata/$out/LC_CTYPE
> +	$stage/$out || ret=$?
> +  if [ $ret -eq 0 ] ; then
> +      # Ignore errors in case some other process has created the same locale.
> +      # This rename operation should be atomic, and it should fail if the
> +      # $out locale already exists (rename fails with ENOTEMPTY).
> +      mv $stage/$out ${common_objpfx}localedata/. 2>/dev/null || true
> +      rm -rf $stage
>    else
> +    rm -rf $stage
>      echo "Charmap: \"${charmap}\" Inputfile: \"${input}\"" \
> -	 "Outputdir: \"${out}\" failed"
> +	 "Outputdir: \"${out}\" failed (exit status $ret)"
>      exit 1
>    fi
>  }

-----BEGIN PGP SIGNATURE-----

iQEBBAEWCgCpFiEEJaa7iN2bdkxrVUHCc4QJ9SDfkZAFAmpOcDAbFIAAAAAABAAO
bWFudTIsMi41KzEuMTIsMiwyXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25z
Lm9wZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQyNUE2QkI4OEREOUI3NjRDNkI1NTQx
QzI3Mzg0MDlGNTIwREY5MTkwDxxzYW1AZ2VudG9vLm9yZwAKCRBzhAn1IN+RkE7Q
AQD45F0j4p5f36ijCCpMS5cgvRTLDccpi4GN687BboYiWwEAodGUdek8tW+j2l2c
4G6EQCflW3uDrShbWatMyY8Y4QQ=
=EfkL
-----END PGP SIGNATURE-----