Re: [bash] Variable-name parsing consumes first byte of following multibyte UTF-8 character

On 8/4/26 11:35 PM, zheng wrote:
> # Bash bug report: variable-name parsing swallows first byte of multibyte UTF-8 char
> 
> ## Summary
> In a UTF-8 locale, `$VAR` immediately followed by a multibyte (non-ASCII)
> character causes bash to treat the first byte of that character as part of the
> variable name. 

Thanks for the report.

Well, certain characters on certain systems. Certainly that particular byte
on macOS.

Bash knows that variable names are composed of bytes, and that only certain
bytes are valid ([_0-9a-zA-Z]). It uses isalnum(3) and isalpha(3) to check,
trusting that in, say, en_US, characters > 127 and <= UCHAR_MAX will fail
both tests.

This isn't strictly kosher, because bash uses setlocale(3) to set the
various locale categories at program startup, so these checks are subject
to LC_CTYPE. In this case, the first byte of the `，' multibyte sequence
returns true for isalnum() (specifically islower()), even though the
numeric value of the byte is 239, and it certainly doesn't render as
anything in Terminal (though isprint() returns 1).

(I seem to remember something about macOS incorrectly returning true for
isalpha/isupper/islower/isalnum for characters in this range, but I can't
remember exactly what. Nevertheless.)

I'll have to change this to use a scheme that doesn't vary with the current
locale. It shouldn't be any trouble -- since it uses the `portable'
character set, it can be computed at build time.

I'll probably have something in the next devel branch push.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [email protected]    http://tiswww.cwru.edu/~chet/

-----BEGIN PGP SIGNATURE-----

wmMEABEIACMWIQR8ATX7CIqvbGbGULm7WGnwZOp0qwUCanTPdQUDAAAAAAAKCRC7WGnwZOp0q+XS
AJ9w/NLMAICzEe+IAE7wJJ+UWlw6xQCfZ8Ww7n6hpWzHxBPiEMwkyyzE4qo=
=uE+p
-----END PGP SIGNATURE-----