Bug#959474: Issues with Chinese language (all variants) when building some pages in buster
"Yao Wei (魏銘廷)" <[email protected]>
| Newsgroups | gmane.linux.debian.devel.www |
|---|---|
| Message-ID | <20200505021839.GA12928__28320.2581036434$1588645463$gmane$org@torchic.m-wei.net> |
Package: www.debian.org Followup-For: Bug #959474 Hi, After a bit of investigation of Perl source code (5.31.11 downloaded from upstream) I found the they have weird handling of whitespace when `feature unicode_strings` turned on. I am not a perl person and I haven't executed the source code yet, so my interpretation might be wrong. When `unicode_strings` is on, `in_uni_8_bit` should true internally, and in three places of pp.c:6040, pp.c:6076, pp.c:6114 `isSPACE_L1` is called to check whether the examining character is a whitespace, by checking whether the character is 0x85 or 0xA0 (handy.h:1611). In the case of the character 包, the last byte of 3-byte UTF-8 code is 0x85, henceforth the problem. -- System Information: Debian Release: bullseye/sid APT prefers unstable APT policy: (500, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 5.6.0-1-amd64 (SMP w/8 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled
signature.asc
(application/pgp-signature, 833 B)
-----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE8htgli/cvAeDZWXRl1ewRNNETDMFAl6wzP4ACgkQl1ewRNNE TDOecQ/9E7yoKT6cSzrihPvTHVCEXo9CwrdK6zZuEcX0+Qd8uvYmSVghb7s4X41Q 3AqThQWmJ4/BNJ/Z2kfYWtXWeYiXoOXa/HK44szT9TWdyh2HQYRk2TdZbSH5z3WX /4V+okfz+FIwRXeFwKnGYpfDFUCtYg0p3DjHcdldF+VUOm0sIU27jEP6WCs8iMM4 MKX8ny14UOV4Dxrt571HAMnWakqiCtcg7RcArPYo/SrWufvs2UvX+kSTRbCY1jRF BioF1o58Y0Zy0TT0bHuaXdcU05vqvXz/FCIjfIWgJTl6PpNK3Z40Aeti3Mfencmi Uakbm8oRwC+/0fF7nkEos6GsrrnLtoc4PHSXZuBDz2ie22sAYqWUxlz8yiiLSlAw Q/zUIxZ+j7VvrvhMiYhBOBGMlkpEM44KxJ9ryPnX+WKKtgCzr/35t8kbZBCY8Y+n IWqtQkX4PcEvleuaCCKiRC/OOuRS3O3Wa6uxrnd+An37lsIgtd+qQVqCV716iKlm Hi5wcg9QtHPOyviQ+bVQvXRjjS6AobEKuq92zGWSVNR7ymHZQQ4mzvBL2/6dyxAD VBg7IZvJ/KPtc0XAzUgtX170AM1HdHulelEKqTlUekhe2zQMMaUD/MSc7h/Py3A7 e7zlpeLlKdabO7vICJBd5MtD0CtWdWKGOD71z5JDVFpgbpmJCS4= =Csv3 -----END PGP SIGNATURE-----