Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

"Yao Wei (魏銘廷)" <[email protected]>
Newsgroups gmane.linux.debian.devel.www
Message-ID <20200505021839.GA12928__28320.2581036434$1588645463$gmane$org@torchic.m-wei.net>
Package: www.debian.org
Followup-For: Bug #959474

Hi,

After a bit of investigation of Perl source code (5.31.11 downloaded
from upstream) I found the they have weird handling of whitespace when
`feature unicode_strings` turned on.  I am not a perl person and I
haven't executed the source code yet, so my interpretation might be
wrong.

When `unicode_strings` is on, `in_uni_8_bit` should true internally, and
in three places of pp.c:6040, pp.c:6076, pp.c:6114 `isSPACE_L1` is
called to check whether the examining character is a whitespace, by
checking whether the character is 0x85 or 0xA0 (handy.h:1611).  In the
case of the character 包, the last byte of 3-byte UTF-8 code is 0x85,
henceforth the problem.

-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.6.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
signature.asc (application/pgp-signature, 833 B)
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE8htgli/cvAeDZWXRl1ewRNNETDMFAl6wzP4ACgkQl1ewRNNE
TDOecQ/9E7yoKT6cSzrihPvTHVCEXo9CwrdK6zZuEcX0+Qd8uvYmSVghb7s4X41Q
3AqThQWmJ4/BNJ/Z2kfYWtXWeYiXoOXa/HK44szT9TWdyh2HQYRk2TdZbSH5z3WX
/4V+okfz+FIwRXeFwKnGYpfDFUCtYg0p3DjHcdldF+VUOm0sIU27jEP6WCs8iMM4
MKX8ny14UOV4Dxrt571HAMnWakqiCtcg7RcArPYo/SrWufvs2UvX+kSTRbCY1jRF
BioF1o58Y0Zy0TT0bHuaXdcU05vqvXz/FCIjfIWgJTl6PpNK3Z40Aeti3Mfencmi
Uakbm8oRwC+/0fF7nkEos6GsrrnLtoc4PHSXZuBDz2ie22sAYqWUxlz8yiiLSlAw
Q/zUIxZ+j7VvrvhMiYhBOBGMlkpEM44KxJ9ryPnX+WKKtgCzr/35t8kbZBCY8Y+n
IWqtQkX4PcEvleuaCCKiRC/OOuRS3O3Wa6uxrnd+An37lsIgtd+qQVqCV716iKlm
Hi5wcg9QtHPOyviQ+bVQvXRjjS6AobEKuq92zGWSVNR7ymHZQQ4mzvBL2/6dyxAD
VBg7IZvJ/KPtc0XAzUgtX170AM1HdHulelEKqTlUekhe2zQMMaUD/MSc7h/Py3A7
e7zlpeLlKdabO7vICJBd5MtD0CtWdWKGOD71z5JDVFpgbpmJCS4=
=Csv3
-----END PGP SIGNATURE-----
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.