drop ambiguous-wide behaviour from Unicode CJK locales
Thomas Wolff <[email protected]>
| Newsgroups | gmane.comp.lib.newlib |
|---|---|
| Message-ID | <[email protected]> |
It seems that ambiguous-wide behaviour (i.e. double-width property for characters in the East Asian Ambiguous width category) for CJK locales with UTF-8 encoding is inconsistent with Linux locale definitions. The attached patch changes that. Characters like ─ ü æ are no longer wide in the following locales: ja_JP.utf8 ko_KR.utf8 zh_*.utf8 but only in ja, ko, zh locales with legacy encoding. Explicit modifiers @cjkwide and @cjknarrow are not affected. Thomas
0001-drop-ambiguous-wide-behaviour-from-Unicode-CJK-local.patch
(text/plain, 2.5 KB)
From a5e6c1daafbf006d4394b405a6392ca6696f8c8c Mon Sep 17 00:00:00 2001 From: Thomas Wolff <[email protected]> Date: Wed, 7 Oct 2020 18:35:54 +0200 Subject: [PATCH] drop ambiguous-wide behaviour from Unicode CJK locales --- newlib/libc/locale/locale.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/newlib/libc/locale/locale.c b/newlib/libc/locale/locale.c index 2c452ba98..968642745 100644 --- a/newlib/libc/locale/locale.c +++ b/newlib/libc/locale/locale.c @@ -78,12 +78,9 @@ This implementation also supports the modifiers <<"cjknarrow">> and <<"cjkwide">>, which affect how the functions <<wcwidth>> and <<wcswidth>> handle characters from the "CJK Ambiguous Width" category of characters described at http://www.unicode.org/reports/tr11/#Ambiguous. -These characters have a width of 1 for singlebyte charsets and a width of 2 -for multibyte charsets other than UTF-8. -For UTF-8, their width depends on the language specifier: -it is 2 for <<"zh">> (Chinese), <<"ja">> (Japanese), and <<"ko">> (Korean), -and 1 for everything else. Specifying <<"cjknarrow">> or <<"cjkwide">> -forces a width of 1 or 2, respectively, independent of charset and language. +These characters have a width of 1 for singlebyte charsets and UTF-8, +and a width of 2 for multibyte charsets other than UTF-8. Specifying +<<"cjknarrow">> or <<"cjkwide">> forces a width of 1 or 2, respectively. This implementation also supports the modifier <<"cjksingle">> to enforce single-width character properties. @@ -903,17 +900,12 @@ restart: /* Determine the width for the "CJK Ambiguous Width" category of characters. This is used in wcwidth(). Assume single width for single-byte charsets, and double width for multi-byte charsets - other than UTF-8. For UTF-8, use double width for the East Asian - languages ("ja", "ko", "zh"), and single width for everything else. + other than UTF-8. For UTF-8, use single width. Single width can also be forced with the "@cjknarrow" modifier. Double width can also be forced with the "@cjkwide" modifier. */ loc->cjk_lang = cjkwide || - (!cjknarrow && mbc_max > 1 - && (charset[0] != 'U' - || strncmp (locale, "ja", 2) == 0 - || strncmp (locale, "ko", 2) == 0 - || strncmp (locale, "zh", 2) == 0)); + (!cjknarrow && mbc_max > 1 && charset[0] != 'U'); if (cjksingle) loc->cjk_lang = -1; /* Disable CJK dual-width */ #ifdef __HAVE_LOCALE_INFO__ -- 2.28.0