Re: Ctype patch for review
"YAMAMOTO, Taku" <[email protected]>
| Newsgroups | gmane.os.freebsd.devel.internationalization |
|---|---|
| Organization | Trans New Technology, Inc. |
| Message-ID | <[email protected]> |
On Mon, 17 Sep 2007 13:21:30 +0400 Andrey Chernov <[email protected]> wrote: > On Mon, Sep 17, 2007 at 10:29:21AM +0200, Petr Hroudn?? wrote: > > 2007/9/16, Andrey Chernov <[email protected]>: > > > The problem is: currently our single byte ctype functions are broken for > > > wide characters locales in the argument range >= 0x80 - they may return > > > false positives. > > > > > > For example, for UTF-8 locale we currently have: > > > iswspace(0xA0)==1 and isspace(0xA0)==1 > > > (because iswspace() and isspace() are the same code) > > > but must have > > > isspace(0xA0)==0 > > > > This is exactly what happens on other OSes and I agree this is the > > right behaviour > > for UTF-8. However, we must ensure, that: > > > > for C locale: isspace(0xA0)==0 > > for ISO8859-* locales: isspace(0xA0)==1 > > for UTF-8 locales: isspace(0xA0)==0 > > The patch test for wide char locale presence first (__mb_cur_max > 1), so > does not affect single byte locales like ISO8859-* > Checking for __mb_cur_max is not enough for certain locales. For example, SJIS has following range for JIS X0201 (a.k.a. HALFWIDTH KANA). /* * JIS X201 */ PUNCT 0xa1-0xa5 SPACE 0xa0 BLANK 0xa0 SPECIAL 0xa1-0xdf PHONOGRAM 0xa6-0xdf SWIDTH1 0xa0-0xdf -- -|-__ YAMAMOTO, Taku | __ < <[email protected]> - A chicken is an egg's way of producing more eggs. - _______________________________________________ [email protected] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-i18n To unsubscribe, send any mail to "[email protected]"