Re: -fshort-wchar option, wchar_t and Native<->Unicode conversion

jshin,

yeah, i'm not convinced we are doing the right thing either.  what we do 
is cast each PRUnichar (in a *PRUnichar) to a wchar_t and then pass that 
to wcrtomb (one PRUnichar at a time).  assuming that wcrtomb does indeed 
expect a code point from ISO 10646/Unicode, i'm not sure how it would 
handle an element of *PRUnichar.  is it valid to zero-pad expand UTF-16 
to form UCS-4?  probably not, in which case our linux code is certainly 
busted.

darin



Jungshik Shin wrote:
> In 
> http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsNativeCharsetUtils.cpp#623, 
> 
> 
> it's assumed that *wchar_t is the same as *PRUnichar when 
> gWCharIsUnicode is true. I'm wondering if it's always the case.
> 
> At the momemnt, Linux build is made with '-fshort-wchar' option to gcc. 
> With that option, wchar_t is short(2byte) and *wchar_t is the same as 
> *PRUnichar as far as non-BMP char is concerned. Even in that case, I'm 
> not sure *wchar_t is identical to *PRUnichar. It's to be note that 
> *PRUnichar is NOT a string of UCS2 BUT a 'UTF-16 string'. I'm not sure 
> what gcc does when '-fshort-wchar' is turned on. That is, I'm wondering 
> whether '-fshort-wchar' makes *wchar_t 'UTF-16 string' as opposed to 
> 'UCS2 string'. If not, UnicodeToNative and NativeToUnicode are broken 
> for non-BMP characters.
> 
> Even more serious is that *wchar_t is a string of PRUInt32's on 
> Linux(glibc,gcc) if '-fshort_wchar' is not used. How about other 
> compilers on other platforms? '-fshort_wchar' is only available in gcc, 
> isn't it? [1]
> 
> 
> How about the interaction between '-fshort-wchar' and 
> '__STDC_ISO_10646__' (ISO C 99)? If the latter is defined, wchar_t is 
> supposed to be UCS4. Does '-fshort-wchar' override it? If so, do we plan 
> to use '-fshort-wchar' permanently?
> 
> A possible work-around for this problem is to use 'iconv' in place of mb 
> rtowc and wcrtomb.
> 
> Jungshik
> 
> 
> [1]I'm aware that on some platforms, wchar_t is locale-dependent and 
> it's not even ISO 10646/Unicode, but let's just leave it aside for this 
> discussion.
>