Re: Filter UTF16 converting to ISO-2022-CN/KR or HZ

You have a good point.

In the case of \u000e and other unrepresentable text, the callback 
specified by the user should be called to properly handle/react to this 
character.

Unfortunately, the person that can verify this problem is on vacation at 
the moment.  Please file a jitterbug on this topic.  Patches are also 
welcome ;-)

George Rhoten
IBM Globalization Center of Competency/ICU  San José, CA, USA



Peter Edberg <[email protected]> 
Sent by: [email protected]
11/30/2004 04:02 PM

To
[email protected]
cc

Subject
Filter UTF16 converting to ISO-2022-CN/KR or HZ






I have been investigating using ICU converters for "ISO-2022-KR", 
"ISO-2022-CN" and "HZ-GB-2312" to replace some of our Mac OS algorithmic 
converters. I noticed when using ucnv_fromUnicode with these converters 
that they do not filter UTF16 values which will be invalid or misleading 
in the ISO-2022 or HZ output.

For example, given UTF16 input 0061 000E 0062 0063, the ISO-2022-KR/CN 
converters produce (after initial G1-designation escape sequence) 61 0E 62 
63. The presence of the 0x0E (shift out) will cause the subsequent 62 63 
to be misinterpreted as GB2312 or KSC5601 text instead of ASCII.

It seems to me that the Unicode 000E should either be ignored or be 
replaced in the ISO-2022 output by a fallback or substitution character.

I plan to file a jitterbug, but first I would like to solicit input on 
what others feel is the appropriate way to handle this sort of situation.

Thanks,
Peter
-- 
----------------------------------------------
Peter Edberg  .  .  .  .  Apple Computer, Inc.
Mac OS Engineering: International & Text Group
Tel: +1 (408) 974-4275, Fax: +1 (408) 862-4566
----------------------------------------------
_______________________________________________
icu mailing list
[email protected]
http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu