Re: Extra Unicode character for MacSymbol 0xE4?

Ciaran,

Thank you for your concern.

On Feb 22, 2006, at 00:14 , Ciaran Hamilton wrote:
> Hi Dan,
>
> I'm working with the Encode module in my Perl programs fairly  
> extensively,
> and firstly I have to thank you for maintaining this. It's such a
> brilliant module as it makes life so much easier when dealing with
> encodings. Thank you!
>
> I have a question about the MacSymbol encoding as implemented by  
> Encode.
> In my tests (and as seen at
> http://search.cpan.org/src/DANKOGAI/Encode-2.14/ucm/ 
> macSymbol.ucm ), it
> seems that the character at position 0xE4 in the MacSymbol charset  
> seems
> to be converted by decode() to two Unicode characters - U+2122 and U 
> +F87F.

You mean this.

<U2122><UF87F> \xE4 |3 # TRADE MARK SIGN, alternate: sans serif

perldoc enc2xs
>        o   CHARMAP starts the character map section.  Each line has  
> a form as
>            follows:
>
>              <UXXXX> \xXX.. |0 # comment
>                ^     ^      ^
>                |     |      +- Fallback flag
>                |     +-------- Encoded byte sequence
>                +-------------- Unicode Character ID in hex
>
>            The format is roughly the same as a header section  
> except for the
>            fallback flag: | followed by 0..3.   The meaning of the  
> possible
>            values is as follows:
>
>            |0  Round trip safe.  A character decoded to Unicode  
> encodes back
>                to the same byte sequence.  Most characters have  
> this flag.
>
>            |1  Fallback for unicode -> encoding.  When seen, enc2xs  
> adds this
>                character for the encode map only.
>
>            |2  Skip sub-char mapping should there be no code point.
>
>            |3  Fallback for encoding -> unicode.  When seen, enc2xs  
> adds this
>                character for the decode map only.

Since it is marked |3, this map is used for decode() only.

> It seems that the second character is part of the Private Use area,  
> and
> because of this, the second character is meaningless to me, without
> knowledge of what layout was used. I would guess that it's used in  
> MacOS
> to signify something or other, but I'm developing on Windows and  
> Linux.

Frankly I don't know that either.  All I did was convert whatever  
apple supplies to Unicode Consortium

http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/SYMBOL.TXT
> 0xE4	0x2122+0xF87F	# TRADE MARK SIGN, alternate: sans serif


> It seems to me that it would make sense for this extra character to be
> removed from the conversion output, since I assume Encode is meant  
> to be
> cross-platform compatible. Alternatively, if it's meant to be there,
> please let me know as I would be interested to know what the extra
> character is used for.

Since the map came from Apple, it is their responsibility to fix the  
original map.  Meanwhile, you can use the quick fix below;

$utf8 = decode("macSymbol", $octet);
$utf8 =~ tr/\x{F87F}//d;

If you are not content with that, you can even make your own  
encoding.  Consult
"perldoc enc2xs" on how to do it.

> Thank you,
>
>  - Ciaran.

Thank YOU.

Dan the Encode Maintainer