Re: Normalisation and matching

Adam M. Costello wrote:

>> Matching is case-insensitive - everybody do not know what case you
>> prefer on your e-mail address (and case is not spoken over a phone
>> when you give your address).
>>
>> If we cannot handle ACE->UCS with case preservation we could as well
>> use a one way hash for the ASCII world.
>
>Do you see what you're saying?  Given:
>
> 1) José
> 2) josé
> 3) kmcsi5csxy
>
>you're saying that if you can't show (1), then it doesn't matter whether
>you show (2) or (3), because neither is any better than the other.  Are
>you serious?

3) is going extream. Only 1) is good.

>
>We already went through all this for IDNA.  Traditionally domain names
>are case-insensitive and case-preserving, but the case-preserving
>part was relaxed for IDNs because it wasn't deemed worth the
>additional required complexity.  We can add an optional mechanism for
>case-preservation later.
>
>We could repeat the same arguments for IMAA, but I see no reason why it
>would play out any differently than it did for IDNA.

It was not good in IDNA and it will break som programs.
But domain names converted to lower case is not as inmportant as names.


>> IMAA should define how non-ASCII e-mail addresses are used in
>> protocols, not how a user interface should handle them.
>
>Just the opposite.  Initially, non-ASCII mail addresses will not appear
>in protocols at all (only ASCII address will), but they will appear in
>user interfaces.  The primary issue that IMAA needs to address is how
>to bridge the gap between non-ASCII user interfaces and existing ASCII
>protocols.  Non-ASCII mail addresses won't appear in protocols until
>new protocols are defined, and each new protocol can then specify how
>non-ASCII mail addresses are to be represented in that protocol.  If
>normalized Unicode is the best way, then that's what they'll specify,
>but there's no need for us to prescribe that choice now.

While this could be good, the character allowed in the international form
may not be restricted by limits in ASCII encoding. 
Still it is bad that IMAA (and IDNA) gives a lot of impact on
the international form that would not exist if you started
from the international perspective.


>But there may be some protocol designers who believe that it's important
>to preserve not only mixed case, but also other presentational
>details that would be destroyed by normalization (like a fullwidth
>@, or a superscript 2).  They might decide that it's better to
>transmit/display/store the mail address exactly as it was originally
>typed, and perform the normalization only momentarily for the purpose of
>matching it to the appropriate mailbox, and then discard the normalized
>form.

That may be so, but you will not get interoperability by that.
I can see no reason to support more than one code point for a
character. If I want to display a character using a wider size I will
do that, but the wide attribute do not belong as a part of the character code.

Actually NFKC is not good in some cases as it removes some characters
that are destinct and will lose their meaning by NFKC. Wide characters
do not belong to that category.


>SpecialCasing.txt is for case mapping, not caseless matching.  The
>relevant file for caseless matching is CaseFolding.txt.  It contains the
>following comment:
>
>    Note that where they can be supported, the full case foldings are
>    superior: for example, they allow "MASSE" and "Maße" to match.
>
>That makes it very clear that SS and sharp-s are supposed to match.
>They both get folded to ss.

I could accept that for matching, even though it is not correct in many
cases. But not when doing a lower case conversion.
"Laße" may not be converted by IMAA to "lasse" because it changes the
meaning of the name.

>There is nothing locale-specific about that mapping.  The only
>locale-specific mappings relate to Turkish i.

Actually it is locale specific, even though Unicode do not say so.
There are several mistakes in Unicode and code points that should never
have existed, making everything unnecessary complex (like having to have
NFKC because the same character exists more than once for some characters).

    Dan