Re: New Version Notification for draft-obispo-epp-idn-00.txt

This email is a little long.  I'm not picking on Keith Gaughan; his
email was just a convenient hook for this.  If you don't want to read
all this, skip to the [Conclusion] at the end.

On Fri, Jan 06, 2012 at 01:35:52PM +0000, Keith Gaughan wrote:

> The graceful way to do this is to have multiple tables per script to check
> labels against. If all the characters in the label are whitelisted in any
> one of those tables, then it's valid. That solves the homograph problem
> without discriminating against minority languages (and Europe still has a lot
> of them), and copes just fine with situations like Serbian, which uses both
> the Latin and Cyrillic scripts. All that's saved by requiring a language code
> is CPU cycles.

I already asked this once, but I do so again: there's considerable
discussion of what's called "label generation rules" and "zone
repertoires" in the recent ICANN VIP draft report.  The report is open
for public comment right now:
http://www.icann.org/en/announcements/announcement-2-23dec11-en.htm.
I'd really like some additional eyes on those sections: I wrote a
substantial amount of that text, and I'm not completely sure it's all
right.  But I think it's a possibly-useful direction.  The section that discusses this at length is section 4, starting on page 39.

In any case, I think the script/language distinction is invidious for
our purposes: both are important considerations.  So I want to expand
a little on what's in the VIP draft and try to explain in more detail
the sort of approach I think will work in the long term.  

There are, in addition, several problems with this discussion of
"tables", the most obvious of which is that we don't have a standard
for such tables that actually works for everyone.  What that really
means is that, for zones that span widely divergent languages and
writing systems, we are already in trouble.

RFC 3743 was an extremely important and clever answer to the peculiar
problems of one sort of writing system.  It was also seminal, however,
and that's a little unfortunate because the CJK systems are extremely
bad analogies for everything else.  (Indeed, Korean is almost unique
in the history of human writing, and Chinese is completely unusual in
the world today.)  One issue with RFC 3743 and all of the work that
has followed from it (including RFC4290) is that the basic "table"
idea conflates a number of functions that just happen to be aligned in
the cases for which the JET guidelines were designed.  One of these is
to specify the code points the registry (i.e. zone operator --
remember that a registry isn't necessarily an operator of TLDs, but
just a repository of information for some administrative area, in this
case DNS zones) will accept in U-labels for the zone.  Another is to
associate an identifier with the subset of code points constrained by
some rules.  Another is to specify alternative labels that may or must
be delegated along with some "fundamental" label (e.g. the one that
was submitted to the registry).  Another is to specify alternative
labels that may not be registered in the presence of that fundamental
label.  And another is to specify alternative labels that must not be
active in the zone when that fundamental label is in the zone.  (All
these "alternative label" bits are one of the kinds of things people
call "variants", and what RFC 3743 meant by "variant".)

Not all of these purposes are actually achieved in any of the table
formats registered with IANA (there is more than one format, alas).
For instance, the specification of the first purpose (what code points
are permitted, which we can call the "zone repertoire") is actually
derivable only from the union of all the tables a registry uses.
Similarly, it works out, the alternatives and their associated
permissions with respect to activation also need to be derived using
all the defined tables.  Some table formats can't distinguish between
"normally activated" and "must be activated", nor between "not
normally activated but may be at applicant's choice" and "must not be
activated".  None to my knowledge can distinguish between "must be
delegated if activated" and "must be aliased if activated".

People are actually using the tables for different purposes.
Verisign, for instance, has recently submitted a very large number of
tables to IANA that effectively permits every Unicode code point that
works for IDNA (and I haven't even validated the second part of this),
but specifies no alternative forms at all; their submissions include
one for the Arabic script.  SaudiNIC, on the other hand, registered a
table for Arabic language (not script) that includes complicated rules
for alternative handling according to the position of a code point in
a candidate label; the "table" also has a number of rules written into
it.

Even if we were to recognise implicitly the different roles outlined
above, while maintaining the somewhat-underdefined notion of a
"table", I think there are certain basic truths about the functioning
of IDN labels at or near the top levels that we're going to need to
keep in mind.  The first is that, with an extremely large well of
potential code points, the DNS registration community is going to have
to converge on conventions for most of these code points relatively
quickly.  If a code point has vastly different behaviour in different
zones, either that code points just won't get used at all, or else the
code point will come to be a source of a lot of confusion, phishing
opportunities, and frustration.  (Or else the outlier behaviours will
get lost, but that's just a variation of "not used".)  The second is
that we are never going to be able to cram every user expectation
about a given code point's behaviour into the global DNS: there are
too many funny ways an identifier system like the DNS can trip people
up.  Paradoxically, the third is that people will use DNS names based
on their preconceptions of how their letters work, and we had better
be prepared for those expectations or we will subject people to very
nasty surprises.  

An effect of points (2) and (3) above is that we actually ought to be
reluctant, rather than eager, to add additional code points whenever
they are not absolutely required.  For instance, there are a lot of
optional marks in Arabic; I think nobody should ever permit them in
high-level zones, because they're just going to cause confusion.
There are words in some languages (Farsi and Nepali both come to mind)
that can't be written without the ZWNJ; but I think nobody should
permit labels of such kinds in multi-language high-level zones _even
though it would be useful_, because the danger of interaction with
other linguistic rules is just too great.  Apostropoids like U+02BC
are more or less impossible to get along without in some languages
(Ukranian comes to mind), but I think adding such a code point to a
high-level multi-language zone is asking for serious trouble
(particularly in the case of Ukranian, where most standard keyboards
don't even have a keycode that usually generates that code point).

Moreover, the way that the zone repertoire gets established is
politically fraught.  Some cases are simple, even for languages
without an academy to establish their rules (English is an excellent
example here).  But one should consider very carefully the political
situation in parts of Eastern Europe and Western Asia before being
glib about the problems of who gets to decide whether a language is
written in Latin, Cyrillic, or Arabic scripts (and, after that, which
subset of each).  This is why I like an approach something like the
sample "approach 3" from section 4.2 of the VIP report: it doesn't
leave us stuck with the Unicode script properties, it doesn't require
us to come up with rules for all the code points we don't understand,
and it still encourages widespread engagement with relevant language
and script users.

[Conclusion]

Because of all of the above, I'd like to step back a bit from the
language-vs-script and table-based way of framing this discussion and
focus more on how one selects which set of code points one wants to
use and, if there are rules governing the behaviour of those code
points, how one expresses one's options under those rules.  This will
make for a more complicated EPP extension, but it will also make for
one that actually matches the different registry policies already in
place today.  By creating an extension that can accommodate the many
different policies in place, we might be able to reduce the number of
extensions that actually get deployed.

Best,

Andrew

-- 
Andrew Sullivan
[email protected]
_______________________________________________
provreg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/provreg