Re: URL internationalization!

Hello Masataka,

On Wed, 19 Feb 1997, you wrote:

> Martin;
> 
> > In the sense that in the "canonical form" of an URL, only
> > ASCII is allowed, my current proposal doesn't change this.
> 
> "only ASCII is allowed"? OK.
> 
> > In terms of encoding (from characters to octets), chaos is the
> > current state,
> 
> As you said "only ASCII is allowed", there is no chaos.

It is important to make the distinction between the characters
represented in an URL and the "URL characters", which are
used to represent whatever has to be represented, be that
characters or other kinds of data. The current syntax draft
does this very nicely, in a Section numbered 1.5, but which
is part of Chapter 2. I copy it here for your enlightment:

# 1.5. Characters, octets, and encodings
# 
#    URLs are sequences of characters. Parts of those sequences of
#    characters are then used to represent sequences of octets. In turn,
#    sequences of octets are (frequently) used (with a character
#    encoding scheme) to represent characters. This means that when
#    dealing with URLs it's necessary to work at three levels:
# 
#                      represented characters
#                                 ^
#                                 |
#                                 v
#                               octets
#                                 ^
#                                 |
#                                 v
#                          URL characters
# 
#    This looks more complicated than necessary if all one is dealing
#    with is file names in ASCII, but is necessary when dealing with the
#    wide variety of systems in use. URL characters may represent octets
#    directly or with escape sequences (Section 2.3). Octets may
#    sometimes represent characters in ASCII, in other character
#    encodings, or sometimes be used to represent data that does not
#    correspond to characters at all.

As I have described in my original mail, the main problem is
that "with A character encoding scheme" leads to chaos, and by
changing this, in the long run, to "with UTF-8", that chaos can
be avoided.


> > In terms of encoding (from characters to octets), chaos is the
> > current state, and this is unsatisfactory and can be improved.
> 
> ISO 2022 has been the law, dispite all the attempt of you trying
> introduce the chaos.

ISO 2022 is a toolbox that needs additional agreements to work.
Iso-8859-X, iso-2022-jp/kr/..., and many others conform to ISO 2022,
but URLs don't have a way to indicate which one is used, nor
would it be wise to introduce one. Or would you want to burden
the user with RFC1522-like constructs?


> > In terms of how to do it, for about the past year, there have
> > been many discussions in particular about UTF-7 or UTF-8.
> 
> UTF-* has nothing to do with the internationalization, not even
> a localization (outside of Europe).

Of course it has. You will be surprised to know how many programs
in use in Japan already use ISO 10646/Unicode. And of course,
ISO 10646 is also a Japanese standard, namely JIS 221.


> See RFC 1815 on how to properly do a localization with ISO 10646.

Well, I invite everybody who needs some cheering up and some
serious laughs to go and read it. For those not familliar with
the ways RFCs can be written, it is important to notice that
everybody can write an RFC.

The main point that RFC 1815 makes (not that this was anything
new :-), namely that it is not required that a system be able
to render all characters of ISO 10646, is again nicely solved
in our case by the availability of the %HH-escaping.


> > In terms of keyboarding, the claim that only an extremely limited
> > character set would be acceptable was long held up by many
> > people. Because it's easy to offer a keyboarding service with
> > an HTML page (in particular with a Java applet),
> 
> You completely misunderstand the keyboarding problem.
> 
> A long-useful proper solution has been to have transliteration
> programs from ASCII to local scripts. The solution works for
> those who are familiar with the local scripts.

True transliteration from Latin script to other scripts may work
in some cases, but as you should know, Japanese is the best
example of a case where it fails horribly.


> The problem is that, most of, say, French using people does
> not know where to find a proper transliteration tool when
> they see an Arabic or a Devanagali character and identify
> it just a strange graphical symbol of some foreign culture.

Looks like you mean transliteration from another script into
Latin. In the true sense of transliteration, that's not easy
either (Japanese again is a good example). As for pure keyboarding
purposes, URLs already have a satisfactory solution, namely
the %HH encoding.


> No search engines are helpful here to find the keyboarding page,
> unless the user can key-in the character.

Arabic and Devanagari, with their many ligatures, are rather
complex. But if the user first gets a list of script examples,
and then a list of combinations including ligatures and contextual
forms where necessary, somebody that really wants to access
the URL in question can do it.

The important point is that it's a bad idea to make the use
of an Arabic URL complicated for those 99.9... % of people
that know Arabic and have Arabic support on their machines,
just to make it a little easier for those 0.0...1 % that
don't.


> And, even then, none Greek people can't type in Greek Alpha as
> Greek Alpha.

The problem that Latin A and Greek Alpha and Cyrillic A have
to be distinguished also exists in JIS 208.
But even currently, we can use l, 1, and I, or 0 and O (include
D here for the Japanese) in an URL. People who ever thought
about putting out URLs like l10DOoI1lI quickly learned their
lesson.


> > because it
> > is unfair and highly inefficient that local users would have
> > to convert their resource names to strange %HH sequences,
> 
> Wrong.
> 
> It is as easy as rewriting "<" in HTML.

Do you do that by hand? Well, you might know the hex code for
this one, and a few others, but do you know the hex codes
for a reasonable number of Kanji (be it in JIS, SJIS, EUC,
or JIS 221)?


> It's easy to offer a translation service with an HTML page (no
> Java necessary here).

Nice if we have such translation services. Just that we need
to know the encoding to be able to translate.


> > Another claim made many times by opponents to URL internationalization
> > was that domain names couldn't be internationalized anyway.
> 
> No. You understand nothing here.

Can you be more specific, please?


> > With my
> > draft-duerst-dns-i18n-00.txt, I showed that it could indeed be done
> > very easily.
> 
> Can you understand how Russian think what
> 
> 	ABC.COM
> 
> is?

Don't understand your point. Can you explain it?

Anyway, I will give you a good example. Assume that a Japanese
wants to find Toshiba on the web, and that Toshiba of course
also would like to be found easily. What would they prefer:

A) That the user goes and tries
	tousiba.co.jp
	toushiba.co.jp
	tohsiba.co.jp
	tohshiba.co.jp
	tosiba.co.jp
	toshiba.co.jp
  [For those not fluent in Japanese, the above are all ways
   in which a Japanese not aware of the Engish spelling of
   Toshiba could assume it to be spelled. And the first four
   (on some software only the first two) are the ones (s)he
   would use in inputting the Japanese name.]

B) That the user can type in, with whatever keyboard software
(s)he is familiar with, the Japanese equivalent, which
transliteraded and upper-cased for easy distinction could look
like
	TOUSIBA.SHA.NICHI
or something similar?


Regards,	Martin.