URL internationalization!

Hello URLers,

As said before, I volunteered to write the parts on
internationalization for the URL process document.
I will be happy to figure as a coauthor and help on
other issues, or to just contribute on internationalization
as a wg member. Below is my input. It's written in
language that is a little bit more thight than the rest
of the draft, because I expect the rest of the draft to
be tightened up as things become clear.

However, even if the language is tighter, the actual requirements
are not tighter. The only thing really required is that developpers
of URL schemes make the necessary considerations regarding
internationalization, and document these considerations. This is
quite similar to what is required in many cases for security.

Looking forward to your questions and comments.

Regards,	Martin.


> INTERNET-DRAFT                                    Larry Masinter
> <draft-ietf-url-process-00>                          Dan Zigmond
> January 4, 1997                             Harald T. Alvestrand
> expires June 4, 1997
> 
>                Guidelines and Process for new URL Schemes

>    Issues:
> 	Registration process isn't really there.

	Internationalization [can be removed once the proposed
	additions are made].


> 2.2.1 Clear mapping from other name spaces
> 
>    In many cases, new URL schemes are defined as ways to translate
>    other protocols and name spaces into the general framework of
>    URLs. The "ftp" URL scheme translates from the FTP protocol, while
>    the "mid" URL scheme translates from the Message-ID field of
>    messages.
> 
>    In either case, the description of the mapping must be complete,
>    must describe how character sets get encoded or not in URLs, must
>    describe exactly how all legal values of the base standard can be
>    represented using the URL scheme, and exactly which modifiers,
>    alternate forms and other artifacts from the base standards are
>    included or not included.
> 
>    In all cases, encoding rules must be made clear: What octets are
>    put into the URL, and if other octets need to be represented, what
>    convention is used to represent them?  Any departure from the %xx
>    convention needs special justification.

Add:

See also Section X, Internationalization, for considerations on mappings.


Add the following chapter or section:

X.Y Internationalization

Most URL schemes represent characters; these characters are encoded
into octets using some kind of character encoding, and then further
encoded into URLs using the %HH escaping mechanism. This process is
straightforward and reversible for a relevant subset of US ASCII,
but not necessarily so for all the other characters used around the
world.

To make user interface and handling issues independent of the
URL scheme, UTF-8 [RFC 2044] should be used to encode characters
represented by URLs wherever possible. Schemes
cannot expect that any kind of user interface is available for
arbitrary or scheme-dependent character encodings.

UTF-8 is fully compatible with US-ASCII, can encode all characters
of the Universal Character Set, and is in most cases easily
distingushable from legacy encodings or random octet sequences.

Note: RFC 2044 specifies UTF-8 in terms of Unicode Version 1.1,
corresponding to ISO 10646 without ammendments. It is widespread
consensus that this should indeed be Unicode Version 2.0,
corresponding to ISO 10646 including ammendment 5
(and future additions thereto).

Protocols and mechanisms underlying a certain scheme may either
directly use UTF-8 or may use a different way to encode characters,
for example because they cannot make use of all 8 bits of an octet.
In the later case, the URL nevertheless should use UTF-8 for
consistency, and the conversion from and to the protocol-specific
encoding should be documented.

The use of multiple national or regional character encodings
without their identification in the protocol or mechanism is
a sad ad-hoc practice and should be avoided or corrected.
The best way to correct it is to specify UTF-8 as the preferred
encoding. URLs already in use should however reflect the octets
expected by the protocol to access the resource as long as a
reversible conversion to UTF-8 is not available and/or the
server is not upgraded to accept UTF-8.

Whatever the solution taken in terms of internationalization
(including the absence of such a solution), it must be clearly
documented, and the absence of a solution or the deviation
from the suggestions above must be well justified.


> 6. References

[RFC 2044]    F. Yergeau, "UTF-8, A Transformation Format of Unicode
                  and ISO 10646", Alis Technologies, October 1996.