Re: URL internationalization!

On Mon, 24 Feb 1997, Masataka Ohta wrote:

> > A major disadvantage of 2022 is its statefulness.
> 
> Can you say bidirectionality?
> 
> Could you please explain how you can process Arabic or Hebrew
> script with ISO 10646 without being stateful?
> 
> I know how to treat such script statelessly. But, ISO 10646 can't

Bidirectionality is indeed a difficult issue for any kind of
work on full internationalization. URLs are no exception.

It is possible to deal with bidirectionality statelessly as
long as you have a rigid line-based format (e.g. raw text
email). It is not possible anymore as soon as you have
automatic formatting of paragraphs.

The Unicode bidirectionality algorithm is designed mainly
for free text. There are some problems when directly applying
it to editing the same text with some markup, e.g. HTML or
LaTeX. For HTML, this has been discussed on the relevant list,
and higher-level extensions (using HTML markup) have been
added to alleviate it.

For URLs, the situation is somewhat similar. We can see
e.g. a Hebrew URL as some Hebrew text pieces,
some ASCII text pieces, and some "markup" such as "/",
":", "#", and so on. What we want is that the markup,
which defines the overall structure, has higher priority
than the text between it. And because an average URL
contains a lot of ASCII parts, it's probably better
if the components of an URL appear LTR (left-to-right) even
in otherwise fully RTL (right-to-left) contexts.

To achieve this, Francois Yergeau has proposed that before
rendering (i.e. a user interface issue only), the "/"
and similar characters are surrounded by LRM (left-to-right
mark) codes to change them from neutral to LTR. In an RTL block,
the URL as a whole would also have to be surrounded by
a LRE-PDF pair. Because it's outside the URL, this doesn't
bother us here. The problem with this approach is that
(rendering) software would have to recognize things as
URLs. Otherwise, they would get sligthly to strongly
garbled; the individual parts (words and markup) would
still be there, but their sequence might be changed.

The alternative is to define that URLs are completely and
strictly LTR, which would mean that they would be embedded
in a LRO-PDF pair. This would be equivalent to what Masataka
is proposing. The disadvantage is that internally, the
RTL characters would have to be in logical backwards order,
which would make it difficult to search for a word both
in plain text and in URLs at the same time, or to copy/paste
such words.

Having the text in an URL in logical sequence is definitely
a plus. This means that whatever individual users of URLs
come up with (different users may prefer different ways
of displaying mixed-direction URLs), future software can care
for it once it's known what users prefer.

The ftpext working group and the URN working group have also
chosen UTF-8 as their preferred character encoding. Bidi
issues were discussed shortly in both groups, and the
agreement was that it's difficult to get to a final conclusion
on this issue now, but it's not necessary to delay progress
in internationalization in general.


Regards,	Martin.