Re: URL internationalization!

Martin;

> > > > What do you mean "visual order"?
> > > 
> > > If you don't know that, I'm afraid you don't understand much
> > > about bidirectionality, or at least you live in a world with
> > > terminology completely different from what is commonly used.
> > 
> > I have no time to be bothered by those terminologies created
> > by someone who does not understand bidirectinality.
> 
> I know a lot of people in Israel and in countries using
> Arabic use these terms, and I have good reasons to assume
> that they know more about bidirectionality than you do.

I have no interest how people in Israel are processing
bidiractionality in structured text with PDA.

URL itself is in plain text.

> At the Multilingual symposium in Tsukuba last week, to which I
> was invited, the following question came up during a demo in one
> of my talks: If you have the sentence
> 
> 	She said: ".GAT NETUG" and turned away.
> 
> on your screen, put the cursor between ".GAT" and "NETUG", and
> hit return, how should the result look? shuld it look like
> 
> 	She said: ".GAT
> 	NETUG" and turned away.
> 
> or should it look like
> 
> 	She said: "NETUG
> 	.GAT" and turned away.
> 
> I showed that my system (based on the distinction between
> visual and logical ordering) did the later.

That's the way to go with structured text.

> People from
> Alis and Apple (both with longstanding experience in the
> Middle East markets) confirmed that this was what their
> systems did.

So?

> If they didn't do the right thing, they would
> not be in that market anymore. Ken'ichi Handa, the leader
> of the mule (multilingual emacs) team, who had asked the
> question, agreed to implement it that way if that was
> what the users needed.

What the users need is, finite-state mechanism to enable
searching by regular expressions.

> Got it. Also got a paper copy of it, in the proceedings of
> last year's workshop. Terse and criptic writing, as always
> in your case.

Apparently, you know nothing about formal language theory.

The paper assumes that the readers have some knowledge on the
difference between finite state automaton and push down automaton.

> I wonder what your understanding of bidirectionality
> would have produced.

> Interestingly enough,
> things such as Boyer-Moore work extremely well with UTF-8,
> but fail quickly for iso 2022.

"Fail quickly"? What do you mean?

Boyer-Moore is just determinization of finite-state automaton
and works perfectly fine over any text with finite-state
structure, including ISO 2022 based ones.

Boyer-Moore does not work at all with structured text with
nesting.

No, it is not impossible to construct non-finite-state
strucutre over ISO 2022.

But, unlike UTF-8, which MUST be non-finite-state, there are
a lot of ISO 2022 based encoding, such as
ISO-2022-{JP,KR,JP-2,CN,CH-EXT}, which is finite state.

Of course, determinization, in general, is exponential problem,
that, unless you carefully implement the system, there will
be a state explosion. But, it is just an implementaiton issue.

> > Wrong. Conversion between logical and visual order needs
> > push down automaton.
> 
> That is fine.

No.

> It only affects one step, namely the display of
> URLs. And it actually helps all the rest, because if we can
> use logical order internally, things such as searching, relative
> URL processing, sorting, and so on, will work nicely without any
> changes.

Searching with push down automaton is a lot lot lot more slower
(O(N**3)) than searching with finite state machines.

The more serious problem is that there is no handy notation,
such as regular expression of finite state search, of search
patterns.

> > > The programs have
> > > to work with all 8 bits, of course. Please note that if we
> > > would use ISO-2022, major changes would be necessary to
> > > avoid that bytes identical to reserved characters are
> > > misinterpreted.
> > 
> > Huh? You are confusing the internal and external representations.
> 
> Are you assuming that all URL-handling programs will be
> rewritten to do conversion from ISO-2022 external representation
> to whatever suitable internal representation?

No. Not at all.

As only the ASCII-only, % notation can be used with URLs, why
do one have to bother for the conversion?

> For UTF-8, there is no need to do any external/internal
> conversion for comparison, you can use a lot of your
> favorit UNIX tools on it directly.

Martin, you are biting yourself. Though you said:

> in your case. And rather blatant inconsistencies, such as
> on the one hand saying that grep is very important, and on

I never said "grep is very important". I said:

	First of all, many text processing tools depend on regular
	expression.
	Plain text on which UNIX "grep" can't be applicable is not
	so meaningful.

Not only grep but also all the UNIX and non-UNIX plain text
processing tools of some complexity, rely on regular expressions.

							Masataka Ohta