Re: Patch to fix bug 134036

(no longer cross-posting, as I think this is not an issue for kde-pim. 
kde-core-devel, maybe).

Hi Diego,

On Saturday 26 April 2008, you wrote:
> Shai Berger wrote:
> > I think deciding to choose an algorithm that contradicts the Unicode
> > standard is a mistake -- you choose to present some texts differently
> > from standard-complying implementations. In the process, you kill the
> > option of putting an RLM/LRM as the first character to set the direction,
> > blur the distinction between strong and weak directionality, and probably
> > damage marks of direction embedding. Can you point me  to the rationale
> > behind this decision?
>
> I kindly disagree...

You disagree that your implementation contradict the standard, that it kills 
the paragraph-level effect of RLM/LRM, that it blurs the distinctions, or 
that it damages embedding? I'll grant you the last two, but the first two are 
valid (LRM/RLM should be obvious. see below about contradicting the 
standard).

> see seccion 4.3 of the Bidi Algorythm , called  Higher Levels protocols,

Higher-level protocols are things like XML and word-processor file formats, 
*not* plain text. Further, Section 4.3 ends with these words:

	When text using a higher-level protocol is to be converted to Unicode plain 
text, for consistent appearance formatting codes should be inserted to ensure 	
that the order matches that of the higher-level protocol.

So -- even if you consider KatePart to be implementing a higher-level 
protocol, as long as it edits plain text, it should save it with the correct 
marks. And this all but implies that it should be reading it according to 
these marks as well.

> and in seccion 3.3.1 we see this text (*):
>
> Because paragraph separators delimit text in this algorithm, this will
> *generally *be the first strong character after a paragraph separator or
> at the very beginning of the text. Note that the characters of type LRE,
> LRO, RLE, or RLO are ignored in this rule. This is because typically
> they are used to indicate that the embedded text is the /opposite/
> direction than the paragraph level.
>

<argumentative>
Check your code, you forgot to ignore {LR,RL}{E,O}. Actually, you should do 
more than ignore them: you should ignore anything between the {LR,RL}E and 
the respective PDF, and count everything between the {LR,RL}O and its PDF 
according to the explicit override (see 2.1, 2.2, 2.3). 
</argumentative>

> I think we can test using another algorythm for automatic determination
> of the text direction if the text itself does not include any LTR/RTL
> markups, nor the user specified any direction (input widgets). I just
> saw many ways in which the older code failed, and I want to test a new
> and "hopefuly better" way.
>

I was going to refer you to 4.2 "conformance", but the argument really boils 
down to understanding what "higher protocol" means. If I understand you 
correctly, you think that any program displaying Unicode text -- in 
particular, a plain-text editor -- is at liberty to define "higher level 
protocols". I find this notion, with all due respect, absurd. In particular, 
when applied to plain-text editors and plain-text mail, this clearly breaks 
compatibility with other implementations of the standard.

It is reasonable to apply the "higher protocol" notion to texts in an 
application context (e.g, since you mentioned Akregator, the feed language 
should determine the base direction of titles and articles; Meir of Whatsup 
has been adamant about this for years). But note the important difference: In 
cases like this, the context -- the "higher protocol" -- is *outside* the 
text whose direction we are trying to decide. Deciding base direction only 
according to the text itself, while proclaiming the right to do so (in a 
different way than the Unicode BiDi Algorithm) based on the notion of a 
higher protocol -- I can only call this a fallacy. It is wrong.

Have fun,
	Shai.


------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/kde-il/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/kde-il/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/