Re: Proposal: Introduction of Neutral Override Character

Hi Behdad,

I agree that it makes sens to bump the embedding level by 2 for
the NO character. 

I just looked at the pango and gtk sources and they both find
the base direction through a call to:
   
    pango_find_base_dir(char *text, int len)

There would be no problem at all changing this to be a wrapper
to fribidi_find_base_dir() (the wrapper is needed as Owen doesn't
want to expose the fribidi API outside of pango). All this function
does it to find the first strong character. It is trivial to change
this to ignore characters between NO and PDF. 

Btw, a related display problem that would be solved by the use of
the NO/PDF pairs is within an HTML editor.

Trying to achive the same thing through markup would be a lot
messier.

I agree with you that NE is needed as well but has a different
purpose. Within an HTML editor this might be used in parallel to
"" within tags. E.g.

     SHE LOOKED LIKE {NO}<img src="her.jpg" 
     alt="{NE}SHE IS PRETTY!{PDF}">{PDF}

The question still remains how to get the editor to insert those
marks...

Regards,
Dov

On Thu, Jul 22, 2004 at 11:16:38AM -0400, Behdad Esfahbod wrote:
> On Thu, 22 Jul 2004, Dov Grobgeld wrote:
> 
> > Hi,
> >
> > After reading Behdad's excellent summary of the current state of
> > BiDi in Gtk, I realized that the problem that Behdad describes
> > on Slide 11 could easily be solved by the introduction of a new
> 
> Seems like I forgot to CC the fribidi list.  Here are the slides,
> focusing the slide Dov is referring to:
> 
> http://behdad.org/download/Presentations/bidi-layouts/#12
> 
> > unicode control character "Neutral Override". This new unicode
> > character should have similar functionality to:
> >
> >    202D;LEFT-TO-RIGHT OVERRIDE;Cf;0;LRO;;;;;N;;;;;
> >    202E;RIGHT-TO-LEFT OVERRIDE;Cf;0;RLO;;;;;N;;;;;
> >
> > And I suggest that for the time being we give the new character
> > the definition:
> >
> >    20EB;NEUTRAL OVERRIDE;Cf;0;NO;;;;;N;;;;;
> >
> > The effect of the character is terminated by PDF. The effect
> > of the character should be different than LRO and RLO though,
> > as it will not change the properties of the characters within
> > its range, but will only prevent the use of the range for
> > the purpose of seeking a base direction.
> 
> Interesting proposal.  I suggest that it adds 2 to the embedding
> level of the characters within, pushing them upper such that they
> are *isolated* from the surrounding characters a bit too.  I
> think this proposed character of yours is quite related to what I
> was proposing to Asmus Freytag last month in Unicode Technical
> Committee meeting in Toronto.  Mine is not an Override character,
> but an Embedding.  So I like to push your character to 20EC and
> put mine in 20EB, resulting in:
> 
>    20EB;NEUTRAL EMBEDDING;Cf;0;NE;;;;;N;;;;;
>    20EC;NEUTRAL OVERRIDE;Cf;0;NO;;;;;N;;;;;
> 
> Now what my character tries to do is th embed sub-paragraphs in
> your paragraph.  The idea is that instead of writing:
> 
>   You said: %s\n
> 
> you write:
> 
>   You said: <NE>%s<PDF>\n
> 
> and the text in the sub-paragraph, will get it's own paragraph
> direction, falling back o the higher level direction if all
> inside is neutral characters.  The details can be written down
> easily.
> 
> But the problem with what you suggest is that...
> 
> > If we implement this addition in pango and fribidi, we get an
> > acceptable solution for Gtk2. All that is needed then in order
> > to solve the layout problem of slide 11 is to wrap the name in
> > a NO/PDF pair as follows :
> >
> >    <NO>Behdad<PDF>:
> 
> Many engines, including Pango and Gtk+, find the paragraph
> direction themselves, by looking up the first letter in the
> paragraph themselves, they just don't leave it to FriBidi.  And
> with your suggestion paragraph direction detection becomes quite
> a more complex project.  This is not necessarily bad, it just
> means that Pango needs to call my new
> fribidi_get_paragraph_direction call that I've added to FriBidi.
> 
> > If we accept this solution we should also send in a proposal
> > to the Unicode consortium to make this a formal addition to
> > the standard.
> 
> Sure, but we need to find real use cases for these.  If we just
> send them our current situation the response will be that it
> belongs to markup, not plain text, which is in fact right.
> 
> > The actual implementation would take Behdad ~5min, and me
> > ~1h. ;-)
> 
> So, lets continue the discussion, we can allocate a couple slots
> in Private Use Area and test with it.
> 
> > Regards,
> > Dov
> 
> --behdad
>   behdad.org