Re: MS-Word bidirectional behaviour
Shachar Shemesh <[email protected]> Sat, 10 Dec 2016 14:19:47 +0200
| Newsgroups | gmane.comp.internationalization.fribidi |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============1473937666==
Content-Type: multipart/alternative;
boundary="------------770A59DF87093D7816B73345"
This is a multi-part message in MIME format.
--------------770A59DF87093D7816B73345
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
On 09/12/16 20:37, Eduardo Casti=C3=B1eyra wrote:
> Hi guys,
>
> We have users in Iran who are complaining about our app not behaving
> the way MS-Word does when it comes to directional ordering.
>
> In most applications, if one writes the sentence "In an attack of an
> F14 780 people died" in Persian with no RTL marks one gets the followin=
g:
>
> =D9=87=D9=88=D8=A7=D9=BE=DB=8C=D9=85=D8=A7=D9=87=D8=A7=DB=8C F14 =DB=B7=
=DB=B8=DB=B0 =D9=86=D9=81=D8=B1 =D8=B1=D8=A7 =D9=85=D8=B5=D8=AF=D9=88=D9=85=
=DA=A9=D8=B1=D8=AF=D9=86=D8=AF.
>
> Obviously the =DB=B7=DB=B8=DB=B0 number is missplaced, it should be on =
the left side
> of F14. Even if the numerals were Persian, most applications get the
> =DB=B7=DB=B8=DB=B0 in the wrong position, and so does FriBiDi. My under=
standing is
> there is only two ways of fixing it
>
> 1) Force the user to insert an RTL mark after F14
> 2) Detect that =DB=B7=DB=B8=DB=B0 is writen in Persian numerals and aut=
omatically
> treat it as an RTL run (maybe FriBiDi should do that?)
>
> However, somehow MS-Word detects when the user changes the keyboard
> layout and that affects the ordering as shown in the following picture.=
>
> https://snag.gy/pbsh7g.jpg=20
Fribidi implements the "Unicode Bidi Algorithm" (henceforth, UBA). It is
defined in techincal report #9 of the Unicode consortium. You can view
it at http://unicode.org/reports/tr9/.
You can see the BiDi parsing of the sentence you wrote at
http://unicode.org/cldr/utility/bidi.jsp?a=3D%D9%87%D9%88%D8%A7%D9%BE%DB%=
8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%DB%B7%DB%B8%DB%B0+%D9%86%D9%81%D8%B=
1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF%D9%86%D8=
%AF.+&p=3DRTL
It is according to an older spec of the UBA, but I don't see anything
there that should make a difference.
You can see that your problem is that both the 14 and the 780 are
catagorized as BiDi class EN. This means that the space between them is
a neutral between two same direction letters, and gets a left to right
direction, hence your problem.
Within the UBA (which is what FreeBidi is doing), I'm afraid there is no
solution other than to insert an RLM, as you've suggested (parsed
sentence:
http://unicode.org/cldr/utility/bidi.jsp?a=3D%D9%87%D9%88%D8%A7%D9%BE%DB%=
8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%E2%80%8F%DB%B7%DB%B8%DB%B0+%D9%86%D=
9%81%D8%B1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF=
%D9%86%D8%AF.+&p=3DRTL)
As for Word, the reason it "works" is that it does not use the UBA in
order to render BiDi. It actually saves the keyboard language with which
each letter was typed. This is both non-standard (obviously) and error
prone. In my experience, it generates a lot of user confusion as to how
to type things so that they turn out correctly on screen.
In short, I have done nothing to help you solve your problem, but I hope
you now understand it better :-)
Shachar
--------------770A59DF87093D7816B73345
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html style="direction: ltr;">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 09/12/16 20:37, Eduardo Castiñeyra
wrote:<br>
</div>
<blockquote
cite="mid:[email protected]"
type="cite">Hi guys,
<br>
<br>
We have users in Iran who are complaining about our app not
behaving the way MS-Word does when it comes to directional
ordering.
<br>
<br>
In most applications, if one writes the sentence "In an attack of
an F14 780 people died" in Persian with no RTL marks one gets the
following:
<br>
<br>
هواپیماهای F14 ۷۸۰ نفر را مصدوم کردند.
<br>
<br>
Obviously the ۷۸۰ number is missplaced, it should be on the left
side of F14. Even if the numerals were Persian, most applications
get the ۷۸۰ in the wrong position, and so does FriBiDi. My
understanding is there is only two ways of fixing it
<br>
<br>
1) Force the user to insert an RTL mark after F14
<br>
2) Detect that ۷۸۰ is writen in Persian numerals and automatically
treat it as an RTL run (maybe FriBiDi should do that?)
<br>
<br>
However, somehow MS-Word detects when the user changes the
keyboard layout and that affects the ordering as shown in the
following picture.
<br>
<br>
<a class="moz-txt-link-freetext" href="https://snag.gy/pbsh7g.jpg">https://snag.gy/pbsh7g.jpg</a>
</blockquote>
Fribidi implements the "Unicode Bidi Algorithm" (henceforth, UBA).
It is defined in techincal report #9 of the Unicode consortium. You
can view it at <a class="moz-txt-link-freetext" href="http://unicode.org/reports/tr9/">http://unicode.org/reports/tr9/</a>.<br>
<br>
You can see the BiDi parsing of the sentence you wrote at
<a class="moz-txt-link-freetext" href="http://unicode.org/cldr/utility/bidi.jsp?a=%D9%87%D9%88%D8%A7%D9%BE%DB%8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%DB%B7%DB%B8%DB%B0+%D9%86%D9%81%D8%B1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF%D9%86%D8%AF.+&p=RTL">http://unicode.org/cldr/utility/bidi.jsp?a=%D9%87%D9%88%D8%A7%D9%BE%DB%8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%DB%B7%DB%B8%DB%B0+%D9%86%D9%81%D8%B1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF%D9%86%D8%AF.+&p=RTL</a><br>
<br>
It is according to an older spec of the UBA, but I don't see
anything there that should make a difference.<br>
<br>
You can see that your problem is that both the 14 and the 780 are
catagorized as BiDi class EN. This means that the space between them
is a neutral between two same direction letters, and gets a left to
right direction, hence your problem.<br>
<br>
Within the UBA (which is what FreeBidi is doing), I'm afraid there
is no solution other than to insert an RLM, as you've suggested
(parsed sentence:
<a class="moz-txt-link-freetext" href="http://unicode.org/cldr/utility/bidi.jsp?a=%D9%87%D9%88%D8%A7%D9%BE%DB%8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%E2%80%8F%DB%B7%DB%B8%DB%B0+%D9%86%D9%81%D8%B1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF%D9%86%D8%AF.+&p=RTL">http://unicode.org/cldr/utility/bidi.jsp?a=%D9%87%D9%88%D8%A7%D9%BE%DB%8C%D9%85%D8%A7%D9%87%D8%A7%DB%8C+F14+%E2%80%8F%DB%B7%DB%B8%DB%B0+%D9%86%D9%81%D8%B1+%D8%B1%D8%A7+%D9%85%D8%B5%D8%AF%D9%88%D9%85+%DA%A9%D8%B1%D8%AF%D9%86%D8%AF.+&p=RTL</a>)<br>
<br>
As for Word, the reason it "works" is that it does not use the UBA
in order to render BiDi. It actually saves the keyboard language
with which each letter was typed. This is both non-standard
(obviously) and error prone. In my experience, it generates a lot of
user confusion as to how to type things so that they turn out
correctly on screen.<br>
<br>
In short, I have done nothing to help you solve your problem, but I
hope you now understand it better :-)<br>
<br>
Shachar<br>
</body>
</html>
--------------770A59DF87093D7816B73345--
--===============1473937666==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZnJpYmlkaSBt
YWlsaW5nIGxpc3QKZnJpYmlkaUBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m
cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmliaWRpCg==
--===============1473937666==--