log2vis() misbehaving with Arabic text?

--===============0648871444==
Content-Type: multipart/alternative; boundary=089e013cbd5eac2a9c0505f33278

--089e013cbd5eac2a9c0505f33278
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi all,
I=E2=80=99m a non-Arabic speaker working with Arabic text, and I sometimes =
see
results from log2vis() that don=E2=80=99t look quite right.

I have a test where I compare the results from two Arabic text reshapers.
The first is pyfribidi which is a thin wrapper around fribidi-0.19.6. The
second reshaper is a Python port of this:
https://github.com/agawish/Better-Arabic-Reshaper/

The two reshapers almost always agree on how to reshape a logical word.
However, in some cases they don=E2=80=99t. For instance, when given this lo=
gical
string:
u'\u062c\u0627\u0630\u0628\u064a\u0651\u0629'

log2vis() puts the Shadda in a different place than the BAR
(Better-Arabic-Reshaper):
log2vis: u=E2=80=99\ufe94\ufef4\u0651\ufe91\ufeab\ufe8e\ufe9f'
bar:     u=E2=80=99\ufe94\u0651\ufef4\ufe91\ufeab\ufe8e\ufe9f'

When I paste quoted versions of the visual representation of these strings
into Google, Google finds 23 instances of the fribidi version and ~37k
versions of the BAR version. To me, that=E2=80=99s a pretty strong argument=
 that
the fribidi version is incorrect.

I=E2=80=99m happy to file a bug if that=E2=80=99s appropriate, but I=E2=80=
=99d much rather learn
that I=E2=80=99m simply misusing the library or making some other mistake. =
Can
someone help me understand what I'm doing wrong?

Thanks in advance
Philip

--089e013cbd5eac2a9c0505f33278
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,<br>I=E2=80=99m a non-Arabic speaker working with A=
rabic text, and I sometimes see results from log2vis() that don=E2=80=99t l=
ook quite right. <br><br>I have a test where I compare the results from two=
 Arabic text reshapers. The first is pyfribidi which is a thin wrapper arou=
nd fribidi-0.19.6. The second reshaper is a Python port of this:<br><a href=
=3D"https://github.com/agawish/Better-Arabic-Reshaper/">https://github.com/=
agawish/Better-Arabic-Reshaper/</a><br><br>The two reshapers almost always =
agree on how to reshape a logical word. However, in some cases they don=E2=
=80=99t. For instance, when given this logical string:<br><span style=3D"fo=
nt-family:courier new,monospace">u&#39;\u062c\u0627\u0630\u0628\u064a\u0651=
\u0629&#39; </span><br><br>log2vis() puts the Shadda in a different place t=
han the BAR (Better-Arabic-Reshaper):<br><span style=3D"font-family:courier=
 new,monospace">log2vis: u=E2=80=99\ufe94\ufef4\u0651\ufe91\ufeab\ufe8e\ufe=
9f&#39;<br>bar:=C2=A0=C2=A0=C2=A0=C2=A0 u=E2=80=99\ufe94\u0651\ufef4\ufe91\=
ufeab\ufe8e\ufe9f&#39;</span><br><br>When I paste quoted versions of the vi=
sual representation of these strings into Google, Google finds 23 instances=
 of the fribidi version and ~37k versions of the BAR version. To me, that=
=E2=80=99s a pretty strong argument that the fribidi version is incorrect. =
<br><br>I=E2=80=99m happy to file a bug if that=E2=80=99s appropriate, but =
I=E2=80=99d much rather learn that I=E2=80=99m simply misusing the library =
or making some other mistake. Can someone help me understand what I&#39;m d=
oing wrong?<br><br>Thanks in advance<br>Philip<br></div>

--089e013cbd5eac2a9c0505f33278--

--===============0648871444==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
fribidi mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/fribidi

--===============0648871444==--