Re: Directionality Standard

--Apple-Mail-12-586879119
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=UTF-8;
	delsp=yes;
	format=flowed

Thank you.
So the answer is no. Unicode does not define the directionality of a =20
paragraph. Then I guess my next question should be why?
I think I have some explaining to do.
Unicode defines a very complex bidi behaviour of characters, and it =20
defines the beginning and ending of a paragraph (I assume). Yet, it =20
doesn't define what directionality this paragraph should take to =20
arrange these characters within the paragraph.
Defining the directionality of a paragraph is more important than =20
defining the language of a text. Yes, language tag can help language =20
aware devices and applications behave accordingly. But directionality =20=

definition is not about ' user friendly' behaviour of a text, it is =20
about reproducing the raw text, as intended by its Unicode encoding.
Understanding this issue I suppose, may be very easy or very =20
difficult, depending on to the extend you were exposed to rtl =20
experience. In the next paragraph, I write a Persian line, throwing a =20=

couple of English words within, and in left to right directionality =20
to give you an idea about what right to left users are experiencing =20
in everyday basis.
=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=8C=D9=
=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=A7=DB=
=8C =20
=D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality =
=D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=
=D8=AA.
In order to read the above phrase correctly in Persian, the order of =20
words should be as I numbered below (from right to left):
=D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=862 =D8=A7=D8=B23 Unicode4 =D8=A7=DB=8C=
=D9=865 =D8=A7=D8=B3=D8=AA6 =DA=A9=D9=877 =DA=86=D8=B1=D8=A78 =20
=D8=A8=D8=B1=D8=A7=DB=8C9 =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81=
10 directionality11 =D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 =20
=D9=86=DA=A9=D8=B1=D8=AF=D9=8713 =D8=A7=D8=B3=D8=AA14.

Of-course I can set this paragraph in my application to "rtl" and =20
thanks to wonders of bidi behaviour of characters, everything will be =20=

put in place:

=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=8C=D9=
=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=A7=DB=
=8C =20
=D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality =
=D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=
=D8=AA.

But I have absolutely no guarantee that my rtl text in an email, in a =20=

text message, in an online forum posting... will be received in rtl =20
setting. This perfectly Unicode encoded text is at the mercy of =20
applications, devices, mediums and platforms. And more likely than =20
not, my rtl paragraph will be received in ltr and in the order that I =20=

numbered above! Even in a more controlled situations such as word =20
processors, as a friend of mine has experienced, this Persian phrase =20
written in rtl setting of Nisus on a Mac, exported in a .doc format, =20
and opened on a Windows platform will produce an rtl, but 'Arabic' =20
document! not only an Arabic script document which is, but an Arabic =20
language document!

You can experiment this dilemma yourself. Set your application to rtl =20=

(which can be done in many applications), write something in English =20
or any Roman language. As long as the whole phrase is Roman, you only =20=

get a misplaced final period in far left. But if you throw a couple =20
of Hebrew words within the phrase, then you'll see what a wrong =20
directionality setting can do to your English. Of-course you are not =20
exposed to this dilemma because the default directionality of all =20
computerized devices and applications is left to right. But it gives =20
you an idea what rtl users are going through in everyday basis.

Again, this is not about requesting a convenience. It is about =20
requesting Unicode to do what it is set to do. Unicode encodes bidi =20
behaviour of characters, the beginning of a paragraph, the end of a =20
paragraph. It must encode its directionality too.

Behnam


On 17-Dec-07, at 4:20 AM, Stephane Bortzmeyer wrote:

> On Sat, Dec 15, 2007 at 11:08:40AM -0500,
>  Behnam <[email protected]> wrote
>  a message of 78 lines which said:
>
>> Is there any Unicode standard to identify a text? i.e. primary
>> script>directionality>language?
>
> Not an Unicode standard but, yes, there is a standard to tag texts to
> indicate language, script, etc. It's RFC 4646. See
> http://www.langtag.net/ for a start.


--Apple-Mail-12-586879119
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=UTF-8

<html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; ">
Thank you.<div>So the answer is no. Unicode does not define the =
directionality of a paragraph. Then I guess my next question should be =
why?</div><div>I think I have some explaining to do.</div><div>Unicode =
defines a very complex bidi behaviour of characters, and it defines the =
beginning and ending of a paragraph (I assume). Yet, it doesn't define =
what directionality this paragraph should take to arrange these =
characters within the paragraph.</div><div>Defining the directionality =
of a paragraph is more important than defining the language of a text. =
Yes, language tag can help language aware devices =
and=C2=A0applications=C2=A0behave accordingly. But directionality =
definition is not about ' user friendly' behaviour of a text, it is =
about reproducing the raw text, as intended by its Unicode =
encoding.</div><div>Understanding this issue I suppose, may be very easy =
or very difficult, depending on to the extend you were exposed to rtl =
experience. In the next paragraph, I write a Persian line, throwing a =
couple of English words within, and in left to right directionality to =
give you an idea about what right to left users are experiencing in =
everyday basis.</div><div>=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2=
 Unicode =D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=
=A7 =D8=A8=D8=B1=D8=A7=DB=8C =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=
=81 directionality =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=
=D9=87 =D8=A7=D8=B3=D8=AA.</div><div>In order to read the above phrase =
correctly in Persian, the order of words should be as I numbered below =
(from right to left):</div><div>=D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=862 =
=D8=A7=D8=B23 Unicode4 =D8=A7=DB=8C=D9=865 =D8=A7=D8=B3=D8=AA6 =DA=A9=D9=87=
7 =DA=86=D8=B1=D8=A78 =D8=A8=D8=B1=D8=A7=DB=8C9 =D9=BE=D8=A7=D8=B1=D8=A7=DA=
=AF=D8=B1=D8=A7=D9=8110 directionality11 =D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 =
=D9=86=DA=A9=D8=B1=D8=AF=D9=8713 =D8=A7=D8=B3=D8=AA14.</div><div><br =
class=3D"webkit-block-placeholder"></div><div>Of-course I can set this =
paragraph in my application to "rtl" and thanks to wonders of bidi =
behaviour of characters, everything will be put in place:</div><div =
style=3D"direction: rtl;"><br =
class=3D"webkit-block-placeholder"></div><div><div style=3D"direction: =
rtl;">=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=
=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=
=A7=DB=8C =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 =
directionality =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87=
 =D8=A7=D8=B3=D8=AA.</div><div style=3D"direction: rtl;"><br =
class=3D"webkit-block-placeholder"></div><div style=3D"direction: ltr; =
">But I have absolutely no guarantee that my rtl text in an email, in a =
text message, in an online forum posting... will be received in rtl =
setting. This perfectly Unicode encoded text is at the mercy of =
applications, devices, mediums and platforms. And more likely than not, =
my rtl paragraph will be received in ltr and in the order that I =
numbered above! Even in a more controlled situations such as word =
processors, as a friend of mine has experienced, this Persian phrase =
written in rtl setting of Nisus on a Mac, exported in a .doc format, and =
opened on a Windows platform will produce an rtl, but 'Arabic' document! =
not only an Arabic script document which is, but an Arabic language =
document!</div><div style=3D"direction: ltr; "><br =
class=3D"webkit-block-placeholder"></div><div style=3D"direction: ltr; =
">You can experiment this dilemma yourself. Set your application to rtl =
(which can be done in many applications), write something in English or =
any Roman language. As long as the whole phrase is Roman, you only get a =
misplaced final period in far left. But if you throw a couple of Hebrew =
words within the phrase, then you'll see what a wrong directionality =
setting can do to your English. Of-course you are not exposed to this =
dilemma because the default directionality of all computerized devices =
and applications is left to right. But it gives you an idea what rtl =
users are going through in everyday basis.</div><div style=3D"direction: =
ltr; "><br class=3D"webkit-block-placeholder"></div><div =
style=3D"direction: ltr; ">Again, this is not about requesting a =
convenience. It is about requesting Unicode to do what it is set to do. =
Unicode encodes bidi behaviour of characters, the beginning of a =
paragraph, the end of a paragraph. It must encode its directionality =
too.</div><div style=3D"direction: ltr; "><br =
class=3D"webkit-block-placeholder"></div><div style=3D"direction: ltr; =
">Behnam</div><div style=3D"direction: rtl;"><br =
class=3D"webkit-block-placeholder"></div></div><div><br><div><div>On =
17-Dec-07, at 4:20 AM, Stephane Bortzmeyer wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div>On =
Sat, Dec 15, 2007 at 11:08:40AM -0500,</div><div>=C2=A0Behnam &lt;<a =
href=3D"mailto:[email protected]">[email protected]</a>&gt; =
wrote=C2=A0</div><div>=C2=A0a message of 78 lines which =
said:</div><div><br></div> <blockquote type=3D"cite"><div>Is there any =
Unicode standard to identify a text? i.e. =
primary</div><div>script&gt;directionality&gt;language?</div> =
</blockquote><div><br></div><div>Not an Unicode standard but, yes, there =
is a standard to tag texts to</div><div>indicate language, script, etc. =
It's RFC 4646. See</div><div><a =
href=3D"http://www.langtag.net">http://www.langtag.net</a>/ for a =
start.</div> </blockquote></div><br></div></body></html>=

--Apple-Mail-12-586879119--