Re: Directionality Standard

--Apple-Mail-14-594686513
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=UTF-8;
	delsp=yes;
	format=flowed

Thank you very much Mark.
Should I conclude that in any case the rtl paragraph is not =20
reproduced the device or application is not Unicode compliant?
This amounts to many!
And why do we need language tag in HTML or other formats if this =20
property is defined within the encoded paragraph?
-B

On 17-Dec-07, at 8:58 PM, Mark Davis wrote:

> There may be some misunderstanding. Unicode does define the default =20=

> direction of a paragraph for use with the bidi algorithm (which =20
> determines the ordering of characters containing bidirectional =20
> scripts like Arabic or Hebrew).
>
> See http://unicode.org/reports/tr9/
>
> Mark
>
> On Dec 17, 2007 4:23 PM, Behnam <[email protected] > wrote:
> Thank you.
> So the answer is no. Unicode does not define the directionality of =20
> a paragraph. Then I guess my next question should be why?
> I think I have some explaining to do.
> Unicode defines a very complex bidi behaviour of characters, and it =20=

> defines the beginning and ending of a paragraph (I assume). Yet, it =20=

> doesn't define what directionality this paragraph should take to =20
> arrange these characters within the paragraph.
> Defining the directionality of a paragraph is more important than =20
> defining the language of a text. Yes, language tag can help =20
> language aware devices and applications behave accordingly. But =20
> directionality definition is not about ' user friendly' behaviour =20
> of a text, it is about reproducing the raw text, as intended by its =20=

> Unicode encoding.
> Understanding this issue I suppose, may be very easy or very =20
> difficult, depending on to the extend you were exposed to rtl =20
> experience. In the next paragraph, I write a Persian line, throwing =20=

> a couple of English words within, and in left to right =20
> directionality to give you an idea about what right to left users =20
> are experiencing in everyday basis.
> =D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=8C=D9=
=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=A7=DB=
=8C =20
> =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality =
=D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=
=D8=AA.
> In order to read the above phrase correctly in Persian, the order =20
> of words should be as I numbered below (from right to left):
> =D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=862 =D8=A7=D8=B23 Unicode4 =D8=A7=DB=
=8C=D9=865 =D8=A7=D8=B3=D8=AA6 =DA=A9=D9=877 =DA=86=D8=B1=D8=A78 =20
> =D8=A8=D8=B1=D8=A7=DB=8C9 =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81=
10 directionality11 =D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 =20
> =D9=86=DA=A9=D8=B1=D8=AF=D9=8713 =D8=A7=D8=B3=D8=AA14.
>
> Of-course I can set this paragraph in my application to "rtl" and =20
> thanks to wonders of bidi behaviour of characters, everything will =20
> be put in place:
>
> =D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=8C=D9=
=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=A7=DB=
=8C =20
> =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality =
=D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=
=D8=AA.
>
> But I have absolutely no guarantee that my rtl text in an email, in =20=

> a text message, in an online forum posting... will be received in =20
> rtl setting. This perfectly Unicode encoded text is at the mercy of =20=

> applications, devices, mediums and platforms. And more likely than =20
> not, my rtl paragraph will be received in ltr and in the order that =20=

> I numbered above! Even in a more controlled situations such as word =20=

> processors, as a friend of mine has experienced, this Persian =20
> phrase written in rtl setting of Nisus on a Mac, exported in a .doc =20=

> format, and opened on a Windows platform will produce an rtl, but =20
> 'Arabic' document! not only an Arabic script document which is, but =20=

> an Arabic language document!
>
> You can experiment this dilemma yourself. Set your application to =20
> rtl (which can be done in many applications), write something in =20
> English or any Roman language. As long as the whole phrase is =20
> Roman, you only get a misplaced final period in far left. But if =20
> you throw a couple of Hebrew words within the phrase, then you'll =20
> see what a wrong directionality setting can do to your English. Of-=20
> course you are not exposed to this dilemma because the default =20
> directionality of all computerized devices and applications is left =20=

> to right. But it gives you an idea what rtl users are going through =20=

> in everyday basis.
>
> Again, this is not about requesting a convenience. It is about =20
> requesting Unicode to do what it is set to do. Unicode encodes bidi =20=

> behaviour of characters, the beginning of a paragraph, the end of a =20=

> paragraph. It must encode its directionality too.
>
> Behnam
>
>
> On 17-Dec-07, at 4:20 AM, Stephane Bortzmeyer wrote:
>
>> On Sat, Dec 15, 2007 at 11:08:40AM -0500,
>>  Behnam <[email protected]> wrote
>>  a message of 78 lines which said:
>>
>>> Is there any Unicode standard to identify a text? i.e. primary
>>> script>directionality>language?
>>
>> Not an Unicode standard but, yes, there is a standard to tag texts to
>> indicate language, script, etc. It's RFC 4646. See
>> http://www.langtag.net/ for a start.
>
>
>
>
> --=20
> Mark


--Apple-Mail-14-594686513
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=UTF-8

<html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; ">Thank you very much =
Mark.<div>Should I conclude that in any case the rtl paragraph is not =
reproduced the device or application is not Unicode =
compliant?</div><div>This amounts to many!</div><div>And why do we need =
language tag in HTML or other formats if this property is defined within =
the encoded paragraph?</div><div>-B</div><div><br><div><div>On =
17-Dec-07, at 8:58 PM, Mark Davis wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite">There may =
be some misunderstanding. Unicode does define the default direction of a =
paragraph for use with the bidi algorithm (which determines the ordering =
of characters containing bidirectional scripts like Arabic or Hebrew). =
<br><br>See <a =
href=3D"http://unicode.org/reports/tr9/">http://unicode.org/reports/tr9/</=
a><br><br>Mark<br><br><div class=3D"gmail_quote">On Dec 17, 2007 4:23 =
PM, Behnam &lt;<a =
href=3D"mailto:[email protected]">[email protected] </a>&gt; =
wrote:<br><blockquote class=3D"gmail_quote" style=3D"border-left: 1px =
solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: =
1ex;"><div style=3D""> Thank you.<div>So the answer is no. Unicode does =
not define the directionality of a paragraph. Then I guess my next =
question should be why?</div><div>I think I have some explaining to =
do.</div><div>Unicode defines a very complex bidi behaviour of =
characters, and it defines the beginning and ending of a paragraph (I =
assume). Yet, it doesn't define what directionality this paragraph =
should take to arrange these characters within the paragraph. =
</div><div>Defining the directionality of a paragraph is more important =
than defining the language of a text. Yes, language tag can help =
language aware devices and=C2=A0applications=C2=A0behave accordingly. =
But directionality definition is not about ' user friendly' behaviour of =
a text, it is about reproducing the raw text, as intended by its Unicode =
encoding. </div><div>Understanding this issue I suppose, may be very =
easy or very difficult, depending on to the extend you were exposed to =
rtl experience. In the next paragraph, I write a Persian line, throwing =
a couple of English words within, and in left to right directionality to =
give you an idea about what right to left users are experiencing in =
everyday basis. </div><div>=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=
=B2 Unicode =D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=
=D8=A7 =D8=A8=D8=B1=D8=A7=DB=8C =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=
=D9=81 directionality =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=
=AF=D9=87 =D8=A7=D8=B3=D8=AA.</div><div>In order to read the above =
phrase correctly in Persian, the order of words should be as I numbered =
below (from right to left): </div><div>=D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=
=862 =D8=A7=D8=B23 Unicode4 =D8=A7=DB=8C=D9=865 =D8=A7=D8=B3=D8=AA6 =
=DA=A9=D9=877 =DA=86=D8=B1=D8=A78 =D8=A8=D8=B1=D8=A7=DB=8C9 =
=D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=8110 directionality11 =
=D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 =D9=86=DA=A9=D8=B1=D8=AF=D9=8713 =
=D8=A7=D8=B3=D8=AA14.</div><div><br></div><div>Of-course I can set this =
paragraph in my application to "rtl" and thanks to wonders of bidi =
behaviour of characters, everything will be put in place: </div><div =
style=3D"direction: rtl;"><br></div><div><div style=3D"direction: =
rtl;">=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode =D8=A7=DB=
=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=
=A7=DB=8C =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 =
directionality =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87=
 =D8=A7=D8=B3=D8=AA.</div><div style=3D"direction: rtl;"><br></div><div =
style=3D"direction: ltr;"> But I have absolutely no guarantee that my =
rtl text in an email, in a text message, in an online forum posting... =
will be received in rtl setting. This perfectly Unicode encoded text is =
at the mercy of applications, devices, mediums and platforms. And more =
likely than not, my rtl paragraph will be received in ltr and in the =
order that I numbered above! Even in a more controlled situations such =
as word processors, as a friend of mine has experienced, this Persian =
phrase written in rtl setting of Nisus on a Mac, exported in a .doc =
format, and opened on a Windows platform will produce an rtl, but =
'Arabic' document! not only an Arabic script document which is, but an =
Arabic language document! </div><div style=3D"direction: =
ltr;"><br></div><div style=3D"direction: ltr;">You can experiment this =
dilemma yourself. Set your application to rtl (which can be done in many =
applications), write something in English or any Roman language. As long =
as the whole phrase is Roman, you only get a misplaced final period in =
far left. But if you throw a couple of Hebrew words within the phrase, =
then you'll see what a wrong directionality setting can do to your =
English. Of-course you are not exposed to this dilemma because the =
default directionality of all computerized devices and applications is =
left to right. But it gives you an idea what rtl users are going through =
in everyday basis. </div><div style=3D"direction: ltr;"><br></div><div =
style=3D"direction: ltr;">Again, this is not about requesting a =
convenience. It is about requesting Unicode to do what it is set to do. =
Unicode encodes bidi behaviour of characters, the beginning of a =
paragraph, the end of a paragraph. It must encode its directionality =
too. </div><div style=3D"direction: ltr;"><br></div><div =
style=3D"direction: ltr;">Behnam</div><div style=3D"direction: =
rtl;"><br></div></div><div><br><div><div>On 17-Dec-07, at 4:20 AM, =
Stephane Bortzmeyer wrote:</div><br><blockquote type=3D"cite"> <div>On =
Sat, Dec 15, 2007 at 11:08:40AM -0500,</div><div>=C2=A0Behnam &lt;<a =
href=3D"mailto:[email protected]" =
target=3D"_blank">[email protected]</a>&gt; wrote=C2=A0</div><div>=C2=
=A0a message of 78 lines which said:</div><div><br> </div> <blockquote =
type=3D"cite"><div>Is there any Unicode standard to identify a text? =
i.e. primary</div><div>script&gt;directionality&gt;language?</div> =
</blockquote><div><br></div><div>Not an Unicode standard but, yes, there =
is a standard to tag texts to </div><div>indicate language, script, etc. =
It's RFC 4646. See</div><div><a href=3D"http://www.langtag.net" =
target=3D"_blank">http://www.langtag.net</a>/ for a start.</div> =
</blockquote></div><br></div></div></blockquote></div> <br><br =
clear=3D"all"><br>-- <br>Mark</blockquote></div><br></div></body></html>=

--Apple-Mail-14-594686513--