RE: Directionality Standard
"Jony Rosenne" <[email protected]> Tue, 18 Dec 2007 06:48:41 +0200
| Newsgroups | gmane.text.unicode.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multipart message in MIME format. ------=_NextPart_000_0038_01C84142.07E40490 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The language tag in HTML does not define directionality. Use a top level = dir tag. =20 See http://www.w3.org/TR/html401/struct/global.html#edef-HTML =20 Jony =20 From: [email protected] [mailto:[email protected]] On = Behalf Of Behnam Sent: Tuesday, December 18, 2007 4:34 AM To: Mark Davis Cc: Stephane Bortzmeyer; ([email protected]) List; Otto Stolz; Magda = Danish (Unicode) Subject: Re: Directionality Standard =20 Thank you very much Mark. Should I conclude that in any case the rtl paragraph is not reproduced = the device or application is not Unicode compliant? This amounts to many! And why do we need language tag in HTML or other formats if this = property is defined within the encoded paragraph? -B =20 On 17-Dec-07, at 8:58 PM, Mark Davis wrote: There may be some misunderstanding. Unicode does define the default = direction of a paragraph for use with the bidi algorithm (which = determines the ordering of characters containing bidirectional scripts = like Arabic or Hebrew).=20 See http://unicode.org/reports/tr9/ Mark On Dec 17, 2007 4:23 PM, Behnam <[email protected] = <mailto:[email protected]> > wrote: Thank you. So the answer is no. Unicode does not define the directionality of a = paragraph. Then I guess my next question should be why? I think I have some explaining to do. Unicode defines a very complex bidi behaviour of characters, and it = defines the beginning and ending of a paragraph (I assume). Yet, it = doesn't define what directionality this paragraph should take to arrange = these characters within the paragraph.=20 Defining the directionality of a paragraph is more important than = defining the language of a text. Yes, language tag can help language = aware devices and applications behave accordingly. But directionality = definition is not about ' user friendly' behaviour of a text, it is = about reproducing the raw text, as intended by its Unicode encoding.=20 Understanding this issue I suppose, may be very easy or very difficult, = depending on to the extend you were exposed to rtl experience. In the = next paragraph, I write a Persian line, throwing a couple of English = words within, and in left to right directionality to give you an idea = about what right to left users are experiencing in everyday basis.=20 =D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode = =D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 = =D8=A8=D8=B1=D8=A7=DB=8C = =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality = =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 = =D8=A7=D8=B3=D8=AA. In order to read the above phrase correctly in Persian, the order of = words should be as I numbered below (from right to left):=20 =D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=862 =D8=A7=D8=B23 Unicode4 = =D8=A7=DB=8C=D9=865 =D8=A7=D8=B3=D8=AA6 =DA=A9=D9=877 = =DA=86=D8=B1=D8=A78 =D8=A8=D8=B1=D8=A7=DB=8C9 = =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=8110 directionality11 = =D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 =D9=86=DA=A9=D8=B1=D8=AF=D9=8713 = =D8=A7=D8=B3=D8=AA14. =20 Of-course I can set this paragraph in my application to "rtl" and thanks = to wonders of bidi behaviour of characters, everything will be put in = place:=20 =20 =D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2 Unicode = =D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 = =D8=A8=D8=B1=D8=A7=DB=8C = =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 directionality = =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 = =D8=A7=D8=B3=D8=AA. =20 But I have absolutely no guarantee that my rtl text in an email, in a = text message, in an online forum posting... will be received in rtl = setting. This perfectly Unicode encoded text is at the mercy of = applications, devices, mediums and platforms. And more likely than not, = my rtl paragraph will be received in ltr and in the order that I = numbered above! Even in a more controlled situations such as word = processors, as a friend of mine has experienced, this Persian phrase = written in rtl setting of Nisus on a Mac, exported in a .doc format, and = opened on a Windows platform will produce an rtl, but 'Arabic' document! = not only an Arabic script document which is, but an Arabic language = document!=20 =20 You can experiment this dilemma yourself. Set your application to rtl = (which can be done in many applications), write something in English or = any Roman language. As long as the whole phrase is Roman, you only get a = misplaced final period in far left. But if you throw a couple of Hebrew = words within the phrase, then you'll see what a wrong directionality = setting can do to your English. Of-course you are not exposed to this = dilemma because the default directionality of all computerized devices = and applications is left to right. But it gives you an idea what rtl = users are going through in everyday basis.=20 =20 Again, this is not about requesting a convenience. It is about = requesting Unicode to do what it is set to do. Unicode encodes bidi = behaviour of characters, the beginning of a paragraph, the end of a = paragraph. It must encode its directionality too.=20 =20 Behnam =20 =20 On 17-Dec-07, at 4:20 AM, Stephane Bortzmeyer wrote: On Sat, Dec 15, 2007 at 11:08:40AM -0500, Behnam <[email protected]> wrote=20 a message of 78 lines which said: =20 Is there any Unicode standard to identify a text? i.e. primary script>directionality>language? =20 Not an Unicode standard but, yes, there is a standard to tag texts to=20 indicate language, script, etc. It's RFC 4646. See http://www.langtag.net/ for a start. =20 --=20 Mark =20 ------=_NextPart_000_0038_01C84142.07E40490 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"> <meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)"> <style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} span.EmailStyle17 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in;} div.Section1 {page:Section1;} --> </style> <!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3DEN-US link=3Dblue vlink=3Dpurple style=3D'word-wrap: = break-word; -webkit-nbsp-mode: space;-webkit-line-break: after-white-space'> <div class=3DSection1> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'>The language tag in HTML does not define directionality. = Use a top level dir tag.<o:p></o:p></span></p> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'><o:p> </o:p></span></p> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'>See <a href=3D"http://www.w3.org/TR/html401/struct/global.html#edef-HTML">http:/= /www.w3.org/TR/html401/struct/global.html#edef-HTML</a><o:p></o:p></span>= </p> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'><o:p> </o:p></span></p> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'>Jony<o:p></o:p></span></p> <p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; color:#1F497D'><o:p> </o:p></span></p> <div> <div style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt = 0in 0in 0in'> <p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span>= </b><span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> [email protected] [mailto:[email protected]] <b>On = Behalf Of </b>Behnam<br> <b>Sent:</b> Tuesday, December 18, 2007 4:34 AM<br> <b>To:</b> Mark Davis<br> <b>Cc:</b> Stephane Bortzmeyer; ([email protected]) List; Otto Stolz; = Magda Danish (Unicode)<br> <b>Subject:</b> Re: Directionality Standard<o:p></o:p></span></p> </div> </div> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Thank you very much Mark.<o:p></o:p></p> <div> <p class=3DMsoNormal>Should I conclude that in any case the rtl = paragraph is not reproduced the device or application is not Unicode = compliant?<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>This amounts to many!<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>And why do we need language tag in HTML or other = formats if this property is defined within the encoded paragraph?<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>-B<o:p></o:p></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> <div> <div> <p class=3DMsoNormal>On 17-Dec-07, at 8:58 PM, Mark Davis = wrote:<o:p></o:p></p> </div> <p class=3DMsoNormal><br> <br> <o:p></o:p></p> <p class=3DMsoNormal style=3D'margin-bottom:12.0pt'>There may be some misunderstanding. Unicode does define the default direction of a = paragraph for use with the bidi algorithm (which determines the ordering of characters containing bidirectional scripts like Arabic or Hebrew). <br> <br> See <a = href=3D"http://unicode.org/reports/tr9/">http://unicode.org/reports/tr9/<= /a><br> <br> Mark<o:p></o:p></p> <div> <p class=3DMsoNormal>On Dec 17, 2007 4:23 PM, Behnam <<a href=3D"mailto:[email protected]">[email protected] </a>> = wrote:<o:p></o:p></p> <div> <p class=3DMsoNormal>Thank you.<o:p></o:p></p> <div> <p class=3DMsoNormal>So the answer is no. Unicode does not define the directionality of a paragraph. Then I guess my next question should be = why?<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>I think I have some explaining to = do.<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>Unicode defines a very complex bidi behaviour of = characters, and it defines the beginning and ending of a paragraph (I assume). Yet, = it doesn't define what directionality this paragraph should take to arrange = these characters within the paragraph. <o:p></o:p></p> </div> <div> <p class=3DMsoNormal>Defining the directionality of a paragraph is more = important than defining the language of a text. Yes, language tag can help = language aware devices and applications behave accordingly. But = directionality definition is not about ' user friendly' behaviour of a text, it is = about reproducing the raw text, as intended by its Unicode encoding. = <o:p></o:p></p> </div> <div> <p class=3DMsoNormal>Understanding this issue I suppose, may be very = easy or very difficult, depending on to the extend you were exposed to rtl = experience. In the next paragraph, I write a Persian line, throwing a couple of English = words within, and in left to right directionality to give you an idea about = what right to left users are experiencing in everyday basis. <o:p></o:p></p> </div> <div> <p class=3DMsoNormal><span lang=3DAR-SA = dir=3DRTL>=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 =D8=A7=D8=B2</span><span = dir=3DLTR></span><span dir=3DLTR></span> Unicode <span lang=3DAR-SA = dir=3DRTL>=D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 = =DA=86=D8=B1=D8=A7 =D8=A8=D8=B1=D8=A7=DB=8C = =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81</span><span dir=3DLTR></span><span dir=3DLTR></span> directionality <span = lang=3DAR-SA dir=3DRTL>=D8=AA=D8=A8=DB=8C=DB=8C=D9=86 =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=D8=AA</span><span = dir=3DLTR></span><span dir=3DLTR></span>.<o:p></o:p></p> </div> <div> <p class=3DMsoNormal>In order to read the above phrase correctly in = Persian, the order of words should be as I numbered below (from right to left): = <o:p></o:p></p> </div> <div> <p class=3DMsoNormal><span lang=3DAR-SA = dir=3DRTL>=D9=BE=D8=B1=D8=B3=D8=B41 =D9=85=D9=862 = =D8=A7=D8=B23</span><span dir=3DLTR></span><span dir=3DLTR></span> Unicode4 <span lang=3DAR-SA = dir=3DRTL>=D8=A7=DB=8C=D9=865 =D8=A7=D8=B3=D8=AA6 =DA=A9=D9=877 = =DA=86=D8=B1=D8=A78 =D8=A8=D8=B1=D8=A7=DB=8C9 =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=8110</span><span = dir=3DLTR></span><span dir=3DLTR></span> directionality11 <span lang=3DAR-SA dir=3DRTL>=D8=AA=D8=A8=DB=8C=DB=8C=D9=8612 = =D9=86=DA=A9=D8=B1=D8=AF=D9=8713 =D8=A7=D8=B3=D8=AA14</span><span = dir=3DLTR></span><span dir=3DLTR></span>.<o:p></o:p></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <div> <p class=3DMsoNormal>Of-course I can set this paragraph in my = application to "rtl" and thanks to wonders of bidi behaviour of characters, everything will be put in place: <o:p></o:p></p> </div> <div> <p class=3DMsoNormal dir=3DRTL = style=3D'text-align:right;direction:rtl;unicode-bidi: embed'><span dir=3DLTR><o:p> </o:p></span></p> </div> <div> <div> <p class=3DMsoNormal dir=3DRTL = style=3D'text-align:right;direction:rtl;unicode-bidi: embed'><span lang=3DAR-SA>=D9=BE=D8=B1=D8=B3=D8=B4 =D9=85=D9=86 = =D8=A7=D8=B2 </span><span dir=3DLTR>Unicode</span><span dir=3DRTL></span><span lang=3DAR-SA><span dir=3DRTL></span> = =D8=A7=DB=8C=D9=86 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =DA=86=D8=B1=D8=A7 = =D8=A8=D8=B1=D8=A7=DB=8C =D9=BE=D8=A7=D8=B1=D8=A7=DA=AF=D8=B1=D8=A7=D9=81 </span><span = dir=3DLTR>directionality</span><span dir=3DRTL></span><span lang=3DAR-SA><span dir=3DRTL></span> =D8=AA=D8=A8=DB=8C=DB=8C=D9=86 = =D9=86=DA=A9=D8=B1=D8=AF=D9=87 =D8=A7=D8=B3=D8=AA.</span><span = lang=3DHE><o:p></o:p></span></p> </div> <div> <p class=3DMsoNormal dir=3DRTL = style=3D'text-align:right;direction:rtl;unicode-bidi: embed'><span lang=3DHE><o:p> </o:p></span></p> </div> <div> <p class=3DMsoNormal>But I have absolutely no guarantee that my rtl text = in an email, in a text message, in an online forum posting... will be received = in rtl setting. This perfectly Unicode encoded text is at the mercy of = applications, devices, mediums and platforms. And more likely than not, my rtl = paragraph will be received in ltr and in the order that I numbered above! Even in a = more controlled situations such as word processors, as a friend of mine has experienced, this Persian phrase written in rtl setting of Nisus on a = Mac, exported in a .doc format, and opened on a Windows platform will produce = an rtl, but 'Arabic' document! not only an Arabic script document which is, = but an Arabic language document! <span lang=3DHE = dir=3DRTL><o:p></o:p></span></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <div> <p class=3DMsoNormal>You can experiment this dilemma yourself. Set your application to rtl (which can be done in many applications), write = something in English or any Roman language. As long as the whole phrase is Roman, you = only get a misplaced final period in far left. But if you throw a couple of = Hebrew words within the phrase, then you'll see what a wrong directionality = setting can do to your English. Of-course you are not exposed to this dilemma = because the default directionality of all computerized devices and applications = is left to right. But it gives you an idea what rtl users are going through in = everyday basis. <o:p></o:p></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <div> <p class=3DMsoNormal>Again, this is not about requesting a convenience. = It is about requesting Unicode to do what it is set to do. Unicode encodes = bidi behaviour of characters, the beginning of a paragraph, the end of a = paragraph. It must encode its directionality too. <o:p></o:p></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <div> <p class=3DMsoNormal>Behnam<o:p></o:p></p> </div> <div> <p class=3DMsoNormal dir=3DRTL = style=3D'text-align:right;direction:rtl;unicode-bidi: embed'><span dir=3DLTR><o:p> </o:p></span></p> </div> </div> <div> <p class=3DMsoNormal><span lang=3DHE = dir=3DRTL><o:p> </o:p></span></p> <div> <div> <p class=3DMsoNormal>On 17-Dec-07, at 4:20 AM, Stephane Bortzmeyer = wrote:<o:p></o:p></p> </div> <p class=3DMsoNormal><br> <br> <o:p></o:p></p> <div> <p class=3DMsoNormal>On Sat, Dec 15, 2007 at 11:08:40AM = -0500,<o:p></o:p></p> </div> <div> <p class=3DMsoNormal> Behnam <<a = href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>> = wrote <o:p></o:p></p> </div> <div> <p class=3DMsoNormal> a message of 78 lines which = said:<o:p></o:p></p> </div> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <blockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'> <div> <p class=3DMsoNormal>Is there any Unicode standard to identify a text? = i.e. primary<o:p></o:p></p> </div> <div> <p = class=3DMsoNormal>script>directionality>language?<o:p></o:p></p> </div> </blockquote> <div> <p class=3DMsoNormal><o:p> </o:p></p> </div> <div> <p class=3DMsoNormal>Not an Unicode standard but, yes, there is a = standard to tag texts to <o:p></o:p></p> </div> <div> <p class=3DMsoNormal>indicate language, script, etc. It's RFC 4646. = See<o:p></o:p></p> </div> <div> <p class=3DMsoNormal><a href=3D"http://www.langtag.net" = target=3D"_blank">http://www.langtag.net</a>/ for a start.<o:p></o:p></p> </div> </div> <p class=3DMsoNormal><o:p> </o:p></p> </div> </div> </div> <p class=3DMsoNormal><br> <br clear=3Dall> <br> -- <br> Mark<o:p></o:p></p> </div> <p class=3DMsoNormal><o:p> </o:p></p> </div> </div> </body> </html> ------=_NextPart_000_0038_01C84142.07E40490--