Re: Unicode UTF-8 Compatible Version of Gutcheck
James Adcock <[email protected]> Thu, 26 Feb 2015 17:13:53 -0800
| Newsgroups | gmane.culture.literature.e-books.gutenberg.volunteers |
|---|---|
| Message-ID | <[email protected]> |
--===============9171387709408488791==
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_003C_01D051E7.98D2C730"
Content-Language: en-us
------=_NextPart_000_003C_01D051E7.98D2C730
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
>Huh. Not obvious to me what he is doing in bookloupe.
What I meant was that it was not obvious to me what kind of coding =
approach was being used in bookloupe. What I did in gutcheck_u was a =
straightforward translation of the programming from being an 8-bit char =
program, to being a 16-bit widechar program. There was some places in =
gutcheck where valid glyph-range assumptions were hard-wired in for the =
127-255 range =E2=80=93 an amalgamation of code pages, and those needed =
to be changed to something more =E2=80=9Creasonable=E2=80=9D assuming =
that people are using Unicode, and using Unicode, hopefully, for some =
sensible reason. And there was a large number of implied-handiness =
tests for straight-quote and straight-double-quote and/or =
straight-apostrophe, and those tests all become somewhat easier and more =
sensible on the Unicode curly handedness versions. In any case =
gutcheck_u attempts to check the sanity of both the straight and curly =
versions =E2=80=93 whatever it finds, and if it finds a mix [at least =
within a paragraph], it should report that too. The straights tests are =
the same as before, assuming I didn=E2=80=99t accidentally break =
something.
=20
Probably what gutcheck_u ought to have is some switches to say which of =
the Latin code pages are being used, vs. not used, so that the testing =
can be more reasoned. It should also probably test better passing =
common non-alphas, and querying uncommon non-alphas =E2=80=93 since =
there are a lot more non-alpha out there in Unicode world which PG =
submitters don=E2=80=99t actually intentionally use very often.
=20
In any case I=E2=80=99m thinking that submitters are going to be wanting =
to submit curlies more and more often. The straights are beginning to =
look increasingly anachronistic. =20
=20
Best. Jim A.
=20
------=_NextPart_000_003C_01D051E7.98D2C730
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta =
http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta =
name=3DGenerator content=3D"Microsoft Word 15 (filtered =
medium)"><title>Message</title><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite lang=3DDE =
link=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p =
class=3DMsoNormal><b><span lang=3DEN-US =
style=3D'color:#1F497D'>></span></b><span lang=3DEN-US =
style=3D'color:#1F497D'>Huh. Not obvious to me what he is doing in =
bookloupe.</span><span lang=3DEN-US><o:p></o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif'><br></span><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'>What I meant was that it was not obvious to =
me what kind of coding approach was being used in bookloupe.=C2=A0 What =
I did in gutcheck_u was a straightforward translation of the programming =
from being an 8-bit char program, to being a 16-bit widechar =
program.=C2=A0 There was some places in gutcheck where valid glyph-range =
assumptions were hard-wired in for the 127-255 range =E2=80=93 an =
amalgamation of code pages, and those needed to be changed to something =
more =E2=80=9Creasonable=E2=80=9D assuming that people are using =
Unicode, and using Unicode, hopefully, for some sensible reason.=C2=A0 =
And there was a large number of implied-handiness tests for =
straight-quote and straight-double-quote and/or straight-apostrophe, and =
those tests all become somewhat easier and more sensible on the Unicode =
curly handedness versions. In any case gutcheck_u attempts to check the =
sanity of both the straight and curly versions =E2=80=93 whatever it =
finds, and if it finds a mix [at least within a paragraph], it should =
report that too. The straights tests are the same as before, assuming I =
didn=E2=80=99t accidentally break something.<o:p></o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'><o:p> </o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'>Probably what gutcheck_u ought to have is =
some switches to say which of the Latin code pages are being used, vs. =
not used, so that the testing can be more reasoned.=C2=A0 It should also =
probably test better passing common non-alphas, and querying uncommon =
non-alphas =E2=80=93 since there are a lot more non-alpha out there in =
Unicode world which PG submitters don=E2=80=99t actually intentionally =
use very often.<o:p></o:p></span></p><p class=3DMsoNormal><span =
lang=3DEN-US style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'><o:p> </o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'>In any case I=E2=80=99m thinking that =
submitters are going to be wanting to submit curlies more and more =
often. =C2=A0The straights are beginning to look increasingly =
anachronistic. =C2=A0<o:p></o:p></span></p><p class=3DMsoNormal><span =
lang=3DEN-US style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'><o:p> </o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'>Best. Jim A.<o:p></o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US =
style=3D'font-size:12.0pt;font-family:"Times New =
Roman",serif;color:#1F497D'><o:p> </o:p></span></p></div></body></ht=
ml>
------=_NextPart_000_003C_01D051E7.98D2C730--
--===============9171387709408488791==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZ3V0dm9sLWQg
bWFpbGluZyBsaXN0Cmd1dHZvbC1kQGxpc3RzLnBnbGFmLm9yZwpodHRwOi8vbGlzdHMucGdsYWYu
b3JnL21haWxtYW4vbGlzdGluZm8vZ3V0dm9sLWQK
--===============9171387709408488791==--