Re: Unicode UTF-8 Compatible Version of Gutcheck
"Al Haines" <[email protected]> Thu, 26 Feb 2015 12:08:40 -0800
| Newsgroups | gmane.culture.literature.e-books.gutenberg.volunteers |
|---|---|
| Message-ID | <BDFF5F47DBF74D199EBA90BAA2D54750@ahainesPC> |
This is a multi-part message in MIME format. --===============1286921445369337217== Content-Type: multipart/alternative; boundary="----=_NextPart_000_000C_01D051BC.F595EF90" This is a multi-part message in MIME format. ------=_NextPart_000_000C_01D051BC.F595EF90 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Interesting--gutcheck_u reported several mismatched double quotes, but all were explainable, e.g. poem fragments inside a quoted paragraph, or two quoted paragraphs with an illustration between. On the other hand, there were many mismatched left/right single quotes--evidently DP's ASCII single quote to Unicode left/right single quotes needs some work. I'll do some more testing of gutcheck_u while WWing submissions. It would be very handy if someone could upgrade Gutspell, which doesn't work properly on texts containing Unicode quotes (single and double), em-dashes, etc. Al -----Original Message----- From: gutvol-d [mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]] On Behalf Of James Adcock Sent: Thursday, February 26, 2015 10:24 AM To: 'Project Gutenberg Volunteer Discussion' Subject: Re: [gutvol-d] Unicode UTF-8 Compatible Version of Gutcheck Off the top of my head, try 48325, which demonstrates the handedness issue. From: gutvol-d [mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]] On Behalf Of Al Haines Sent: Wednesday, February 25, 2015 9:37 PM To: 'Project Gutenberg Volunteer Discussion'; 'James Adcock' Subject: Re: [gutvol-d] Unicode UTF-8 Compatible Version of Gutcheck I'd like to give it a try. Can you upload it somewhere? Or you can send me a copy as a zipped attachment. And if you can mention the etext numbers of several of the files you tested it against, I can cross-check them with Gutcheck and Bookloupe (http://www.juiblex.co.uk/pgdp/bookloupe/index.html). Al -----Original Message----- From: gutvol-d [mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]] On Behalf Of James Adcock Sent: Wednesday, February 25, 2015 7:49 PM To: gutvol-d-4LCm/o+qPMZ96Xl/[email protected] Subject: [gutvol-d] Unicode UTF-8 Compatible Version of Gutcheck I've created a Unicode UTF-8 Compatible Version of Gutcheck, calling it gutcheck_u -- if anyone wants to try it. This was primarily an exercise in finding and changing 8-bit char coding dependencies to 16-bit widechar coding dependencies, but it did require some additional coding. Currently it is a somewhat Windows-dependent implementation, so if you want to run it on another OS it would take a little bit of work. I find it more pleasant to use if one's development file format of choice is UTF-8, and/or if one are doing such things as left-handed / right-handed quotes. Testing it on PG released files I am in fact finding a fair amount of left / right handedness errors that are not currently being discovered. Let me know if you want to try it, same distribution terms as the original. Jim Adcock [email protected] ------=_NextPart_000_000C_01D051BC.F595EF90 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML xmlns=3D"http://www.w3.org/TR/REC-html40" xmlns:v =3D=20 "urn:schemas-microsoft-com:vml" xmlns:o =3D=20 "urn:schemas-microsoft-com:office:office" xmlns:w =3D=20 "urn:schemas-microsoft-com:office:word" xmlns:m =3D=20 "http://schemas.microsoft.com/office/2004/12/omml"><HEAD> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = charset=3Dus-ascii"> <TITLE>Message</TITLE> <META name=3DGENERATOR content=3D"MSHTML 10.00.9200.17229"> <STYLE>@font-face { font-family: Cambria Math; } @font-face { font-family: Calibri; } @font-face { font-family: Tahoma; } @page WordSection1 {size: 8.5in 11.0in; margin: 1.0in 1.0in 1.0in 1.0in; = } P.MsoNormal { FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; MARGIN: 0in 0in 0pt } LI.MsoNormal { FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; MARGIN: 0in 0in 0pt } DIV.MsoNormal { FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; MARGIN: 0in 0in 0pt } A:link { COLOR: #0563c1; TEXT-DECORATION: underline; mso-style-priority: 99 } SPAN.MsoHyperlink { COLOR: #0563c1; TEXT-DECORATION: underline; mso-style-priority: 99 } A:visited { COLOR: #954f72; TEXT-DECORATION: underline; mso-style-priority: 99 } SPAN.MsoHyperlinkFollowed { COLOR: #954f72; TEXT-DECORATION: underline; mso-style-priority: 99 } SPAN.EmailStyle17 { FONT-FAMILY: "Calibri",sans-serif; COLOR: windowtext; mso-style-type: = personal } SPAN.EmailStyle18 { FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d; mso-style-type: = personal-reply } .MsoChpDefault { FONT-SIZE: 10pt; mso-style-type: export-only } DIV.WordSection1 { page: WordSection1 } </STYLE> <!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></HEAD> <BODY lang=3DEN-US link=3D#0563c1 vLink=3D#954f72> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN=20 class=3D804425719-26022015>Interesting--gutcheck_u reported several = mismatched=20 double quotes, but all were explainable, e.g. poem fragments inside a = quoted=20 paragraph, or two quoted paragraphs with an illustration between. =20 </SPAN></FONT></DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN=20 class=3D804425719-26022015></SPAN></FONT> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN = class=3D804425719-26022015>On the=20 other hand, there were many mismatched left/right single = quotes--evidently DP's=20 ASCII single quote to Unicode left/right single quotes needs some=20 work.</SPAN></FONT></DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN=20 class=3D804425719-26022015></SPAN></FONT><FONT color=3D#0000ff size=3D2=20 face=3DArial><SPAN class=3D804425719-26022015></SPAN></FONT><FONT = color=3D#0000ff=20 size=3D2 face=3DArial></FONT> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN = class=3D804425719-26022015>I'll=20 do some more testing of gutcheck_u while WWing = submissions.</SPAN></FONT></DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial></FONT> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial></FONT> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial><SPAN = class=3D804425719-26022015>It=20 would be very handy if someone could upgrade Gutspell, which doesn't = work=20 properly on texts containing Unicode quotes (single and double), = em-dashes,=20 etc.</SPAN></FONT></DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial></FONT> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial></FONT> </DIV> <DIV><SPAN class=3D804425719-26022015><FONT color=3D#0000ff size=3D2=20 face=3DArial>Al</FONT></SPAN></DIV> <DIV><SPAN class=3D804425719-26022015><FONT color=3D#0000ff size=3D2=20 face=3DArial></FONT></SPAN> </DIV> <DIV><FONT color=3D#0000ff size=3D2 face=3DArial></FONT> </DIV> <BLOCKQUOTE=20 style=3D"PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px = solid; MARGIN-RIGHT: 0px"=20 dir=3Dltr> <DIV></DIV> <DIV lang=3Den-us class=3DOutlookMessageHeader dir=3Dltr = align=3Dleft><FONT size=3D2=20 face=3DTahoma>-----Original Message-----<BR><B>From:</B> gutvol-d=20 [mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]] <B>On Behalf Of </B>James=20 Adcock<BR><B>Sent:</B> Thursday, February 26, 2015 10:24 = AM<BR><B>To:</B>=20 'Project Gutenberg Volunteer Discussion'<BR><B>Subject:</B> Re: = [gutvol-d]=20 Unicode UTF-8 Compatible Version of Gutcheck<BR><BR></FONT></DIV> <DIV class=3DWordSection1> <P class=3DMsoNormal><SPAN style=3D"COLOR: #1f497d">Off the top of my = head, try=20 48325, which demonstrates the handedness issue.<o:p></o:p></SPAN></P> <P class=3DMsoNormal><SPAN style=3D"COLOR: = #1f497d"><o:p> </o:p></SPAN></P> <DIV> <DIV=20 style=3D"BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; = BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; = PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in"> <P class=3DMsoNormal><B>From:</B> gutvol-d=20 [mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]] <B>On Behalf Of </B>Al=20 Haines<BR><B>Sent:</B> Wednesday, February 25, 2015 9:37 = PM<BR><B>To:</B>=20 'Project Gutenberg Volunteer Discussion'; 'James = Adcock'<BR><B>Subject:</B>=20 Re: [gutvol-d] Unicode UTF-8 Compatible Version of=20 Gutcheck<o:p></o:p></P></DIV></DIV> <P class=3DMsoNormal><o:p> </o:p></P> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 10pt; FONT-FAMILY: 'Arial',sans-serif; COLOR: = blue">I'd like=20 to give it a try. Can you upload it somewhere? Or you can = send me=20 a copy as a zipped attachment. </SPAN><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"><o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 10pt; FONT-FAMILY: 'Arial',sans-serif; COLOR: = blue">And if=20 you can mention the etext numbers of several of the files you tested = it=20 against, I can cross-check them with Gutcheck and Bookloupe (<A=20 = href=3D"http://www.juiblex.co.uk/pgdp/bookloupe/index.html">http://www.ju= iblex.co.uk/pgdp/bookloupe/index.html</A>).</SPAN><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"><o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 10pt; FONT-FAMILY: 'Arial',sans-serif; COLOR: = blue">Al=20 </SPAN><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"><o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <DIV> <P class=3DMsoNormal><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"> <o:p></o:p></SPAN></P></DIV> <BLOCKQUOTE=20 style=3D"BORDER-TOP: medium none; BORDER-RIGHT: medium none; = BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; = PADDING-LEFT: 4pt; MARGIN: 5pt 0in 5pt 3.75pt; BORDER-LEFT: blue 1.5pt = solid; PADDING-RIGHT: 0in"> <P class=3DMsoNormal style=3D"MARGIN-BOTTOM: 12pt"><SPAN=20 style=3D"FONT-SIZE: 10pt; FONT-FAMILY: = 'Tahoma',sans-serif">-----Original=20 Message-----<BR><B>From:</B> gutvol-d [<A=20 = href=3D"mailto:gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]">mailto:gutvol-d-bounces@= lists.pglaf.org</A>]=20 <B>On Behalf Of </B>James Adcock<BR><B>Sent:</B> Wednesday, February = 25,=20 2015 7:49 PM<BR><B>To:</B> <A=20 = href=3D"mailto:gutvol-d-4LCm/o+qPMZ96Xl/[email protected]">gutvol-d-4LCm/o+qPMZ96Xl/[email protected]</A><BR>= <B>Subject:</B>=20 [gutvol-d] Unicode UTF-8 Compatible Version of Gutcheck</SPAN><SPAN=20 style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Times New = Roman',serif"><o:p></o:p></SPAN></P> <P class=3DMsoNormal>I’ve created a Unicode UTF-8 Compatible = Version of=20 Gutcheck, calling it gutcheck_u -- if anyone wants to try = it.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>This was primarily an exercise in finding and = changing=20 8-bit char coding dependencies to 16-bit widechar coding = dependencies, but=20 it did require some additional coding.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>Currently it is a somewhat Windows-dependent=20 implementation, so if you want to run it on another OS it would take = a=20 little bit of work.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>I find it more pleasant to use if one’s = development file=20 format of choice is UTF-8, and/or if one are doing such things as=20 left-handed / right-handed quotes.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>Testing it on PG released files I am in fact = finding a=20 fair amount of left / right handedness errors that are not currently = being=20 discovered.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>Let me know if you want to try it, same = distribution=20 terms as the original.<o:p></o:p></P> <P class=3DMsoNormal><o:p> </o:p></P> <P class=3DMsoNormal>Jim Adcock<o:p></o:p></P> <P class=3DMsoNormal><A=20 = href=3D"mailto:[email protected]">[email protected]</A><o:p></o:p></P></BLOCKQUOT= E></DIV></BLOCKQUOTE></BODY></HTML> ------=_NextPart_000_000C_01D051BC.F595EF90-- --===============1286921445369337217== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZ3V0dm9sLWQg bWFpbGluZyBsaXN0Cmd1dHZvbC1kQGxpc3RzLnBnbGFmLm9yZwpodHRwOi8vbGlzdHMucGdsYWYu b3JnL21haWxtYW4vbGlzdGluZm8vZ3V0dm9sLWQK --===============1286921445369337217==--