Re: MODTOOLS-17 To osis2mod, added conversion of hex and decimal numeric entities to UTF-8, with special handling of <, >, &, ', and ".
Nathan Phillip Brink <[email protected]> Thu, 7 Aug 2025 01:27:28 -0400
| Newsgroups | gmane.comp.literature.sword.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============7758584396427267889== Content-Type: multipart/alternative; boundary="------------0vK0uDdixjuFFQgSDaX5nWfX" Content-Language: en-US This is a multi-part message in MIME format. --------------0vK0uDdixjuFFQgSDaX5nWfX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hello David, Prior to DM’s fix, “&x20;” would be interpreted “ ” even though it is invalid syntax. Also, “ ” would be treated as an error incorrectly. After DM’s fix, “&x20;” is properly treated as an error and treated as if the ampersand were meant to be a literal ampersand in the text and “ ” is properly treated as “ ”. You compare this behavior to that of a web browser which uses similar technology and behaves the same by checking this jsfiddle: https://jsfiddle.net/binki/mwnLv49f/ . Sorry, I am just reading the Jira issue and the code—I don’t have a build environment set up so I can’t actually test it. But it looks to me like the changes DM made do indeed fix something here. Thanks. On 8/7/2025 12:24 AM, David Haslam wrote: > Hi DM, > > I’m puzzled. > > You seems to have thought there was a bug which actually wasn’t. > > Please refer to > https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references > <https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references?wprov=sfti1#> > > > The # was not a bug ! > > Regards > > David > > Sent from Proton Mail <https://proton.me/mail/home> for iOS > > > On Wed, Aug 6, 2025 at 22:20, DM Smith < [email protected] > <mailto:On Wed, Aug 6, 2025 at 22:20, DM Smith <<a href=>> wrote: >> I’ve just checked in a change for osis2mod. >> >> MODTOOLS-17 To osis2mod, added conversion of hex and decimal numeric >> entities to UTF-8, with special handling of <, >, &, ', and ". >> >> Also: >> * Fixed a bug in hex numeric entities which defined &xHHHH; rather >> than &#xHHHH;. >> * Added entity sanity check of maximum length of 32. >> * Refactored entity handling into handleEntities and comment handling >> into handleComments. >> * Changed t_entitytype and t_commentstate into class enums EntityType >> and CommentState. >> * Added -d 1024 for entity and comment parsing. >> >> Note: The coding allows for 0 padding of the numeric entities. >> Note: The 5 need to be treated specially. >> & or & → & >> < or < → < >> > or > → > >> " or " → " or " >> ' or ' → ' or ' >> When converted to these forms, " should be transformed into " >> except in attributes using " and likewise ' into ' except in >> attributes having ‘. >> >> I need to update the wiki to match. >> >> In Him, >> DM Smith >> >> _______________________________________________ >> sword-devel mailing list: [email protected] >> http://crosswire.org/mailman/listinfo/sword-devel >> Instructions to unsubscribe/change your settings at above page > > _______________________________________________ > sword-devel mailing list:[email protected] > http://crosswire.org/mailman/listinfo/sword-devel > Instructions to unsubscribe/change your settings at above page --------------0vK0uDdixjuFFQgSDaX5nWfX Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <p>Hello David,</p> <p>Prior to DM’s fix, “&x20;” would be interpreted “ ” even though it is invalid syntax. Also, “&#x20;” would be treated as an error incorrectly.</p> <p>After DM’s fix, “&x20;” is properly treated as an error and treated as if the ampersand were meant to be a literal ampersand in the text and “&#x20;” is properly treated as “ ”. You compare this behavior to that of a web browser which uses similar technology and behaves the same by checking this jsfiddle: <a moz-do-not-send="true" href="https://jsfiddle.net/binki/mwnLv49f/" class="moz-txt-link-freetext">https://jsfiddle.net/binki/mwnLv49f/</a> .</p> <p>Sorry, I am just reading the Jira issue and the code—I don’t have a build environment set up so I can’t actually test it. But it looks to me like the changes DM made do indeed fix something here.</p> <p>Thanks.</p> <p><br> </p> <div class="moz-cite-prefix">On 8/7/2025 12:24 AM, David Haslam wrote:<br> </div> <blockquote type="cite" cite="mid:c4ig_DZsZMezuNDYPT-L0bfOwcZyOpmc-KGsw-AH20X9BTeC1CgfeGh3i_lfQDEAIaC9jwn_hjh7zLRJlpBGt2OwZF_khnay8Ot9Ft8KWAE=@protonmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="auto"> Hi DM, </div> <div dir="auto"> <br> </div> <div dir="auto"> I’m puzzled. </div> <div dir="auto"> <br> </div> <div dir="auto"> You seems to have thought there was a bug which actually wasn’t. </div> <div dir="auto"> <br> </div> <div dir="auto"> Please refer to <a dir="auto" href="https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references?wprov=sfti1#" moz-do-not-send="true">https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references</a> </div> <div dir="auto"> <br> </div> <div dir="auto"> The # was not a bug ! </div> <div dir="auto"> <br> </div> <div dir="auto"> Regards </div> <div dir="auto"> <br> </div> <div dir="auto"> David </div> <div> <br> </div> <div id="protonmail_mobile_signature_block"> <div> Sent from <a href="https://proton.me/mail/home" moz-do-not-send="true">Proton Mail</a> for iOS </div> </div> <div> <br> </div> <div> <br> </div> On Wed, Aug 6, 2025 at 22:20, DM Smith < <a href="mailto:On Wed, Aug 6, 2025 at 22:20, DM Smith <<a href=" class="" moz-do-not-send="true">[email protected]</a>> wrote: <blockquote class="protonmail_quote" type="cite"> I’ve just checked in a change for osis2mod. <br> <br> MODTOOLS-17 To osis2mod, added conversion of hex and decimal numeric entities to UTF-8, with special handling of <, >, &, ', and ". <br> <br> Also: <br> * Fixed a bug in hex numeric entities which defined &xHHHH; rather than &#xHHHH;. <br> * Added entity sanity check of maximum length of 32. <br> * Refactored entity handling into handleEntities and comment handling into handleComments. <br> * Changed t_entitytype and t_commentstate into class enums EntityType and CommentState. <br> * Added -d 1024 for entity and comment parsing. <br> <br> Note: The coding allows for 0 padding of the numeric entities. <br> Note: The 5 need to be treated specially. <br> &#38; or &#x26; → &amp; <br> &#60; or &#x3C; → &lt; <br> &#62; or &#x3E; → &gt; <br> &#34; or &#x22; → &quot; or " <br> &#39; or &#x27; → &apos; or ' <br> When converted to these forms, &quot; should be transformed into " except in attributes using " and likewise &apos; into ' except in attributes having ‘. <br> <br> I need to update the wiki to match. <br> <br> In Him, <br> DM Smith <br> <br> _______________________________________________ <br> sword-devel mailing list: <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> <br> <a class="moz-txt-link-freetext" href="http://crosswire.org/mailman/listinfo/sword-devel">http://crosswire.org/mailman/listinfo/sword-devel</a> <br> Instructions to unsubscribe/change your settings at above page <br> </blockquote> <br> <fieldset class="moz-mime-attachment-header"></fieldset> <pre wrap="" class="moz-quote-pre">_______________________________________________ sword-devel mailing list: <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> <a class="moz-txt-link-freetext" href="http://crosswire.org/mailman/listinfo/sword-devel">http://crosswire.org/mailman/listinfo/sword-devel</a> Instructions to unsubscribe/change your settings at above page </pre> </blockquote> </body> </html> --------------0vK0uDdixjuFFQgSDaX5nWfX-- --===============7758584396427267889== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ sword-devel mailing list: [email protected] http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page --===============7758584396427267889==--