Re: Rendering issues with Finnish Umlauts in FinPR
Tobias Klein <[email protected]>
| Newsgroups | gmane.comp.literature.sword.devel |
|---|---|
| Message-ID | <[email protected]> |
Thanks Troy! I'll have a look at the EncodingFilters. I think this is something not fully clear from the SWORD documentation/examples. Maybe these transformation points you had mentioned in the thread below should be described somewhere in the developer wiki? Best regards, Tobias On 4/3/23 6:45 PM, Troy A. Griffitts wrote: > Dear Tobias, > > Please be sure to note my comment to you below in this thread. It is > likely the cause of your rendering issues, while other apps have no > problems. > > In brief, it says that I haven't seen anywhere that you tell SWORD > what markup and encoding you want from the engine. If this is the case > you will get whatever the modules are encoded / marked up as, which > might be various things. > > Hope this helps, > > Troy > > On January 22, 2023 12:03:22 PM MST, "Troy A. Griffitts" > <[email protected]> wrote: > > Hey guys, > > Sorry for not jumping in on this thread more quickly. > > Please remember, SWORD has 4 transformation points, each moving > from the module source (as described in the .conf file) to the > client's request: > > RenderFilters - markup, e.g., GBF, ThML, OSIS -> XHTML > > StripFilters - prep before searching > > OptionFilters - turning on an off markup in the text stream based > on user options, e.g., Strongs Number, Words of Christ in Red, etc. > > EncodingFilters - e.g., 8859 - > UTF-8 > > > Module team: be sure the module has the correct Encoding value in > the .conf file (or the default) > > Tobias, be sure you are creating your SWMgr with the correct > MarkupFilterMgr to do the transformation you desire, e.g., see: > > https://crosswire.org/svn/sword/trunk/examples/cmdline/outrender.cpp > > Hope this helps, > > Troy > > > On 1/22/23 10:39, Fr Cyrille wrote: >> HI David, >> If you send me the file, I can convert it quickly in osis. I >> script it from imp to usfm and the with u2o.py. >> >> Le 22/01/2023 à 16:54, David Haslam a écrit : >>> Thanks Tobias, >>> >>> The problem is that CrossWire no longer accepts module >>> submissions that use IMP format for the build process. >>> >>> We’d need to have a script (or equivalent TextPipe filter) to >>> convert IMP to OSIS (whether directly or indirectly through some >>> other intermediate file format). >>> >>> I’m not currently in a practical position to work on that kind >>> of task. >>> Is anyone else up to it? >>> >>> Best regards, >>> >>> David >>> >>> Sent from Proton Mail for iOS >>> >>> >>> On Sun, Jan 22, 2023 at 15:39, Tobias Klein >>> <[email protected]> wrote: >>>> >>>> The FinPR module that David sent me works fine without >>>> rendering issues! (see screenshot below) >>>> >>>> It would be good to upgrade the module in the repo accordingly. >>>> >>>> Best regards, >>>> Tobias >>>> >>>> On 1/22/23 8:31 AM, David Haslam wrote: >>>>> Thanks Kristóf. >>>>> >>>>> The rendering problem could have been fixed a decade ago!!! >>>>> >>>>> Checking through my email archives yesterday, I discovered >>>>> that I had rebuilt the FinPR module exactly 10 years ago! That >>>>> rebuild used mod2imp and imp2vs and included a fix to the text >>>>> encoding implemented on the IMP textfile). The message was >>>>> sent to the modules address on 2013-01-21 but presumably never >>>>> progressed by Chris Little who was then still supposed to be >>>>> responsible for module releases and updates. He went >>>>> permanently AWOL from CrossWire around that time. >>>>> >>>>> Back then we had not narrowed the policy for submitted source >>>>> text to be OSIS XML only. >>>>> >>>>> I wrote privately to Tobias last night, forwarding the email >>>>> of 10 years ago complete with both attachments. He will >>>>> examine those today. >>>>> >>>>> Aside: I also replaced <…> by {…} where these had wrapped the >>>>> ch:vs references that recorded av11n in the original upstream >>>>> source. In 2012, there had been no suitable av11n available in >>>>> SWORD but which we do have more recently. >>>>> >>>>> mod2osis should not be used, as has already been noted. >>>>> A round trip with mod2osis and osis2mod is not lossless, >>>>> unlike one with mod2imp and imp2vs. >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> David >>>>> >>>>> Sent from Proton Mail for iOS >>>>> >>>>> >>>>> On Sat, Jan 21, 2023 at 23:15, Kristof Szabo >>>>> <kristof.szabo-S0/[email protected]> wrote: >>>>>> I managed to get Ezra running (it was some libicu70 mess), >>>>>> and yes, the accented characters in this module are broken >>>>>> (as other modules accented characters are OK; I assume it is >>>>>> not a font issue). I tried the conf file change, but it >>>>>> didn't work either. >>>>>> >>>>>> The mitigation was to rebuild the module, mod2osis leaves >>>>>> some garbage in the OSIS, but that would be easy to clean, >>>>>> anyway osis2mod is possible with this garbage left in and >>>>>> tada we have a proper accents. >>>>>> >>>>>> image.png >>>>>> >>>>>> As the module was updated last only 3,5 yrs ago I assume the >>>>>> maintainer is still active, ie. they can be reached. >>>>>> >>>>>> Or I can have a look too, the challenge is, that such a >>>>>> module rebuild can open pandora's box, if I run some tests >>>>>> (https://github.com/krisek/sword-test) or David checks them, >>>>>> then for sure there will be some issues. I'm happy to fix >>>>>> some of them, but I definitely do not speak Finnish, so I'm >>>>>> not sure this would be a responsible action. If Dom gives me >>>>>> the go I can fix syntax & submit, but I don't want to end up >>>>>> in the rabbit hole :) Best would be to reach out to the >>>>>> original maintainer. >>>>>> >>>>>> Kind regards, >>>>>> k- >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Jan 21, 2023 at 8:26 PM Greg Hellings >>>>>> <[email protected]> wrote: >>>>>> >>>>>> Is Ezra properly setting encoding on the content it >>>>>> renders? Is it maybe setting a font that doesn't have the >>>>>> proper code points? >>>>>> >>>>>> --Greg >>>>>> >>>>>> On Sat, Jan 21, 2023, 13:12 Tobias Klein >>>>>> <[email protected]> wrote: >>>>>> >>>>>> Hi Kristof, David, >>>>>> >>>>>> Adding Encoding=UTF-8 to the module conf file >>>>>> ~/.sword/mods.d/finpr.conf does not solve my issue. >>>>>> >>>>>> The text still looks the same as before ... >>>>>> >>>>>> What else could I do to further debug this? >>>>>> >>>>>> Best regards, >>>>>> Tobias >>>>>> >>>>>> On 1/21/23 5:18 PM, Kristof Szabo wrote: >>>>>>> Hi Thomas, >>>>>>> >>>>>>> I suppose the problem is that finpr.conf contains no >>>>>>> encoding information (check the Hun* modules for >>>>>>> reference), and if there is nothing specified >>>>>>> Latin-1 is the default. mod2osis (shouldn't be used >>>>>>> !! :)) shows that the module is in UTF-8, so there >>>>>>> is a misalignment. >>>>>>> >>>>>>> https://wiki.crosswire.org/DevTools:conf_Files#:~:text=Plaintext-,Encoding,-UTF%2D8%0AUTF >>>>>>> >>>>>>> Kind regards, >>>>>>> Kristof >>>>>>> >>>>>>> On Sat, Jan 21, 2023 at 4:49 PM David Haslam >>>>>>> <dfhdfh-g/[email protected]> wrote: >>>>>>> >>>>>>> Hi Thomas, >>>>>>> >>>>>>> What about other Finnish modules? >>>>>>> eg. FinPR92, FinRK, FinSTLK2017 >>>>>>> >>>>>>> Presumably you already tested (eg) German >>>>>>> modules and found that umlauts and eszett are >>>>>>> both rendered aright? >>>>>>> >>>>>>> Btw. FinPR renders aright in PocketSword >>>>>>> (iOS/iPadOS). >>>>>>> >>>>>>> David >>>>>>> >>>>>>> Sent from Proton Mail for iOS >>>>>>> >>>>>>> >>>>>>> On Sat, Jan 21, 2023 at 15:25, Tobias Klein >>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> When retrieving the text of the FinPR module I >>>>>>>> am getting some rendering issues with the >>>>>>>> Finnish Umlauts. This is based on a user's >>>>>>>> problem report. >>>>>>>> >>>>>>>> >>>>>>>> Romans 5:8 returns like this in >>>>>>>> node-sword-interface / Ezra: >>>>>>>> >>>>>>>> Mutta Jumala osoittaa rakkautensa meit� kohtaan >>>>>>>> siin�, ett� Kristus, kun me viel� olimme >>>>>>>> syntisi�, kuoli meid�n edest�mme. >>>>>>>> >>>>>>>> >>>>>>>> While it should like like this (rendered text >>>>>>>> copied from Xiphos): >>>>>>>> >>>>>>>> Mutta Jumala osoittaa rakkautensa meitä kohtaan >>>>>>>> siinä, että Kristus, kun me vielä olimme >>>>>>>> syntisiä, kuoli meidän edestämme. >>>>>>>> >>>>>>>> >>>>>>>> This occurs both on Linux and macOS (have not >>>>>>>> tested on Windows yet). >>>>>>>> >>>>>>>> Any pointers what could be the root cause? I >>>>>>>> generally have not observed rendering issues >>>>>>>> with other modules. >>>>>>>> >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Tobias >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> sword-devel mailing list: [email protected] >>>>>>> http://crosswire.org/mailman/listinfo/sword-devel >>>>>>> Instructions to unsubscribe/change your settings >>>>>>> at above page >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> sword-devel mailing list:[email protected] >>>>>>> http://crosswire.org/mailman/listinfo/sword-devel >>>>>>> Instructions to unsubscribe/change your settings at above page >>>>>> _______________________________________________ >>>>>> sword-devel mailing list: [email protected] >>>>>> http://crosswire.org/mailman/listinfo/sword-devel >>>>>> Instructions to unsubscribe/change your settings at >>>>>> above page >>>>>> >>>>>> _______________________________________________ >>>>>> sword-devel mailing list: [email protected] >>>>>> http://crosswire.org/mailman/listinfo/sword-devel >>>>>> Instructions to unsubscribe/change your settings at above >>>>>> page >>>>>> >>>>> >>>>> _______________________________________________ >>>>> sword-devel mailing list:[email protected] >>>>> http://crosswire.org/mailman/listinfo/sword-devel >>>>> Instructions to unsubscribe/change your settings at above page >>> >>> _______________________________________________ >>> sword-devel mailing list:[email protected] >>> http://crosswire.org/mailman/listinfo/sword-devel >>> Instructions to unsubscribe/change your settings at above page >> >> >> _______________________________________________ >> sword-devel mailing list:[email protected] >> http://crosswire.org/mailman/listinfo/sword-devel >> Instructions to unsubscribe/change your settings at above page > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > _______________________________________________ > sword-devel mailing list: [email protected] > http://crosswire.org/mailman/listinfo/sword-devel > Instructions to unsubscribe/change your settings at above page _______________________________________________ sword-devel mailing list: [email protected] http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page