Re: Blackberry/Funambol: Character encoding woes
Jan Schneider <[email protected]>
| Newsgroups | gmane.comp.horde.sync |
|---|---|
| Message-ID | <[email protected]> |
Zitat von Ian Turner <[email protected]>: > Hello list, > > Just set up SyncML support with Funambol to Blackberry. Works very well! > Thanks for your efforts in creating such a wonderful software project. I am > encountering two problems with the SyncML support in Horde; this e-mail > describes the second such problem. > > It does not appear that character encodings are respected during the > sync. I'm > not sure if this is a blackberry problem, funambol problem, Apache problem, > PHP problem, or Horde problem, but I can't find any FAQs about it, > so I thought > you, dear list, might know. > > Example: > New memo on blackberry containing non-ASCII string "Ûñíçòðë Té§t™". > This string encoded as UTF-8: > 0000000: c39b c3b1 c3ad c3a7 c3b2 c3b0 c3ab 2054 .............. T > 0000010: c3a9 c2a7 74e2 84a2 ....t... > This string encoded as ISO-8859-1 (note that '™' is not available in > iso-8859-1) > 0000000: dbf1 ede7 f2f0 eb20 54e9 a774 ....... T..t > This string encoded as XML/HTML character entities: > Ûñíçòðë Té§t™ > > When I create a memo containing this string on blackberry, Horde appears to > attempt to commit this string to the database, a mix of HTML character > entities and iso-8859-1. As this string is not valid UTF-8 (which is the > database locale), it is rejected by the (postgres) database with the error > "invalid byte sequence for encoding 'UTF8': 0xdbf1" > 0000000: dbf1 ede7 f2f0 eb20 54e9 a774 2623 3834 ....... T..tT > 0000010: 3832 3b0a 82;. > > There appear to be two problems here: One, that Horde is trying to write > iso-8859-1 encoded text to a UTF-8 database, and Two that HTML character > entities have somehow made it this far without being decoded (perhaps because > '™' is not available in iso-8859-1). It does appear that the original text > made it into Horde in the correct encoding, since /tmp/sync/data.txt contains > the correct HTML character entities. Based on this file, it seems > like perhaps > the problem is in the conversion from text/x-s4j-sifn to text/x-vnote. > > Contents of /tmp/sync/data.txt: > > Input received from client (text/x-s4j-sifn): > <note> > <Subject>Test Unicode</Subject> > <Body>Ûñíçòðë Té§t™</Body> This is broken data, SIF-N is supposed to be unencoded UTF-8 data. > <Categories></Categories> > <Color></Color> > <Height></Height> > <Width></Width> > <Left></Left> > <Top></Top> > </note> > > > Input converted for server (text/x-vnote): > BEGIN:VNOTE > VERSION:1.1 > BODY;ENCODING=QUOTED-PRINTABLE;CHARSET=UTF-8:=DB=F1=ED=E7=F2=F0=EB > T=E9=A7t™\; > SUMMARY:Test Unicode > CATEGORIES: > END:VNOTE > > When synchronizing in the opposite direction, i.e. creating a non-ASCII note > in Horde and then synchronizing to blackberry, things work as expected. > > I have attached the full contents of /tmp/sync. Any thoughts you can share on > this mystery would be much appreciated. > > Versions: > Horde 3.3.8 > Notes (mnemo) H3 (2.2.3) > PHP 5.3.3 > > Sincerely, > > --Ian Turner > ` -- The Horde Project http://www.horde.org/ -- sync mailing list - Join the hunt: http://horde.org/bounties/#sync Frequently Asked Questions: http://horde.org/faq/ To unsubscribe, mail: [email protected]