Re: Unicode string handling
"Joe Pham" <[email protected]>
| Newsgroups | gmane.comp.mobile.bitpim.devel |
|---|---|
| Message-ID | <[email protected]> |
>You would truncate it until it fits the size available, the byte
>length is the limiting factor not the number of characters. I have
>implemented it to work this way at the moment if you look at the
>_update function.
Yes, I see that you back off 1-char at a time until the encoding result fits the length. However, the single-byte terminator (if present) still would mess it up. Consider:
100 STRING { 'encoding': 'utf_16le', 'terminator': ',' } test
then
*.test='abc' would produce 'a\x00b\x00c\x00,' which would certainly fail to decode.
>None of the phones use more than a single byte for terminating
>characters afaik,
I believe that is true.
>so the limitation is theoretical,
Not quite, I'm working on a Motorola phone that supports UTF_16LE natively and expects a u'\x00' ('\x00\x00') as the terminator.
>it was also present before I made any changes.
Also true. But your current implementation leaves no way around the encoding/decoding process (hence my previous suggestion of leaving 'encoding' as None).
>I think the safest way to do this would be to deprecate the STRING
>class and leave it untouched and introduce a new one for unicode
>support.
I'm beginning to think that it's not such a bad idea after all!
>The terminator applies to the string on the phone which is in the
>phone encoding. I don't see where the problem is.
The problem is when the phone (a Motorola in this case) supports unicode string natively (UTF_16LE) with a terminator of '\x00\x00', and anything after that is garbage which most certainly will fail to decode.
-Joe Pham
_____________________________________________________________________
Call Anyone, Anytime, Anywhere in the World - FREE!
Free Internet calling from NetZero Voice
Visit http://www.netzerovoice.com today!
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642