RE: Unicode string handling

> Not quite, I'm working on a Motorola phone that supports 
> UTF_16LE natively and expects a u'\x00' ('\x00\x00') as the 
> terminator.
> 
> >it was also present before I made any changes.
> 
> Also true.  But your current implementation leaves no way 
> around the encoding/decoding process (hence my previous 
> suggestion of leaving 'encoding' as None).

Oh bugger, it sounds like this has to be fixed, I was quietly hoping that
no-one would need more than single-byte byte strings :(. I'll spend some
time fixing up the code, I'll try to get something done in the next day.

> >I think the safest way to do this would be to deprecate the STRING 
> >class and leave it untouched and introduce a new one for unicode 
> >support.
> 
> I'm beginning to think that it's not such a bad idea after all!

I'll do this then. I'll call it USTRING.

> >The terminator applies to the string on the phone which is 
> in the phone 
> >encoding.  I don't see where the problem is.
> 
> The problem is when the phone (a Motorola in this case) 
> supports unicode string natively (UTF_16LE) with a terminator 
> of '\x00\x00', and anything after that is garbage which most 
> certainly will fail to decode.

The function that extracts the string from the packet (readfrombuffer) stops
when it hits the terminator even if there is more data available.
The current code is hard coded for one-byte terminators, but once fixed it
should work. I think the terminator would have to be assessed on a byte
boundary which is a multiple of its length to prevent \x8000\x0080 from
triggering terminator detection. 
The terminator is currently treated as an integer not a character, so I
think an extra parameter "terminator_length" is going to be required to make
0 into a 16-bit 0.

Simon



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642