Re: utf8 horror stories

On Sep 16, 2004, at 11:52 AM, John Williams wrote:

> I hit the string concatenation thing too.  If one string has the utf8 
> flag
> on and the other has the flag off, the "off" string is "decoded" to 
> utf8.
> If it actually had utf8 data in it, you get doubly-utf8-encoded data,
> where each byte>=128 is converted to 2 utf8 bytes.

Oh, is that what that means? Ick.

> URI::Escape also doesn't work with utf8 strings unless you have URI
> version 1.31 (released in June) and change your code to call
> uri_escape_utf8 instead of uri_escape.

I don't use it. Phew!

> Perl desperately needs a "default-to-utf8" mode.  My site has data 
> coming
> from so many different sources, that I found it easier to turn all the
> utf8 flags off and treat everything as binary.  Luckily I'm not doing
> character counting operations very much.

Sooner or later you probably will. In my next major project, I'll use 
Autrijus' module to turn on utf8 everywhere and be done with it.

Regards,

David



-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM.
Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php