utf8 horror stories
John Williams <[email protected]>
| Newsgroups | gmane.comp.web.mason.devel |
|---|---|
| Message-ID | <[email protected]> |
On Thu, 16 Sep 2004, David Wheeler wrote: > > If anyone wants to exchange utf8 horror stories, I have a few... > > Here you go: > > http://www.justatheory.com/computers/programming/perl/utf8_trials.html I hit the string concatenation thing too. If one string has the utf8 flag on and the other has the flag off, the "off" string is "decoded" to utf8. If it actually had utf8 data in it, you get doubly-utf8-encoded data, where each byte>=128 is converted to 2 utf8 bytes. URI::Escape also doesn't work with utf8 strings unless you have URI version 1.31 (released in June) and change your code to call uri_escape_utf8 instead of uri_escape. Perl desperately needs a "default-to-utf8" mode. My site has data coming from so many different sources, that I found it easier to turn all the utf8 flags off and treat everything as binary. Luckily I'm not doing character counting operations very much. ~ John Williams ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php