PHP and UTF-8

Hi

I did some more research about UTF-8 and it seems that is all a huge 
nightmare. The main point seems to be that PHP always uses 8bit 
characters and not wide characters for string representation. This 
effetctively means that even mbstrings just is a big hack. Damn them PHP 
developers ;)

I hacked up a small utility in tools/i18n/i18n-testfunctions.php that 
checkes all PHP files under the current directory for functions that 
might get confused on UTF-8 characters. Try to call it in 
r3/binarycloud. I guess we will have a lot of problems :(

So basically there are several options I see (feel free to add your own 
ones)

  * Run, don't walk, don't look back - run to something supporing UTF-8 
like Python or even Perl or Java. Damn it, even VB stores everything as 
Unicode internally as does everything on Windows. Is the Unix world 
*that* ignorant?
  * Try to rewrite all these methods (or at least the ones frequently 
used) in PHP and use multi byte safe methods only as well as the iconv 
functions. This will make everything slower but after all we spend most 
of the time in rendering - look at the debug output for simple benchmark.
  * Don't do anything and pray for PHP not breaking. Recommend mbstrings.
  * Double check everything and replace the functions in the places only 
that will break binarycloud when using UTF-8 in them with multi byte 
characters. For example, using functions on file names won't break 
anything since we can simply force people to use 7 bit ASCII here. Same 
with config ids etc. The places we have to double check include:
   - Smarty plugins, as they will surely break IMO
   - Validators and Filters
   - The env package
   - Creole (maybe)

I'd opt for the last point.

Regards,

Manuel

-- 
GPG key: http://www.ggnore.net/~mholtgrewe/personal/files/public_key.asc
_______________________________________________
dev mailing list
dev-PnctHDZWAvB/Cz2I37pSEPZ4XP/[email protected]
http://lists.binarycloud.com/mailman/listinfo/dev