V3 Speed-bump: UTF-8

Folks,

The last remaining issue on the V3 format is UTF-8 encoding. I really want
to do this, but the implementation route that I can see is going to slow me
down a bit. Here's a summary - if anyone has any suggestions or insights,
I'd be very grateful:

Currently text is stored and manipulated as 8-bit data, encoded by the
locale.
UTF-8 is an encoding of Unicode text representation. Unicode2UTF8() and
UTF8toUnicode() are trivial (by design!).

Ideally, I'd like to have locale2UTF8() and UTF82locale() functions at the
interface to the file write/read routines. Any ideas on how to get/write
these?

Alternately, I can build the entire project with UNICODE defined, which
makes all the internal text representations in Unicode. Thus, Unicode-UTF-8
routines can be used directly. This requires non-trivial changes to the
code, though, which will take a while.

Am I missing anything? Suggestions? Ideas?

	Thanks,

		Rony




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642