Re: New invalid UTF-8 proposal using tags and %nn

On 9/3/09 1:03 PM, "William Spitzak" <[email protected]> wrote:

> Okay it is obvious there is a failure to understand what I need, but
> that conversely I have been proposing solutions that are incompatible
> with existing yaml.
> 
> Here is a new proposal that I believe is entirely compatible, and can be
> implemented by a libyaml-using program (though I would still use my own
> library to avoid the overhead of multiple passes through the string):
> 
> UTF-8 TAG PROPOSAL:
> 
> The tag "!!utf8" means the string must be interpreted as UTF-8 but that
> invalid bytes are preserved. Ideally the writer only produces this tag
> if the string really contains invalid encoding:

I do not know if this is indicative of any kind of standard BUT... Perl has
an encoding called UTF-X which is a superset of UTF-8 plus all the invalid
code points. That is, in Perl you can say "\x{FFFF}". You'll get a UTF-8
marked string but if you feed it to something that actually checks for
proper Unicode, it will be flagged as an error.

Perhaps you are wishing for Perl's UTF-X. Perhaps UTF-X has an an existence
outside of Perl. In any case, it is not Unicode - it is larger than Unicode.

I mention this only to point out a possible implementation to steal ideas
from.

Josh


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july