Re: Invalid UTF-8

Oren Ben-Kiki wrote:
> I think I did not understand your proposal.
> 
> Here is what I _thought_ you suggested:
> 
> 1 - When a YAML library that _writes_ a file,
> _if_ given a UTF-8 string with invalid bytes,
> it will escape them as "\XNN".
> 
> 2 - When a YAML library _reads_ a file,
> _if_ it returns UTF-8 to its caller,
> it will convert "\XNN" back to the original invalid UTF-8 bytes.
> 
> 3 - When a YAML library that _reads_ a file,
> _if_ it returns UTF-16/UTF-32 to its caller,
> it will treat "\XNN" to an error (throw exception or whatever).
> 
> This means that the same file will be read by one YAML library and be
> rejected with an error by another. This is unacceptable.
> 
> Your post seemed to indicate you are proposing something else, but I am
> not certain what it is. Can you explain exactly how you see "\XNN"
> working in the above three cases?
> 
> Oren.

The above description matches exactly what I am proposing, except \XNN 
only throws an error if arranged such that the result is not valid UTF-8.

The strings I want CANNOT be placed into UTF-16. If they could be put 
into UTF-16 then that resulting UTF-16 string does not represent itself. 
Therefore an error MUST be thrown.

A better way to describe my proposal is to say that UTF-8 to UTF-16 
decode() may throw an error. If libyaml has to call that then of course 
it can throw that error, but there is no reason to throw the error if it 
does not call it!


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july