Re: New invalid UTF-8 proposal using tags and %nn

On Fri, 2009-09-04 at 10:51 +0200, Gabriel de Perthuis wrote:
> A that encodes arbitrary, data in yaml files by escaping the bytes
> that would break yaml structure (utf-8 and separators) with
> percent-encoding should be called something like urlencode or
> percent-encode.
>
> While it is most convenient (by being readable in the serialized form)
> when the data is mostly utf-8, I don't think there should be a
> requirement that language represent this as character strings. These
> are arbitrary byte strings, and it should be the library user's choice
> and responsibility to customize its loader to transform the byte
> string into an invalid character string. The only way this tag needs
> to relate with utf8, is to know what bytes are invalid utf8 and
> invalid yaml.

You can not avoid the need to relate with utf8, since the string may
contain non-ASCII Unicode characters, either directly or as \xnn and
\unnnn and \Unnnnnnnn escape sequences; so in order to populate the byte
array, you must specify an encoding - specifically, utf-8. Once you
accept that this tag _does_ relate with utf-8, calling it !!utf-u or !!
utf-url or something like that starts to make much more sense.

As for what is returned by the library, I agree that the only "safe"
choice is a byte array. However, the library _may_ choose to return
something else (such as a word array or even an "almost valid string")
depending on the platform, as long as it properly preserves the data.

Have fun,

    Oren Ben-Kiki


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july