Re: : in the flow context for PyYAML and SnakeYAML

Hi William,

If you take a look at the function yaml_parser_scan_plain_scalar, lines 
3424-3433, you'll see:

/* Check for 'x:x' in the flow context. TODO: Fix the test "spec-08-13". */

if (parser->flow_level
         && CHECK(parser->buffer, ':')
         && !IS_BLANKZ_AT(parser->buffer, 1)) {
     yaml_parser_set_scanner_error(parser, "while scanning a plain scalar",
             start_mark, "found unexpected ':'");
     goto error;
}

This condition is only checked in the flow context and it produces an 
error if there is a colon followed by a non-whitespace character. 
Therefore it should produce an error for {a:1} and I don't understand 
how you could get {"a": "1"}.

The YAML spec (1.1 and I suppose 1.2 too) interprets {a:1} as {"a:1": null}.

Thanks,
Kirill



On 08/24/2010 11:19 PM, William Spitzak wrote:
> This is the C version of libYAML, looked up in my raw version of the
> downloaded code from http://pyyaml.org/download/libyaml/yaml-0.1.3.tar.gz
>
> The source code file scanner.c contains several examples where it does
> not require a space after a ':' if inside a "flow" (meaning inside curly
> or square brackets). "parser->flow_level" is non-zero in this case.
>
> The function yaml_parser_fetch_next_token() has this code to see if we
> are looking at a colon (called a "value indicator"). I am unsure if this
> code is actually called when using the parser however:
>
> /* Is it the value indicator? */
>
> if (CHECK(parser->buffer, ':')
> && (parser->flow_level || IS_BLANKZ_AT(parser->buffer, 1)))
> return yaml_parser_fetch_value(parser);
>
> The function yaml_parser_scan_plain_scalar() uses the following code to
> decide if a character ends an unquoted scaler. Besides skipping a test
> for space after the colon, this also causes the characters ",?[]{}" to
> end a scaler. I know for a fact that this code is used:
>
> /* Check for indicators that may end a plain scalar. */
>
> if ((CHECK(parser->buffer, ':') && IS_BLANKZ_AT(parser->buffer, 1))
> || (parser->flow_level &&
> (CHECK(parser->buffer, ',') || CHECK(parser->buffer, ':')
> || CHECK(parser->buffer, '?') || CHECK(parser->buffer, '[')
> || CHECK(parser->buffer, ']') || CHECK(parser->buffer, '{')
> || CHECK(parser->buffer, '}'))))
> break;
>
>
> Kirill Simonov wrote:
>> Hi William,
>>
>> On 08/24/2010 08:32 PM, William Spitzak wrote:
>>> libyaml, which I thought was following the spec, quite specifically does
>>> not require a ':' in a "flow" context. There is code in there that
>>> literally says it is doing this, so somebody thought it was correct.
>>>
>>> Here is a conversion using the event api of libyaml to read and write
>>> some 1-line files (the output has "flow" and all quoting forced on to
>>> make it understandable):
>>>
>>> a:1 -> "a:1"
>>> a: 1 -> {"a": "1"}
>>> {a:1} -> {"a": "1"}
>>> {a: 1} -> {"a": "1"}
>>>
>>> So libyaml is doing what I think is being requested.
>>
>> Not sure how you get these. The first two examples and the last one
>> are non-controversial. However I'm getting a scanner error on {a:1}:
>>
>> >>> import yaml
>> >>> yaml.load("{a:1}", Loader=yaml.CLoader)
>> Traceback (most recent call last):
>> [...]
>> yaml.scanner.ScannerError: while scanning a plain scalar
>> in "<byte string>", line 1, column 2
>> found unexpected ':'
>> in "<byte string>", line 1, column 3
>>
>> Note that CLoader uses libyaml bindings internally.
>>
>> Perhaps you are using some patched version of libyaml?
>>
>>
>> Thanks,
>> Kirill


------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users 
worldwide. Take advantage of special opportunities to increase revenue and 
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d