Re: YAML Multi-Layer Spec

Most parsers in the real world use the term lexer. PyYaml is the first
place I saw the term scanner used.

On Sat, Mar 5, 2016 at 2:22 AM, Andrey Somov <[email protected]> wrote:

> Minor proposal:
> let us use 'scan' instead of 'lex'
>
> 1. I could not find the word 'lex', but could find 'scan'
> 2. scan sounds better
> 3. scanner is already used in PyYAML and SnakeYAML
>
> Cheers,
> Andrey
>
>
>
> On Fri, Mar 4, 2016 at 9:40 PM, Ingy dot Net <[email protected]> wrote:
>
>> Since this thread seems to be winding down, I've changed the subject.
>> About a year ago I was driving home from a conference where I gave a talk
>> on YAML. I had a sort of epiphany where I realized the source of a lot of
>> the contentions we had (and still have) around things like duplicate keys,
>> key order, equality and such.
>>
>> YAML processing is stack based: See the diagram in
>> https://github.com/ingydotnet/yaml-dev-kit/blob/guide/yaml-loader-guide.pod#readme
>>
>> A Load operation is:
>> (source)->read->(chars)->lex->(tokens)->parse->(events)->compose->(graph)->construct->(native)
>>
>> The YAML Spec talks about all of these things, but primarily it defines
>> precisely a token grammar.
>>
>> My big realization was that the spec needs to "spec" each layer and
>> transform. They are each distinct, and need to be defined and reasoned
>> about distinctly.
>>
>> For instance, it is not good enough to say "On input, a YAML processor
>> must support the UTF-8 and UTF-16 character encodings. For JSON
>> compatibility, the UTF-32 encodings must also be supported."
>>
>> You would want to state this in the "Reader" section.
>>
>> Statements about Key Order and Duplicate Keys can't be made about "YAML"
>> but about YAML processing at the graph layer. At the parser level there can
>> be no concept of key order or duplication. In a streaming processor, it is
>> almost certain that key order will be preserved.
>>
>> ...
>>
>> I don't think we need to be alarmed in any way. I think the spec is fine
>> for what it is. I am hoping to capture all of this in a more accessible
>> form in the dev kit <https://github.com/ingydotnet/yaml-dev-kit>. Also
>> with tests that force the issues. I think that will be much easier to
>> reason about than the spec.
>>
>> On Fri, Mar 4, 2016 at 9:04 AM, Oren Ben-Kiki <[email protected]> wrote:
>>
>>>
>>>
>>> On Fri, Mar 4, 2016 at 7:01 PM, Osamu TAKEUCHI <[email protected]> wrote:
>>>
>>>> Let me confirm one point. Does this allow identity-based
>>>> comparison of mapping nodes with some specific tags?
>>>
>>>
>>> Not sure what you mean. In YAML each mapping node has its own identity
>>> which is different from all other nodes. The only way for two mapping nodes
>>> to have the same identity is via an alias (anchor and reference).
>>>
>>>
>>>> > But that issue doesn't seem to be the one raised here;
>>>> >the problem seems to be with the "early duplicates detection".
>>>> >Which, again, is explicitly not required by the spec.
>>>> >So I'm not certain what the actual problem is with the current
>>>> >rules. Is this a case of the perceived rules being different
>>>> >from the actual rules in the spec?
>>>>
>>>> If you don't see any problem to relax the equality detection
>>>> as above, neither do I.
>>>>
>>>
>>> There's no need to "relax" it, it is "relaxed" already.
>>>
>>> Oren.
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Yaml-core mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/yaml-core
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Yaml-core mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/yaml-core
>>
>>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Yaml-core mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/yaml-core
>
>

------------------------------------------------------------------------------

_______________________________________________
Yaml-core mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/yaml-core