Re: Next YAML: drop equality definition

Oren,

Thank you for your suggestive reply.

> You seem to assume that a YAML processor _must_ preserve
> the identity of scalars.That is, it _must not_, for example,
>use interned strings for keys.

Yes. But IMO, even with the current spec, a YAML processor
can not use a simple interned string to represent a
tag-unresolved scalar node. Note that a YAML processor
must internally represent the scalar node by an instance of
some class that at least contains the information that the
node appeared without an explicit tag. Otherwise, the
processor can not distinguish it from node with an explicit
!!str tags. It is necessary for the tag resolution at the
later stage. Once the processor somehow resolves the tag to
!!str, it can safely use interned string for the scalar
because it knows how to evaluate equality of !!str.


> It is crucial to the discussion. We all agree that key
>duplication detection _must_ be done at the application
>layer, but the point is that a _limited_ form of key
>duplication detection _may_ and _should_ be done,
>especially in YAML processors that do not even _have_
>an application layer. This is because, as you put it,
>"it makes the serialized documents much more readable
>or portable".

I will not say "must." Instead, I say "should be done as
far as it is possible." I know a processor without
construction stage can do very little in this sense when
my proposal is accepted. But I do not take it so seriously.
The anyway-imperfect key-duplication detection is just for
some educational purpose by nature. It is not _directly_
beneficial to users as opposed to my proposal.


>     ... without declaring
>     possible non-preservation of identity for
>     scalars, nobody will think a data with an
>     identity-based equality evaluation must be
>     stored as a collection node and must not as a
>     scalar node. It brings some surprise to users.
>
> Really? People would be very surprised to hear that
>{ a: 1, a: 2 } is actually OK because some application
>somewhere _may_ decide it wants scalar string keys to
>use "identity-based equality".

I missed to state the point clearly.
See the next example:

? !element '<a href="index.html">Top</a>'
: insert it before #the-box in red
? !element '<a href="index.html">Top</a>'
: insert it after #the-box in blue

I think the document above is readable enough unless
a schema-blind YAML processor tries to judge the data
equality and identity for unknown data type. At least
for me, it is as readable as the next.

? !element {from_html: '<a href="index.html">Top</a>'}
: insert it before #the-box in red
? !element {from_html: '<a href="index.html">Top</a>'}
: insert it after #the-box in blue

As is expected, the constructed object will not have
a from_html property. The fragment of html is given to
the builder method of !element to construct an DOM
element.

Then, will it be so bad to allow construction of a
class object from a scalar node?

It is not strange for me to allow the former because
we allow the latter.

There are many use cases similar to the above. If a
class has a builder function that accepts a string,
a number or whatever that can be stored in a YAML's
scalar node, it is natural to represent the class
object by a scalar value in a YAML document with
appropriate tag on it either explicitly or implicitly.


>     So, let's make the spec simpler by dropping the
>     definition of YAML's own equality and identity
>     preservation.
>
> There's no such thing as not addressing the issue of
>identity and equality in the spec. Either you _require_
>a YAML processor to preserve the identity of scalars
>(including, horribly, simple integers), or you do not.
>Either way it needs to be stated in the spec.

A schema-blind YAML processor can not judge if a
tag-unspecified scalar node indeed expresses a
simple integer until tag resolution is completed.
It is allowed to resolve

{ 1: 2, 1: 3 }

as

{ !not_integer 1 : 2, !not_integer 1 : 3 }

Although the current spec do not allow (!not_integer 1)
to be different from (!not_integer 1), I do not see
strong reason to forbid such an evaluation, with the
above example in my mind.


>     while two YAML nodes
>     of different values and even different tags can
>     be evaluated to be equal by some specific
>     evaluators. Note that javascript do not
>     natively distinguish an integer 0x01 with a
>     sequence [1] as mapping keys.
>
>
> You keep conflating false positives with false negatives.
>False negatives are _fine_. It is OK for the processor
>to miss some cases of key duplication. In fact it is
>expected. The application is the final arbiter of key
>equality. You can keep on piling as many examples of
>"the processor can't detect keys in <some example> as
>duplicated" as you want. OF COURSE there are such cases.

This part was just a generic reminder for readers that
the equality evaluation is strongly dependent on the
specific applications. So, it is not theoretically
needed but hopefully will help readers' understanding.


> Now, in JavaScript, PHP, Perl, Ruby, Python, C++, and
>any other valid YAML system, { a: 1, a: 2 } _does_ have
>a duplicate key. So a processor _is_ allowed and _should_
>complain about this, _regardless_ of the application-defined
>equality operator.

Again, as you explained in detail, a schema-blind YAML
system can not determine that the keys are of !!str,
even in this simple case. It might be resolved as

   { !not_string a : 1, !not_string a : 2}.

So, the equality is not defined by the programing languages
but only defined by the YAML spec.


>     A well-behaved YAML
>     processor _must_ be schema aware,
>
> Now this is just plain wrong. YamlReference is a
>YAML processor. It implements the parsing stage.
>It has no clue whatsoever what schema is used.
>  Schema-blind YAML processing is, for me, an
>important use case.

I should have written this as

   A well-behaved YAML processor that tries to detect key
   duplication _must_ be schema aware.

Then, YamlReference is ok because it do not detect key
duplication at all, although it should do so if it is a
well-behaved processor. ;)


BTW, ambiguity of data semantics such as the one between
{ a: 1 } and { !not_string a: 1 } is only for the
schema-blind processors. For the readers of the document
and for the real tools to manipulate it, the explicit or
implicit schema makes the ambiguity clear.

So, even if

     !element '<a href="index.html">Top</a>'

constructs a class object, it will not surprise a user
who is really interested in the document.

Best,
Osamu Takeuchi


------------------------------------------------------------------------------