Re: Next YAML: drop equality definition

Oren,

I expect we share the same thinking that the
definition of equality belongs to the domain
specific data type, not to the serialization
language. So, unless it makes the serialized
documents much more readable or portable, a
serialization language should not determine
its own equality or identity definition.
Incoherence of the data semantics between
the native data and serialized document will
make users surprised and reduce the
readability.

I believe nobody is willing to define node
equality in JSON or XML specification to
improve the readability or portability.
Nobody compliments YAML on being more
readable and portable than JSON and XML
by giving its own definition of node equality
and providing non-preservation of identity
for scalar nodes.


In the reality, the equality defined in the
YAML spec is almost always neglected and the
equality evaluation is done by the native
equality evaluators in the existing YAML
libraries and applications. I believe the
situation will not change in the future
because people do not see large benefit to
compare nodes under YAML's standard instead
of the data's own standard.

I agree duplicate key should be detected by
YAML processors because we do not want users
to use duplicate keys for overwriting the
values of predefined keys. The key order in
a YAML mapping should not have meaning.
This part of the specification is valuable to
have better readability and portability of a
YAML document. If YAML libraries detect such
misusage and raise errors or warnings, such
misusage can be effectively avoided.

But it can be done without defining equality
in YAML spec. YAML processor can use native
equality evaluator of the data at its
construction stage and it should do so.
If the layered structure of the YAML processor
do not allow it, the layered structure itself
should be revised. I don't see how the layered
structure is related to the current topic,
though.


Similarly, I do not want to forbid PHP users
to store a PHP's native key-order-aware hash
into a key-order-unaware YAML mapping unless
the specific application really place
importance on the key order. If the key order
actually matters, they should store it in a
!!omap as the YAML spec advises. But otherwise,
they are allowed to store it in a !!map. Then,
the document will express the meaning of the
data more correctly. The data semantics belongs
to the data itself, not to the programing
language nor to the serialization language.


Meaningfulness of the data identity should
also belong to the specific data types. As
shown by the previous examples, the difference
in the semantics of a scalar node and that of
a complex node is not always clear. I imagine
that it is much unclearer than what was
expected in the current YAML spec. If we allow
equality evaluation by identity for collection
nodes, it is more coherent to allow it also for
scalar nodes. Actually, without declaring
possible non-preservation of identity for
scalars, nobody will think a data with an
identity-based equality evaluation must be
stored as a collection node and must not as a
scalar node. It brings some surprise to users.

Such restriction will improve YAML's readability
and portability very little if any. Actually, I
believe the restriction is currently not known
widely and very few libraries and applications
have ever utilized it. I don't think many
existing YAML document loose its meaning if we
drop the restriction. I don't see any kind of
problems will be caused in *realistic* use cases
by dropping. I do not expect it will be widely
utilized for building libraries, storing data
and understanding documents in the future,
regardless of what is written in the spec.


So, let's make the spec simpler by dropping the
definition of YAML's own equality and identity
preservation.

What I want to say in the spec is:
A well-behaved processor _should_ detect a
duplicate key and flag it as an error if it
can correctly evaluate equality of nodes.
It _must_ aware that a data with some specific
tags may have some custom comparison algorithms,
including the one based on the data identity.
Namely, two YAML nodes of same values and same
tags can be evaluated to be unequal by an
identity-based evaluator, while two YAML nodes
of different values and even different tags can
be evaluated to be equal by some specific
evaluators. Note that javascript do not
natively distinguish an integer 0x01 with a
sequence [1] as mapping keys.

> var test = { 0x01: "Find me!" };
> if(test[[1]] === "Find me!")
>alert("Surprised?");

It is also warned that tags of nodes can be
implicitly specified by the path of the node
from the root. So, a schema-blind YAML processor
can never know how to resolve a tag for any
tag-unspecified node. A well-behaved YAML
processor _must_ be schema aware, which refers
to the schema to resolve the tag correctly
and to evaluate the equality correctly from the
resolved tag.

Best,
Osamu Takeuchi


On 2016/03/05 14:23, Oren Ben-Kiki wrote:
> There is no requirement that early detection
>catch 100% of the duplicatekeys. There is a
>requirement that early detection will never
> incorrectlyflag different keys as being duplicate.
>
> So: A YAML processor _need not_ preserve the
>identity of scalars.  Therefore it is allowed to
>(correctly) flagequal scalar keys. Becausean
>application _must not_ rely on scalar identity
>to compare scalar keys.
>
> The fact that "1+2i" might be equal to
>{ r: 1, i: 2 } just means thatearly detection
>isn't perfect. Which is OK since the final definitive
>equality check is done by the application anyway.
>
> What we really want is to _allow_ YAML processors
>to flag { a: 1, a : 2 }as a duplicate key, because
> this is the most common type of error. Therules
>_allow_ early flagging of these errors, without
>waiting for theapplication, and a well-behaved
>processor _should_ flag this as an error,as early
>as the entry to the compose stage.
>
> Oren.
>
>
> On Fri, Mar 4, 2016 at 9:51 PM, Osamu TAKEUCHI <[email protected] <mailto:[email protected]>> wrote:
>
>     Oren,
>
>     Thank you for your understanding of my poor description.
>
>         Note, this only applies to complex keys.
>         The spec clearly says identity is not preserved for scalars.
>
>
>     The exact statement is
>
>          > A YAML processor may treat equal scalars as if they were
>          >identical.
>
>
>     So, YAML processor may also preserve scalars' identity.
>     Is this true?
>
>     Then, this may apply to scalars, too.
>
>
>     Actually, As Kirill pointed out, a scalar node could be
>     equal to a collection node under a realistic schema as
>     below.
>
>     x: 1.0i
>     y: { re: 0.0, im: 1.0 }
>     z: { rho: 1.0, phi: 1.5707963267948966 }
>
>     Similarly, a class object that has its string representation
>     can also be stored as a scalar node, which may have its own
>     weird comparator with customized hash generator. It may be
>     an identity-based comparison. It may ignore some fields for
>     comparison. If YAML library ignores the native comparator
>     for such scalar nodes, users will be surprised.
>
>     This is from my experience implementing C# library.
>     Many of C# classes have so-called TypeConverter that converts
>     the native object to/from its string representation for
>     serializing purpose. In such cases, it is natural to store
>     such an object in a scalar node in a YAML document, with
>     an explicit/implicit tag to represent the data type.
>     So, a scalar node with an explicit/implicit tag may
>     no more be a simple scalar in the native object form.
>
>     It is like:
>
>     Greeting:
>        - "Hello world!"
>        - !System.Drawing.Point 100,100
>        - !System.Drawing.Font Times New Roman, 14pt
>
>     With such a use case in mind, distinguishing scalar nodes
>     from collection nodes in the preservation of identity is
>     not natural for me. I prefer to preserve scalars'
>     identity in my library and to apply native comparison
>     method to both of scalar node and collection node,
>     if the YAML spec allows.
>
>     I know it is very rare to have such objects as keys
>     of a mapping and the definition of equality causes
>     any problem. At the same time, I don't see any benefit
>     to define objects' equality in YAML spec in different
>     manner from the native one.
>
>     In my opinion, the definition of equality should belong
>     to the domain specific data type, not to the serialization
>     language.
>
>     Best,
>     Osamu Takeuchi
>
>


------------------------------------------------------------------------------