Re: Equality of YAML nodes

Hi Oren,

Thanks for your response.

I agree that the way you go is completely selfconsistent and 
theoretically very smart. But I feel it narrowers the application 
of YAML.


Let me go through your response.

> > ---
> > &A { &B { *B, *A }, *A, null }
> > 
> > ok: &A { }                             # 1
> > ok: &A { &B { } }                      # 2
> > ok: &A { &B { *B } }                   # 3
> > ok: &A { &B { *B, *A } }               # 4
> > NG: &A { &B { *B, *A }, *A }           # 5
> > ok: &A { &B { *B, *A }, *A, null }     # 6

> I didn't understand the problem. You have shown intermediate steps 
> that have duplicate keys; but the final result is OK, which is all 
> that matters. Can you clarify?

I wanted to discuss the way how a library build such a node.

I think, most of libraries fail to build the node because they at 
first create a map node object and then add the key / value pairs 
one by one to it. When the library try to add the pair (*A: nil) to
&A { &B { *B, *A } } as the fifth step, the map node object
will find duplicated keys and throw an exception (or neglect it).

Do you think there is a good way to avoid such a problem?


> > class Test
> > end
> >
> > # Create a hash with instances of Test
> > p hash = { Test.new => 1, Test.new => 2 }
> >
> > # {#<Test:0x7ff2d60c>=>1, #<Test:0x7ff2d5f8>=>2}
> >
> > # Note that two instances of Test class are not
> > # equal to each other, because ruby compares
> > # class instances, by default, by their identity.
>
> As opposed to YAML, yes, which will cause problems when dumping it.
> Consider what would happen if the data would be loaded by a language
> that compares the content by default... Anything based on Prolog and/or
> LISP for example.

My conclusion is different.

The Prolog and LISP code should compare the nodes by their identity
because they know, from the Tag, that the nodes represent instances 
of ruby's Test class.

The way how an object should be compared should belong to the 
object's type. It should not be changed across the environment or 
across the YAML processor's parsing/composing/construction stages.


> Well, technically, the author of the Test class has a bug. If he's
> comparing test objects based on their identity, he should serialize this
> identity somehow to make it explicit (!ruby/object:Test { !
> ruby/MagicDefaultIdentityUniqueHash Hash: 0x7ff2c9f0 } or whatever).
> Objects with the same hash should be serialized as anchors and
> references. You may even justify loading the "UniqueHash" tag into
> whatever the default unique identity hash would be for calling "new
> Test" instead of preserving it exactly.

I agree with you that this works for the Test class. But how about 
for Hash class. As I showed, ruby's Hash class has different equality 
evaluation from the YAML's. Do you say it is a bug to convert ruby's 
Hash to YAML's mapping?

Remember ruby did not treat  &A { *A }  equals to  &B { *B } .

>>  p YAML.load("{ &A { *A }, &B { *B } }") #4
>>  # {{{...}=>nil}=>nil, {{...}=>nil}=>nil}

Then, if we go strict, a library have to do the following.

!ruby/object:Hash
 UniqueHash: brabrabra
 Entries:
   ? !ruby/object:Hash &A
     UniqueHash: brabrabri
     Entries:
       ? *A
   ? !ruby/object:Hash &B
     UniqueHash: brabrabro
     Entries:
       ? *B

I do not think people accept this, while I can barely accept 
the following.

!ruby/object:Hash
 ? !ruby/object:Hash &A
   ? *A
 ? !ruby/object:Hash &B
   ? *B

Of course, this is not what I want, too. I like the next best,
though it might be unportable.

{ &A { *A }, &B { *B } }

That's the reason I proposed my not portable but practical 
definition of equality in YAML.


> So this mapping is *practically* unusable as it is, once it is
> de/serialized to YAML. Having the YAML library reject the file is
> better, as it forces the author to consider what he is trying to 
> achieve and do _that_.

Ok, my example was too much simplified.

USA:
 Presidents: !President[]
  - &PR1 
    name: George Washington
  - &PR2 
    name: John Adams
  (snip)
  - &PR41 
    name: George Bush
  - &PR42 
    name: William Clinton
  - &PR43 
    name: George Bush
  - &PR44 
    name: Barack Obama
 Parties: !Party[]
  - &PA1 
    name: Republican
  - &PA2 
    name: Democratic
  (snip)
 PresidentToParty: &MAP
  (snip)
  *PR41: *PA1
  *PR42: *PA2
  *PR43: *PA1
  *PR44: *PA2

Note that *PR? are the instances of President class and *PA? are
those of Party class. The tags are resolved from the parent's
President[] and Party[] tags.

In this example, we need some extra property like UniqueHash in
President class just to make the YAML document to be valid. Note 
that *PR41 and *PR43 are equal to each other by the YAML's 
definition.

I think the mapping node *MAP is usable if it can be safely 
loaded from this quasi-YAML document.


> > > A YAML processor may treat equal scalars as if they were
> > > identical.
> > I have not understood the importance of this description, though.
> 
> It is useful to declare that scalars don't have "identity" as such, but
> "objects" do. That is, "objects" have identity, but scalars don't -
> e.g., the number 4 has no "identity", but the "!!point { x: 4 }" does.
> This maps to the way 99% of the programs model their data (you can
> change the 'x' coordinate of the point and it will remain the "same"
> point, but you can't change the number 4 to the number 5, ever).
> 
> In practical terms this allows YAML libraries to avoid having to
> re-serialize large scalars (e.g., binary data, texts, etc.). Not the
> _most_ useful thing in the world, but it does have its place.

I do not like this, because YAML's scalar nodes are not always mapped
to the language native scalars. Remember String in ruby are objects 
rather than scalar. DateTime objects of ruby and C# too.

#!/usr/bin/ruby

a = "abc"
b = a
a[0]= "A"
p b
# "Abc"

I prefer to have the nodes identity not flexible for the scalars, 
too, in order to maximize the portability of YAML between languages.


> > I can not think of any other meaning of nodes equality in the
> > parsing and composition stages. Then, how about the meaning after
> > the composition?
> 
> After composing native objects, YAML has no say at all about anything.
> The application may wreak whatever havoc it wants on these structures;
> that's its job after all.

YAML does not say anything for the equality of the native objects 
after construction. But YAML prevents us to construct the native 
objects if it judges they are equal on its standard, when they 
appear as keys in a YAML mapping.

I think this is inconsistent. So, what you say is probably that we 
must always have the UniqueHash property when we have to compare 
objects by their identity, isn't it?


>> 1. When a node has a Tag that is not a YAML's standard tag, the YAML 
>>  parser and composer should evaluate nodes' equality only from the 
>>  identity of the nodes, because the parser and composer do not know 
>>  how to compare the data correctly. If the equality in their native 
>>  data form matters, it should be checked at the construction stage,
>>  where the library has full access to the native object and  to its 
>>  equality operator.
>
>> 3. For !!seq and !!map, the YAML parser and composer should evaluate 
>>  the equality only from the identity of the nodes. This means, even 
>>  if a mapping node has two collection nodes that have same content,
>>  the library should not reject such an input, instead, they should 
>>  pass through it to the constructor. 
>
> This would greatly weaken portability of YAML data between systems.

Let me separate the issues.

For the 1st rule of mine, it will not weaken the protability of YAML 
at any sense. Because when the data have Tags that specify the way to 
compare the native objects, the data can be correctly treated in any
environment. The portability is completely preserved.


For the 3rd rule of mine, it indeed weakens the portability. Even so,
I wanted to allow the users to map their language native Hash object 
into a YAML's mapping node seamlessly. For me, the current spec seems 
to forbid it as discussed above.


If you think YAML's !!map and !!seq should have their own way of 
equality evaluation independent of a specific language, it might 
be an option to allow the directives like next.

%ALIAS !!map !ruby/object:Hash
%ALIAS !!seq !ruby/object:Array

This will preserve the portablity.
If no library is aware of YAML's node equality, it will change nothing,
though.

Best,
Osamu TAKEUCHI



------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf