Type mapping between native data type to YAML tags (was Re: utf8u tag proposal)

Hi BlueG,

> To always have an exact match using tags like that would result in every
> programming language having its own set of tags. While this is fine for
> serialization and deserialization using the same language and library (and,
> therefore, useful to solitary applications), but it has problems when we
> need to deserialize to and from different languages or libraries (when we
> have systems of applications, such as in an enterprise or because we are
> using utilities created by others that may be written in other languages).

Sorry, I did not mean we should always use tags incompatible 
to YAML's standard. Here is the type mapping in my library.
I should have shown this earlier.

+---------------------------+---------------+
| C# Type                   | YAML          |
+---------------------------+---------------+
| bool                      | !!bool        |
+---------------------------+---------------+
| int                       | !!int         |
+---------------------------+---------------+
| double                    | !!float       |
+---------------------------+---------------+
| string                    | !!str, !!null |
+---------------------------+---------------+
| N/A currently             | !!binary      |
+---------------------------+---------------+
| DateTime                  | !!timestamp   |
+---------------------------+---------------+
| object[]                  | !!seq         |
+---------------------------+---------------+
| Dictionary<object,object> | !!map         | *Not fully 
+---------------------------+---------------+  supported yet

If an application represents its data only with these types 
and pass it to my library, the resulting YAML will not have 
any local tags. But to do so, the application probably has 
to convert its native data to a YAML friendly form. 
This is a pain to be portable.
# Conversion from Dictionary<object,object> to !!map
# has been forgot to be implemented. I'm willing to have it.

Note that the situation is same as the ruby's YAML library. 
For example, it dose not automatically convert Symbol into 
!!str; so if you have Symbol values in your data, you will 
find local tags in your YAML document. If you do not like 
it, you have to convert them into String before you pass it
to the library.


I underline that only the application knows whether 
or not it really needs local tags and how much portability 
it needs. So, a library and YAML should not force too much 
portability to the application if it is not needed. 
Remember the portability almost always accompanies with 
some pain.


I think, for many applications, there is no chance to have 
their configuration files or saved data be read by any 
other application, because they are really solitary.
For such an application, the file format does not have 
to be very much portable. It can be only understandable 
to the application itself and the possible supporting 
tools that know the detail of the data structure. Even 
in such cases, they might want to use YAML as the data 
container because YAML is well organized and readable. 
We have no reason to prevent them from using YAML.

For example, a window framework system might make use of my 
library. They serialize the top level windows' position, color, 
font, caption and the child controls on them into YAML. 
There can be variety of controls on variety of forms and panels. 
My library allows such a system to serialize and to rebuild 
all the windows and controls at once with the next code.

  var serializer = new YamlSerializer();
  
  // Serialize and discard
  var yaml = serializer.Serialize( Framework.ToplevelWindows );
  Framework.ToplevelWindows.Dispose();

  // !Framework.WindowCollection
  // ICollection.Items:
  //   - !Framework.MainWindow
  //     Position: { ??? }
  //     Caption: Hello!
  //     ???
  //     Controls:
  //       ICollection.Items:
  //         - !Framework.TextBox
  //           Position: { ??? }
  //           Text: Edit this!
  //           ???
  //         - !Framework.Button
  //           Position: { ??? }
  //           Caption: OK
  //           ???
  //         ???
  //   - !Framework.ToolWindow
  //     ???
  
  // Rebuild
  Framework.ToplevelWindows = 
      (WindowCollection) serializer.Deserialize( yaml )[0];

Oops, I found that my library currently does not serialize the 
events. Then, this example must not be practical. Even so, 
I think I have shown that, in some application, local tags are 
inevitable and such a YAML document does not have to be very 
much portable, or can not be.

I agree some applications need portability for their YAML 
files, like BlueG's example. As shown, such applications 
can also make use of my library with preserving their 
document's portability.

A general purpose library should leave the judgement to its 
users.


> As an aside too, in C#, it really isn't necessary to explicitly state what
> type each member of a serialized class is. When deserializing, that is
> already known from the type of the class being deserialized (and,
> consequently the type of each member as it is deserialized). The only thing
> that would need to be known is the type of the root object. This can be
> accomplished either by the application stating what that type when it calls
> the deserializer or by storing it as a scalar at the beginning of the
> document or by using a local tag to specify the type. This is one way of
> defining a schema (in this case, the schema is implied by the type system)
> and is why I mentioned schemas as a solution to the problem of mapping data
> types. Schemas can be implicit or explicit. Using a class hierachy would be
> implicit whereas an external schema, such as I mentioned, or something
> similar would be explicit.

I did not see your point. If you wrote that C#'s data
type can be known from the variable's type, that is not 
always true. Note that a variable defined as SomeClass 
can contain an object of DescendantClass.

  class SomeClass { }
  
  class DescendantClass: SomeClass { }
  
  SomeClass some = new DescendantClass();

In that sense, a variable defined as object class can 
contain any object in C# because any data type in C# is 
a descendant of object class.

  object obj = null;
  obj = "a";
  obj = IntPtr.Zero;
  obj = true;
  obj = new SomeClass();

Again, the situation is as the same for ruby, except for 
the three differences; in ruby, every variable is defined 
as Object; in ruby, the method invocation is resolved 
completely at runtime; in C#, null is not an object.


>> >   string yaml = serializer.Serialize(new Test());
>> >   // %YAML 1.2
>> >   // ---
>> >   // !Test
>> >   // a: 3
>> >   // b: 2.1
>> >   // ...
>> > 
>> > Here, the Tags !System.Byte and !System.UIng64 are not 
>> > specified explicitly to the fields a and b but implicitly 
>> > resolved from the tag !Test to the parent node. They are not 
>> > converted to !!int for serialization.
> 
> Right, the schema is implied by the Test type. As an aside, most
> applications are going to know what type the root object is and so can
> provide that type to the deserializer. For this reason, it usually isn't
> necessary to specify the type of the root node either.

I'm now willing to have an option to my library with which
we can omit the tag for the root object. As you wrote, it will 
make the YAML document look better in some cases. Thanks.


Best,
Osamu TAKEUCHI.


------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf