Implicit Typing

My $.02 about implicit typing, as it seems to be biting many people...

We seem to have different interpretations of how the YAML type
repository should be used. This ambiguity has leaked into the YAML
implementations, and we need to clear things up.

The following is MVHO, Ingy/Clark may have a different angle on things: 

The YAML type repository is NOT intended to be interpreted as "here is a
set of types to use by default". While _some_ of the types there
certainly should be available by default, not all should. Instead, the
repository should be interpreted as "here are some types, if you need
them, use them in this way to maximize portability between
applications". This is why the types are not part of the spec itself -
this set of types is dynamic and can grow without bound, without
affecting parser implementations.

This punts the question of "what should be the default set of implicit
types"? We have intentionally avoided defining one so far. As Clark
points out, however, we now have a "gold standard" to work with, JSON,
which defines a particular set of types that "everyone" has learned to
accept. It is small, and limited, but very "unsurprising".

Therefore, if implementations reduced the set of built-in types to
JSON-compatible types (plus possibly some useful types we designate as
default that are sufficiently "unsurprising", such as << etc.), the
problem would - mostly - go away.

The boolean case raises a second issue however. There is a conflict
between "correctly" guessing the types of data with utter lack of schema
information, and "correctly" interpreting the value of data when the
schema is known. In the boolean case, this means that when reading a
random-off-the-street file without any knowledge what it means,
interpreting Y as a string seems to be the right thing to do. At the
same time, when a server application is reading reading a boolean field
in its own dedicated configuration file, interpreting the same Y
character as the boolean value true also seems as the right thing to do.

This leaves us in something of a bind. The immediate solution that comes
to mind is:

- First, for the same type (e.g. boolean), there would be two sets of
regexps. One would be used to detect the type (assuming it is supported)
in the basic set of types. The second would allow for additional
variants but would only be applicable if the application "knows" the
field type already.

- Second, we specify that unless the schema is explicitly specified
(using a tag), a parser should only use the basic set of types.
Specifying a tag for a node (such as the root node) informs the parser
of the expected type of the node, and if this type is a complex type, it
may induce types on some of the contained nodes. For example the type
'map' gives the parser no knowledge of the contained types, but the type
'point' type specified the type 'float' for its 'x' and 'y' values.

- Finally, we need a format way to specify schemas/tags/types that will
allow parsers to do the right thing.

An immediate objection problem to this approach is the infamous DTD
problem - parsing a document with and without a DTD yielded different
results, which was (and still is) a big pain when working with XML. In
our case, parsing a document containing tags with and without knowledge
of what they mean would yield different results.

YAML side-steps this problem, neatly or messily depending on your point
of view. The spec clearly defines what it means to create a "partial"
representation of a YAML file. Specifically, if the parser does not
recognize the explicit tags used in the document, it is expected to
report this fact and not attempt to assign a type to these nodes anyway.
In contrast, in XML, parsing an XML document with and without a DTD
yields the same information model, so there is no way for the
application to realize it is missing information.

That said, a YAML parser may choose to forge ahead and use the basic set
of types on the data, hoping for the best. Doing so, however,
potentially changes the semantics of the data. For example, Y becomes a
string instead of a boolean). Hence this "damn the torpedoes, full steam
ahead" approach should only be used with care and never as the default
option (for a parser library anyway).

Hope this helps,

	Oren Ben-Kiki

P.S. About the spec status - I have tinkered with my YamlReference
parser implementation, and it is now a fully streaming parser that can
handle arbitrarily large inputs (Haskell is a tricky language...). I am
waiting for some technical work on the yaml.org servers to upload an
HTML interface that will allow people to view the results of parsing
YAML fragments, and use these to report bugs or dispute the spec. Once
this is up I'll start working on an updated spec version - hopefully
this would be "the" final 1.1 spec, period.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/