Re: YAML simplification

--===============4773882568884946396==
Content-Type: multipart/alternative; boundary="94eb2c05ae48543ae905669ea656"

--94eb2c05ae48543ae905669ea656
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Andrey/Peter,

Thanks for bringing this up. I=E2=80=99ll try to state my vision and what w=
e are
aiming for in YAML 1.3.

There are categories of change for the above: Syntax and Schema. Syntax is
an obvious concept. Schema (especially for YAML) is more complex and
nuanced.

There is a non-obvious concept to truly understanding YAML: any given YAML
document (by itself) has no known precise meaning. Put another way, any
YAML document can *mean* anything. What is needed to give meaning to a YAML
document is the concept of =E2=80=9CSchema=E2=80=9D.

At present, Schema is a very grey area. At the time that we put out the 1.2
spec, we agreed amongst ourselves, that the preferred default schema is the
same (as much as applicable) to JSON. i.e. all plain, non-numeric scalars
are loaded as strings except for the values [true, false, null].
Unfortunately we never retired the YAML type repository which suggests many
other implicits. It is no wonder that there is so much variance between
implementations.

The concept of YAML schema has not been yet been defined. I started working
on a project called =E2=80=9CSchemaType <http://www.schematype.org/>=E2=80=
=9D a couple
years ago, but it is still a WIP.

The future of YAML, imho, lies primarily in schema-aware implementations.
Here is some pseudo-code to demonstrate:

schema-object =3D YAML.Schema.load(=E2=80=98something-awesome.schema=E2=80=
=99)

data-object =3D YAML.load-file(=E2=80=98something-awesome.yaml=E2=80=99, sc=
hema=3Dschema-object)

yaml-text =3D YAML.dump(data-object, schema=3Dschema-object)


Where the default schema is something as simple as possible. Personally I
imagine that *all* non-numeric scalars load as strings and we use [!true,
!false, !null] for those values. As you can see, this idea of Schema goes
beyond validation. It also contains formatting info on how data should be
dumped/emitted.

An interesting thing is that all the current implementations do have
=E2=80=9Cschemas=E2=80=9D but not in any concise doc. The schema for a give=
n implementation
is encoded in programming source code. The result is that most
implementations are consistent with themselves but not with other
implementations. This makes YAML useful in a contained environment but less
so between heterogenous environments.

The cool thing is that we could, right now, write up schema docs for all
the existing implementations. Of course they wouldn=E2=80=99t *do* anything=
 besides
be a quick reference on what to expect a particular implementation to do.
But once we move towards having schema aware implementations, then we can
move forward to what we *want* while still supporting the old schemas for
legacy code.

Note: the above is the solution for schema related things, not syntax.

For syntax, a group of us are working on YAML 1.3. See:

   - YAML Test Suite <https://github.com/yaml/yaml-test-suite>
   - YAML Spec RFCs <https://github.com/yaml/yaml-spec/wiki/YAML-RFC-Index>

The focus of 1.3 is to tighten up al the edge cases of the 1.2 spec, such
that everything =E2=80=9Cimportant=E2=80=9D still works, but everything uni=
ntended or truly
undesirable is removed. Instead of working on a new spec to drive this (yes
a spec will be needed) we focus on an empirical test suite. We also are
making RFCs for each change, though several will be rejected, or will need
to wait past 1.3.

The bigger focus is that we do things in such a way, as that we end up with
all the implementations doing things in a consistent way. This will require
a Developer=E2=80=99s Guide, with clear advice on what should be considered=
 and
done.

Andrey, I=E2=80=99ll let you know where I fall on your 6 points now. Let me=
 preface
that although the world sees YAML as a better config language, YAML will
remain bigger than that. YAML is a human friendly serialization language.
It just happens that the features we used to make YAML, ended up being a
reasonable choice for config. :)

1) Absolutely. The default schema should be JSON or simpler (as noted
above). But schema-aware implementations can easily use legacy schemas.
2) Complex keys will stay, but in block form we will require an explicit
indicator =E2=80=98?=E2=80=99. YAML will stick to its serialization languag=
e guns.
3) I used to agree with getting rid of folded scalars, but now it is widely
used and to good effect. Without it, a stray =E2=80=98:=E2=80=99 or =E2=80=
=98#=E2=80=99 can mess up a
multiline. > stays for 1.3.
4/5) To be able to encode any printable value as a block literal, these are
needed. They are certainly edge cases, but they aren=E2=80=99t hard to impl=
ement.
Keeping them in 1.3.
6) %YAML and %TAG are the only directives. This topic deserves it=E2=80=99s=
 own
email. I want to get rid of them. %TAG takes a ton of effort to implement
for little value imho. If you have schema, the need evaporates. %YAML is
not really useful, because there is no clear/precise/concise meaning to
1.0/1/1/1.2. The spec is just not precise enough to be the final say on
what a YAML doc means, sadly. Also nobody uses %YAML now, and I don=E2=80=
=99t want
to force people to start using this.

Soooo for #6, not sure about %TAG for 1.3. %YAML will stay for 1.3. It=E2=
=80=99s
easy to implement.

Note: I really think if %YAML is needed (i.e. we can=E2=80=99t get the info=
 any
other way), then it might turn into:

%YAML http://ingy.net/awesome.schema


i.e., you associate the doc to a schema, not a version. The schema can
define what the spec version is and a whole lot more. This is a lot more
compelling for people to add into their YAML files, because people can
follow the link to read for themselves exactly what is intended. If it=E2=
=80=99s
only a version #, there are very few people in the world (if anybody at
all) who knows exactly what a version means. Eventually the *test suite*
will have tests for every case in every version, but nobody could be
expected to have the entire scope of the test suite in their head.

re: https://github.com/yaml/yaml-spec/wiki which was mostly written in
2011, we=E2=80=99ll need to look over those ideas again and move them over =
to the
RFCs.

Development discussion is active on FreeNode IRC in #yaml-dev and most
discussion of things happens in GItHub issues on projects under
https://github.com/yaml/
Andrey I know you aren=E2=80=99t a fan of IRC, but this email list is prett=
y much
retired. IRC is still used on many Major projects.

Everyone: YAML development resources are greatly needed. Don=E2=80=99t hesi=
tate to
get involved.

Peter, I think my answers to Andrey should answer your stances. With
schema-awareness the default is trivial and interoperable but any other
behavior only requires a new schema. FYI SchemaType is designed to be very
Object Oriented, which is to say, new complex schemas can =E2=80=9Cinherit=
=E2=80=9D from
existing ones (if they exist of course) and just override a few properties.
Getting exactly what you want/need should be easy.

Cheers, Ingy

On Sun, Mar 4, 2018 at 4:33 AM, Andrey Somov <[email protected]> wrote=
:

> Hi all,
> I see 2 major directions for YAML evolution:
>
> 1. radical simplification
> 2. more features
>
> The advantages of the first:
> a) easy adoption
> b) rock solid spec - because there is nothing to change like in JSON
> c) no need to mention version (1.1, 1.2. 1.3, 2.0 etc)
> d) we can drop %YAML directive because all the versions are supposed to b=
e
> the same
>
> I do not claim that simplification does not have disadvantages but does
> make YAML more user friendly.
>
> (who complains that boolean has 2 values instead of 18 ? )
>
> I would like to vote to the features which can and should be dropped:
>
> 1. all the implicit types except the ones from JSON (users should be able
> to define custom implicit types) Recommended Schema_s_ should become
> Schem_a_
> 2. Complex keys (let us allow JavaScript to implement the spec). No need
> to discuss the 1024 limit
> 3. Folded scalar
> 4. Chomping indicators
> 5. Indentation indicators (who is using it and why?)
> 6. Directives
>
>
> Feel free to add your vision here.
>
> Cheers,
> Andrey
> SnakeYAML developer
>
> P.S. I fully support proposals (https://github.com/yaml/yaml-spec/wiki)
> which make YAML  stricter.
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Yaml-core mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/yaml-core
>
>

--94eb2c05ae48543ae905669ea656
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Andrey/Peter,<div><br></div><div>Thanks for bringing this =
up. I=E2=80=99ll try to state my vision and what we are aiming for in YAML =
1.3.</div><div><br></div><div>There are categories of change for the above:=
 Syntax and Schema. Syntax is an obvious concept. Schema (especially for YA=
ML) is more complex and nuanced.</div><div><br></div><div>There is a non-ob=
vious concept to truly understanding YAML: any given YAML document (by itse=
lf) has no known precise meaning. Put another way, any YAML document can *m=
ean* anything. What is needed to give meaning to a YAML document is the con=
cept of =E2=80=9CSchema=E2=80=9D.</div><div><br></div><div>At present, Sche=
ma is a very grey area. At the time that we put out the 1.2 spec, we agreed=
 amongst ourselves, that the preferred default schema is the same (as much =
as applicable) to JSON. i.e. all plain, non-numeric scalars are loaded as s=
trings except for the values [true, false, null]. Unfortunately we never re=
tired the YAML type repository which suggests many other implicits. It is n=
o wonder that there is so much variance between implementations.</div><div>=
<br></div><div>The concept of YAML schema has not been yet been defined. I =
started working on a project called =E2=80=9C<a href=3D"http://www.schematy=
pe.org/">SchemaType</a>=E2=80=9D a couple years ago, but it is still a WIP.=
</div><div><br></div><div>The future of YAML, imho, lies primarily in schem=
a-aware implementations. Here is some pseudo-code to demonstrate:</div><div=
><br></div><blockquote style=3D"margin:0px 0px 0px 40px;border:none;padding=
:0px"><div>schema-object =3D YAML.Schema.load(=E2=80=98something-awesome.sc=
hema=E2=80=99)</div></blockquote><blockquote style=3D"margin:0px 0px 0px 40=
px;border:none;padding:0px"><div>data-object =3D YAML.load-file(=E2=80=98so=
mething-awesome.yaml=E2=80=99, schema=3Dschema-object)</div></blockquote><b=
lockquote style=3D"margin:0px 0px 0px 40px;border:none;padding:0px"><div>ya=
ml-text =3D YAML.dump(data-object, schema=3Dschema-object)</div></blockquot=
e><br><div>Where the default schema is something as simple as possible. Per=
sonally I imagine that <b>all</b>=C2=A0non-numeric scalars load as=C2=A0str=
ings and we use [!true, !false, !null] for those values. As you can see, th=
is idea of Schema goes beyond validation. It also contains formatting info =
on how data should be dumped/emitted.</div><div><br></div><div>An interesti=
ng thing is that all the current implementations do have =E2=80=9Cschemas=
=E2=80=9D but not in any concise doc. The schema for a given implementation=
 is encoded in programming source code. The result is that most implementat=
ions are consistent with themselves but not with other implementations. Thi=
s makes YAML useful in a contained environment but less so between heteroge=
nous environments.</div><div><br></div><div>The cool thing is that we could=
, right now, write up schema docs for all the existing implementations. Of =
course they wouldn=E2=80=99t *do* anything besides be a quick reference on =
what to expect a particular implementation to do. But once we move towards =
having schema aware implementations, then we can move forward to what we *w=
ant* while still supporting the old schemas for legacy code.</div><div><br>=
</div><div>Note: the above is the solution for schema related things, not s=
yntax.</div><div><br></div><div>For syntax, a group of us are working on YA=
ML 1.3. See:</div><div><ul><li><a href=3D"https://github.com/yaml/yaml-test=
-suite">YAML Test Suite</a></li><li><a href=3D"https://github.com/yaml/yaml=
-spec/wiki/YAML-RFC-Index">YAML Spec RFCs</a></li></ul></div><div>The focus=
 of 1.3 is to tighten up al the edge cases of the 1.2 spec, such that every=
thing =E2=80=9Cimportant=E2=80=9D still works, but everything unintended or=
 truly undesirable is removed. Instead of working on a new spec to drive th=
is (yes a spec will be needed) we focus on an empirical test suite. We also=
 are making RFCs for each change, though several will be rejected, or will =
need to wait past 1.3.</div><div><br></div><div>The bigger focus is that we=
 do things in such a way, as that we end up with all the implementations do=
ing things in a consistent way. This will require a Developer=E2=80=99s Gui=
de, with clear advice on what should be considered and done.</div><div><br>=
</div><div>Andrey, I=E2=80=99ll let you know where I fall on your 6 points =
now. Let me preface that although the world sees YAML as a better config la=
nguage, YAML will remain bigger than that. YAML is a human friendly seriali=
zation language. It just happens that the features we used to make YAML, en=
ded up being a reasonable choice for config. :)</div><div><br></div><div>1)=
 Absolutely. The default schema should be JSON or simpler (as noted above).=
 But schema-aware implementations can easily use legacy schemas.</div><div>=
2) Complex keys will stay, but in block form we will require an explicit in=
dicator =E2=80=98?=E2=80=99. YAML will stick to its serialization language =
guns.</div><div>3) I used to agree with getting rid of folded scalars, but =
now it is widely used and to good effect. Without it, a stray =E2=80=98:=E2=
=80=99 or =E2=80=98#=E2=80=99 can mess up a multiline. &gt; stays for 1.3.<=
/div><div>4/5) To be able to encode any printable value as a block literal,=
 these are needed. They are certainly edge cases, but they aren=E2=80=99t h=
ard to implement. Keeping them in 1.3.</div><div>6) %YAML and %TAG are the =
only directives. This topic deserves it=E2=80=99s own email. I want to get =
rid of them. %TAG takes a ton of effort to implement for little value imho.=
 If you have schema, the need evaporates. %YAML is not really useful, becau=
se there is no clear/precise/concise meaning to 1.0/1/1/1.2. The spec is ju=
st not precise enough to be the final say on what a YAML doc means, sadly. =
Also nobody uses %YAML now, and I don=E2=80=99t want to force people to sta=
rt using this.</div><div><br></div><div>Soooo for #6, not sure about %TAG f=
or 1.3. %YAML will stay for 1.3. It=E2=80=99s easy to implement.<br></div><=
div><br></div><div>Note: I really think if %YAML is needed (i.e. we can=E2=
=80=99t get the info any other way), then it might turn into:</div><div><br=
></div><blockquote style=3D"margin:0px 0px 0px 40px;border:none;padding:0px=
"><div>%YAML <a href=3D"http://ingy.net/awesome.schema">http://ingy.net/awe=
some.schema</a><br></div></blockquote><br><div>i.e., you associate the doc =
to a schema, not a version. The schema can define what the spec version is =
and a whole lot more. This is a lot more compelling for people to add into =
their YAML files, because people can follow the link to read for themselves=
 exactly what is intended. If it=E2=80=99s only a version #, there are very=
 few people in the world (if anybody at all) who knows exactly what a versi=
on means. Eventually the <b>test suite</b> will have tests for every case i=
n every version, but nobody could be expected to have the entire scope of t=
he test suite in their head.</div><div><br></div><div>re:=C2=A0<a href=3D"h=
ttps://github.com/yaml/yaml-spec/wiki">https://github.com/yaml/yaml-spec/wi=
ki</a>=C2=A0which was mostly written in 2011, we=E2=80=99ll need to look ov=
er those ideas again and move them over to the RFCs.</div><div><br></div><d=
iv>Development discussion is active on FreeNode IRC in #yaml-dev and most d=
iscussion of things happens in GItHub issues on projects under=C2=A0<a href=
=3D"https://github.com/yaml/">https://github.com/yaml/</a></div><div>Andrey=
 I know you aren=E2=80=99t a fan of IRC, but this email list is pretty much=
 retired. IRC is still used on many Major projects.</div><div><br></div><di=
v>Everyone: YAML development resources are greatly needed. Don=E2=80=99t he=
sitate to get involved.</div><div><br></div><div>Peter, I think my answers =
to Andrey should answer your stances. With schema-awareness the default is =
trivial and interoperable but any other behavior only requires a new schema=
. FYI SchemaType is designed to be very Object Oriented, which is to say, n=
ew complex schemas can =E2=80=9Cinherit=E2=80=9D from existing ones (if the=
y exist of course) and just override a few properties. Getting exactly what=
 you want/need should be easy.</div><div><br></div><div>Cheers, Ingy</div><=
/div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Sun, Mar =
4, 2018 at 4:33 AM, Andrey Somov <span dir=3D"ltr">&lt;<a href=3D"mailto:pu=
[email protected]" target=3D"_blank">[email protected]</a>&gt;</spa=
n> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div><div=
><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Hi a=
ll,<br></div></div>I see 2 major directions for YAML evolution:<br><br></di=
v>1. radical simplification<br></div>2. more features<br><br></div>The adva=
ntages of the first:<br></div>a) easy adoption <br></div>b) rock solid spec=
 - because there is nothing to change like in JSON<br></div><div>c) no need=
 to mention version (1.1, 1.2. 1.3, 2.0 etc)<br></div>d) we can drop %YAML =
directive because all the versions are supposed to be the same<br><br>I do =
not claim that simplification does not have disadvantages but does make YAM=
L more user friendly.<br><br></div><div>(who complains that boolean has 2 v=
alues instead of 18 ? )<br><br></div><div>I would like to vote to the featu=
res which can and should be dropped:<br><br></div><div>1.
 all the implicit types except the ones from JSON (users should be able=20
to define custom implicit types) Recommended Schema_s_ should become=20
Schem_a_<br></div><div>2. Complex keys (let us allow JavaScript to implemen=
t the spec). No need to discuss the 1024 limit<br></div><div>3. Folded scal=
ar<br></div><div>4. Chomping indicators<br></div><div>5. Indentation indica=
tors (who is using it and why?)<br></div><div>6. Directives<br><br><br></di=
v><div>Feel free to add your vision here.<br><br></div><div>Cheers,<br></di=
v><div>Andrey<br></div><div>SnakeYAML developer<br><br></div><div>P.S. I fu=
lly support proposals (<a href=3D"https://github.com/yaml/yaml-spec/wiki" t=
arget=3D"_blank">https://github.com/yaml/yaml-<wbr>spec/wiki</a>) which mak=
e YAML=C2=A0 stricter.<div class=3D"m_-6623790634836952272gmail-yj6qo"></di=
v><div class=3D"m_-6623790634836952272gmail-adL"><br></div></div></div></di=
v></div></div></div></div></div></div><br></div>
<br>------------------------------<wbr>------------------------------<wbr>-=
-----------------<br>
Check out the vibrant tech community on one of the world&#39;s most<br>
engaging tech sites, Slashdot.org! <a href=3D"http://sdm.link/slashdot" rel=
=3D"noreferrer" target=3D"_blank">http://sdm.link/slashdot</a><br>_________=
_____________________<wbr>_________________<br>
Yaml-core mailing list<br>
<a href=3D"mailto:[email protected]">[email protected]=
rge.<wbr>net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/yaml-core" rel=3D"n=
oreferrer" target=3D"_blank">https://lists.sourceforge.net/<wbr>lists/listi=
nfo/yaml-core</a><br>
<br></blockquote></div><br></div>

--94eb2c05ae48543ae905669ea656--


--===============4773882568884946396==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--===============4773882568884946396==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Yaml-core mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/yaml-core

--===============4773882568884946396==--