Re: LibYAML tests fail in master (2021-12-20)
Ingy dot Net <[email protected]> Mon, 20 Dec 2021 12:51:02 -0500
| Newsgroups | gmane.text.yaml.general |
|---|---|
| Message-ID | <CAHJtQJ4YE19fZS+7fGJ11P17w6P+Pi27GcLXtdSv6L5uxeAofA@mail.gmail.com> |
--===============3022812636078745531==
Content-Type: multipart/alternative; boundary="000000000000e2e23a05d397880b"
--000000000000e2e23a05d397880b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Hi Andrey!
As a reminder, we don't use the mailing list for this or almost anything
anymore.
The YAML language design team <https://yaml.org/spec/1.2.2/ext/team/> is
very active these days and we have a host of modern ways to engage.
The short answer to your query is that in this case libyaml is right and
pyyaml is wrong.
Apologies for the libyaml tests not running out of the box.
That's on me, and I'll try to find some time to fix this at some point.
https://play.yaml.io/main/parser?input=3DImJhciIJ shows the results of 14
YAML parsers, and PyYAML, Ruamel (fork of PyYAML) and SnakeYAML get this
one wrong.
The New Reference Parser <https://github.com/yaml/yaml-reference-parser/>
there is literally *generated* from the spec productions and therefore is
almost always correct in its interpretation.
That might be a useful resource for you.
The productions involved in the (recently published
<https://yaml.com/blog/2021-10/new-yaml-spec/>) 1.2.2 spec
<https://yaml.org/spec/1.2.2/> are:
- https://yaml.org/spec/1.2.2/#rule-c-ns-flow-map-json-key-entry
- https://yaml.org/spec/1.2.2/#rule-c-flow-json-node
- https://yaml.org/spec/1.2.2/#rule-c-double-quoted
- https://yaml.org/spec/1.2.2/#rule-s-separate
- https://yaml.org/spec/1.2.2/#rule-s-separate-in-line
- https://yaml.org/spec/1.2.2/#rule-s-white
- Which is spaces and tabs
Put another way, non-indentation whitespace is usually tabs and spaces.
Note: I'm not personally a fan of allowing tabs anywhere (except in literal
scalar content), but that's where the 1.2 productions stand.
I assume you are aware of the YAML test suite
<https://github.com/yaml/yaml-test-suite>.
Our testing of compliant frameworks is shown in the YAML Test Matrix
<https://matrix.yaml.info/>.
SnakeYAML <https://bitbucket.org/snakeyaml/snakeyaml/src/master/> is
failing quite a bit, but then again so are libyaml and PyYAML.
Unfortunately we don't yet have support for SnakeYAML Engine
<https://bitbucket.org/snakeyaml/snakeyaml-engine/src/master/> yet but we
just need someone to write a port of the tester
<https://github.com/yaml/yaml-runtimes/blob/3411ed2868d4bc5b8c9c5f421565d2c=
887892329/docker/java/utils/java/src/test/java/org/yaml/editor/Snake2Events=
Test.java>
to
have it included in the results.
Also, speaking of libyaml, I should point out the new C library for yaml
called libfyaml <https://github.com/pantoniou/libfyaml> which is quite an
amazing work.
Regarding the testing of libyaml (and everything else), running 'make clean
run-tests export' in the YAML test suite directory creates an export.tsv
file which we copy/paste into this Google Sheet
<https://docs.google.com/spreadsheets/d/e/2PACX-1vSlIvGsUqrCvf4s9PhVJeAyvvl=
x0rOfZFgox4ZexwCYpPH0gdkQN36o0AJaDageHbLp-FgY538FLO5c/pubhtml?gid=3D0&singl=
e=3Dtrue>
(with
results totals) when the suite is updated.
So that's an alternate way of testing libyaml...
We have a rather new community platform <https://matrix.to/#/#chat:yaml.io>
for engaging directly with the core team about YAML development or anything
else YAML related.
If you haven't used element/matrix, I'd describe it as completely open and
secure Slack.
It works great, has great desktop and phone clients but also works just as
good in a browser, and is trivial to join.
I'll also reply directly to your PyYAML issue
<https://github.com/yaml/pyyaml/issues/594>. :)
Almost all this info is on the plain old https://yaml.org/ site which we
are actively keeping up to date.
But personally, Andrey, I really hope you join the new chat.
We really want SnakeYAML to be as good as possible, and have been hoping to
engage with you!
BTW, no foul in posting here.
It gave me a great excuse to promote some of the recent YAML work.
I hope you and other readers find it useful.
Thanks and happy end of 2021!
=E2=80=94 Ingy
On Mon, Dec 20, 2021 at 2:33 AM Andrey Somov <[email protected]> wrote=
:
> Hi all,
> I wanted to have a look at LibYAML to understand the reason for
> https://github.com/yaml/pyyaml/issues/594
>
> Unfortunately the tests fail in master.
> Can you please help me to make the tests run ?
>
> tests/run-all-tests.sh
> ...
> make[2]: Entering directory
> '/home/ooosssmmm/rev-projects/yaml/libyaml/tests'
> PASS: test-version
> PASS: test-reader
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> Testsuite summary for yaml 0.2.5
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> # TOTAL: 2
> # PASS: 2
> # SKIP: 0
> # XFAIL: 0
> # FAIL: 0
> # XPASS: 0
> # ERROR: 0
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> make[2]: Leaving directory
> '/home/ooosssmmm/rev-projects/yaml/libyaml/tests'
> make[1]: Leaving directory
> '/home/ooosssmmm/rev-projects/yaml/libyaml/tests'
> git branch --track run-test-suite origin/run-test-suite
> fatal: A branch named 'run-test-suite' already exists.
> make: [Makefile:888: tests/run-test-suite] Error 128 (ignored)
> git worktree prune
> git worktree add tests/run-test-suite run-test-suite
> Preparing worktree (checking out 'run-test-suite')
> HEAD is now at 7b73fcf Move testing code into run-test-suite-code branch
> make -C tests/run-test-suite test
> make[1]: Entering directory
> '/home/ooosssmmm/rev-projects/yaml/libyaml/tests/run-test-suite'
> git clone https://github.com/yaml/yaml-test-suite data --branch=3Ddata
> Cloning into 'data'...
> remote: Enumerating objects: 16712, done.
> remote: Counting objects: 100% (2667/2667), done.
> remote: Compressing objects: 100% (1313/1313), done.
> remote: Total 16712 (delta 1663), reused 2067 (delta 1160), pack-reused
> 14045
> Receiving objects: 100% (16712/16712), 4.84 MiB | 2.24 MiB/s, done.
> Resolving deltas: 100% (10668/10668), done.
> (cd data && git reset --hard daf799e56372aa93c324d111c73b47a431d7e03b)
> fatal: Could not parse object 'daf799e56372aa93c324d111c73b47a431d7e03b'.
> make[1]: *** [Makefile:25: data] Error 128
> make[1]: Leaving directory
> '/home/ooosssmmm/rev-projects/yaml/libyaml/tests/run-test-suite'
> make: *** [Makefile:882: test-suite] Error 2
> _______________________________________________
> Yaml-core mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/yaml-core
>
--000000000000e2e23a05d397880b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div dir=3D"ltr">Hi=C2=A0Andrey!<div><br></div><div>As a r=
eminder, we don't use the mailing list for this or almost anything anym=
ore.</div><div>The <a href=3D"https://yaml.org/spec/1.2.2/ext/team/">YAML l=
anguage design team</a> is very active these days and we have a host of mod=
ern ways to engage.</div><div><br></div><div>The short answer=C2=A0to your =
query is that in this case libyaml is right and pyyaml is wrong.<br>Apologi=
es for the libyaml tests not running out of the box.</div><div>That's o=
n me, and I'll try to find some time to fix this at some point.<br><br>=
<a href=3D"https://play.yaml.io/main/parser?input=3DImJhciIJ">https://play.=
yaml.io/main/parser?input=3DImJhciIJ</a> shows the results of 14 YAML parse=
rs, and PyYAML, Ruamel (fork of PyYAML) and SnakeYAML get this one wrong.<b=
r>The <a href=3D"https://github.com/yaml/yaml-reference-parser/">New Refere=
nce Parser</a> there is literally <u>generated</u> from the spec production=
s and therefore is almost always correct in its interpretation.</div><div>T=
hat might be a useful resource for you.<br><br>The productions involved in =
the (<a href=3D"https://yaml.com/blog/2021-10/new-yaml-spec/">recently publ=
ished</a>) <a href=3D"https://yaml.org/spec/1.2.2/">1.2.2 spec</a>=C2=A0are=
:</div><div><ul><li><a href=3D"https://yaml.org/spec/1.2.2/#rule-c-ns-flow-=
map-json-key-entry">https://yaml.org/spec/1.2.2/#rule-c-ns-flow-map-json-ke=
y-entry</a></li><ul><li><a href=3D"https://yaml.org/spec/1.2.2/#rule-c-flow=
-json-node">https://yaml.org/spec/1.2.2/#rule-c-flow-json-node<br></a></li>=
<ul><li><a href=3D"https://yaml.org/spec/1.2.2/#rule-c-double-quoted">https=
://yaml.org/spec/1.2.2/#rule-c-double-quoted</a></li></ul></ul><li><a href=
=3D"https://yaml.org/spec/1.2.2/#rule-s-separate">https://yaml.org/spec/1.2=
.2/#rule-s-separate</a></li><ul><li><a href=3D"https://yaml.org/spec/1.2.2/=
#rule-s-separate-in-line">https://yaml.org/spec/1.2.2/#rule-s-separate-in-l=
ine</a></li><ul><li><a href=3D"https://yaml.org/spec/1.2.2/#rule-s-white">h=
ttps://yaml.org/spec/1.2.2/#rule-s-white</a></li><li>Which is spaces and ta=
bs</li></ul></ul></ul>Put another way, non-indentation whitespace is usuall=
y tabs and spaces.<br>Note: I'm not personally a fan of allowing tabs a=
nywhere (except in literal scalar content), but that's where the 1.2 pr=
oductions stand.=C2=A0<br><div><br></div>I assume you are aware of the <a h=
ref=3D"https://github.com/yaml/yaml-test-suite">YAML test suite</a>.=C2=A0<=
/div><div>Our testing of compliant frameworks is shown in the <a href=3D"ht=
tps://matrix.yaml.info/">YAML Test Matrix</a>.<br><a href=3D"https://bitbuc=
ket.org/snakeyaml/snakeyaml/src/master/">SnakeYAML</a>=C2=A0is failing quit=
e a bit, but then again so are libyaml and PyYAML.</div><div>Unfortunately =
we don't yet have support for <a href=3D"https://bitbucket.org/snakeyam=
l/snakeyaml-engine/src/master/">SnakeYAML=C2=A0 Engine</a>=C2=A0yet but we =
just need someone to write a port of <a href=3D"https://github.com/yaml/yam=
l-runtimes/blob/3411ed2868d4bc5b8c9c5f421565d2c887892329/docker/java/utils/=
java/src/test/java/org/yaml/editor/Snake2EventsTest.java">the tester</a>=C2=
=A0to have it included in the results.</div><div>Also, speaking of libyaml,=
I should point out the new C library for yaml called <a href=3D"https://gi=
thub.com/pantoniou/libfyaml">libfyaml</a>=C2=A0which is quite an amazing wo=
rk.</div><div><br></div><div>Regarding the testing of libyaml (and everythi=
ng else), running '<font face=3D"monospace">make clean run-tests export=
'</font> in the YAML test suite directory creates an export.tsv file wh=
ich we copy/paste into this <a href=3D"https://docs.google.com/spreadsheets=
/d/e/2PACX-1vSlIvGsUqrCvf4s9PhVJeAyvvlx0rOfZFgox4ZexwCYpPH0gdkQN36o0AJaDage=
HbLp-FgY538FLO5c/pubhtml?gid=3D0&single=3Dtrue">Google Sheet</a>=C2=A0(=
with results totals) when the suite is updated.<br>So that's an alterna=
te way of testing libyaml...</div><div><br></div><div>We have a rather <a h=
ref=3D"https://matrix.to/#/#chat:yaml.io">new community platform</a> for en=
gaging directly with the core team about YAML development or anything else =
YAML related.<br></div><div>If you haven't used element/matrix, I'd=
describe it as completely open and secure Slack.</div><div>It works great,=
has great desktop and phone clients but also works just as good in a brows=
er, and is trivial to join.</div><div><br></div><div>I'll also reply di=
rectly to your <a href=3D"https://github.com/yaml/pyyaml/issues/594">PyYAML=
issue</a>. :)</div><div><br></div><div>Almost all this info is on the plai=
n old=C2=A0<a href=3D"https://yaml.org/">https://yaml.org/</a> site which w=
e are actively keeping up to date.</div><div><br></div><div>But personally,=
Andrey, I really hope you join the new chat.<br>We really want SnakeYAML t=
o be as good as possible, and have been hoping to engage with you!</div><di=
v><br></div><div>BTW, no foul in posting here.</div><div>It gave me a great=
excuse to promote some of the recent YAML work.</div><div>I hope you and o=
ther readers find it useful.</div><div><br></div><div>Thanks and happy end =
of 2021!</div><div><br></div><div>=E2=80=94 Ingy</div></div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Dec 20, 2021=
at 2:33 AM Andrey Somov <<a href=3D"mailto:[email protected]">publ=
[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex"><div dir=3D"ltr"><div>Hi all,</div><div>I wanted to have =
a look at LibYAML to understand the reason for <a href=3D"https://github.co=
m/yaml/pyyaml/issues/594" target=3D"_blank">https://github.com/yaml/pyyaml/=
issues/594</a></div><div><br></div><div>Unfortunately the tests fail in mas=
ter.</div><div>Can you please help me to make the tests run ?<br></div><div=
><br></div><div>tests/run-all-tests.sh</div><div>...<br></div><div>make[2]:=
Entering directory '/home/ooosssmmm/rev-projects/yaml/libyaml/tests=
9;<br>PASS: test-version<br>PASS: test-reader<br>=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>Testsuite summary=
for yaml 0.2.5<br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D<br># TOTAL: 2<br># PASS: =C2=A02<br># SKIP: =C2=A0=
0<br># XFAIL: 0<br># FAIL: =C2=A00<br># XPASS: 0<br># ERROR: 0<br>=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>ma=
ke[2]: Leaving directory '/home/ooosssmmm/rev-projects/yaml/libyaml/tes=
ts'<br>make[1]: Leaving directory '/home/ooosssmmm/rev-projects/yam=
l/libyaml/tests'<br>git branch --track run-test-suite origin/run-test-s=
uite<br>fatal: A branch named 'run-test-suite' already exists.<br>m=
ake: [Makefile:888: tests/run-test-suite] Error 128 (ignored)<br>git worktr=
ee prune<br>git worktree add tests/run-test-suite run-test-suite<br>Prepari=
ng worktree (checking out 'run-test-suite')<br>HEAD is now at 7b73f=
cf Move testing code into run-test-suite-code branch<br>make -C tests/run-t=
est-suite test<br>make[1]: Entering directory '/home/ooosssmmm/rev-proj=
ects/yaml/libyaml/tests/run-test-suite'<br>git clone <a href=3D"https:/=
/github.com/yaml/yaml-test-suite" target=3D"_blank">https://github.com/yaml=
/yaml-test-suite</a> data --branch=3Ddata<br>Cloning into 'data'...=
<br>remote: Enumerating objects: 16712, done.<br>remote: Counting objects: =
100% (2667/2667), done.<br>remote: Compressing objects: 100% (1313/1313), d=
one.<br>remote: Total 16712 (delta 1663), reused 2067 (delta 1160), pack-re=
used 14045<br>Receiving objects: 100% (16712/16712), 4.84 MiB | 2.24 MiB/s,=
done.<br>Resolving deltas: 100% (10668/10668), done.<br>(cd data &&=
; git reset --hard daf799e56372aa93c324d111c73b47a431d7e03b)<br>fatal: Coul=
d not parse object 'daf799e56372aa93c324d111c73b47a431d7e03b'.<br>m=
ake[1]: *** [Makefile:25: data] Error 128<br>make[1]: Leaving directory =
9;/home/ooosssmmm/rev-projects/yaml/libyaml/tests/run-test-suite'<br>ma=
ke: *** [Makefile:882: test-suite] Error 2</div></div>
_______________________________________________<br>
Yaml-core mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Yaml-c=
[email protected]</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/yaml-core" rel=3D"n=
oreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/y=
aml-core</a><br>
</blockquote></div></div>
--000000000000e2e23a05d397880b--
--===============3022812636078745531==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--===============3022812636078745531==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Yaml-core mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/yaml-core
--===============3022812636078745531==--