Pushing for full HTML5 support

--===============6512398554305029893==
Content-Type: multipart/alternative; boundary=047d7b5dbac0bcb52c04ffe6228c

--047d7b5dbac0bcb52c04ffe6228c
Content-Type: text/plain; charset=UTF-8

Hi,

What a difference a little 'i' can make ;=)) In this case from full
functionality to a CRASH.

I needed to add some diagnostic debug output to check exactly what tidy was
doing, so...

Spot the error ;=))

The original -
        if (lexer->token->type != TextNode
            || !(lexer->insert || lexer->inode))
            return lexer->token;
        return lexer->itoken = TY_(InsertedToken)( doc );

To -
        if (lexer->token->type != TextNode
            || !(lexer->insert || lexer->inode)) {
            node =  lexer->token;
            GTDBG(doc,"lex-token",node);
            return node;
        }
        lexer->itoken = TY_(InsertedToken)( doc );
        node = lexer->token;  /* LOOK HARD HERE ;=)) */
        GTDBG(doc,"lex-inserted",node);
        return node;

And at that particular moment lexer->token contained a node that had been
FREED, thus POOF!!!

Of course in the release GTDBG() macro compiles to nothing... It is really
ONLY for MSVC Debug mode...

So am back on track ;=)) and the test cases run all some 230 without a
problem.

I wanted to do a compare of the test cases when using the 2009 release, and
this tidy5, but that proved quite difficult. First it is very difficult to
compare the output message text files, since in tidy5 the 'information'
strings have been changed/enhanced, so that means they are ALL different.

And comparing the tidied html output can also be quite difficult, since
there are some indented 'space' changes. Well ok, a lot of that can be
avoided by using the diff -w switch, but there is still the question of why
was the indented 'space' changed?

Also it is made difficult on tests where there is no doctype was given on
the input. It seems tidy5 will always 'prefer' the simple HTML5 <!DOCTPE
html>, rather than the 4.01 or 4.01 Transitional used by Tidy 2009...

But still of the 220 compares of the tidied HTML 103 were the SAME, at
least when using the -w to ignore space differences...

And of the 117 different, about 56 were just different in the doctype as
mentioned, and a good many others were only indent space difference, but
still to analyse and understand the others, and decide whether it is a good
or bad difference...

And still to look at the 4 or so missing cases, where tidy5 output a file
while tidy 2009 did not. Or is that reversed?

It would be nice if others could do a clone of my -
 https://github.com/geoffmcl/tidy-fork
compile it, and provide some feed back...

First I hope people will use, and LIKE the cmake build system. There are
some cmake enhancements that could be added if needed, desired...

In unix/linux systems cmake supports, can generate Unix Makefiles, Ninja,
CodeBlocks, Eclipse, Kdevelop3, Sublime builds...

In windows it supports ALL versions of MSVC from 6 up to 12+, and where
available both 32-bit and 64-bit, plus NMake Makefiles, Watcom, MSYS,
MingGW, as well as the previously mentioned unix build systems...

All through one easily maintainable CMakeLists.txt file. If this can be
generally agreed then the multiple other build systems can be removed.

Anyway, in general it seems things are LOOKING TIDY ;=))

Regards,
Geoff.

--047d7b5dbac0bcb52c04ffe6228c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi,=C2=A0</div><div><br></div><div>What a difference =
a little &#39;i&#39; can make ;=3D)) In this case from full functionality t=
o a CRASH.</div><div><br></div><div>I needed to add some diagnostic debug o=
utput to check exactly what tidy was doing, so...</div>
<div><br></div><div>Spot the error ;=3D))</div><div><br></div><div>The orig=
inal -</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (lexer-&gt;token-&gt;type !=
=3D TextNode</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 || !(lexer=
-&gt;insert || lexer-&gt;inode))</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 return lexer-&gt;token;</div>
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 return lexer-&gt;itoken =3D TY_(InsertedTo=
ken)( doc );</div><div><br></div><div>To -</div><div>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 if (lexer-&gt;token-&gt;type !=3D TextNode</div><div>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 || !(lexer-&gt;insert || lexer-&gt;inode)) {</d=
iv>
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 node =3D =C2=A0lexer-&gt;tok=
en;</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 GTDBG(doc,&quot;lex=
-token&quot;,node);</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ret=
urn node;</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 }</div><div>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 lexer-&gt;itoken =3D TY_(InsertedToken)( doc );</div>
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 node =3D lexer-&gt;token; =C2=A0/* LOOK HA=
RD HERE ;=3D)) */</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 GTDBG(doc,&quot;lex=
-inserted&quot;,node);</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 return node;</=
div><div><br></div><div>And at that particular moment lexer-&gt;token conta=
ined a node that had been FREED, thus POOF!!!</div>
<div>=C2=A0 =C2=A0 =C2=A0=C2=A0</div><div>Of course in the release GTDBG() =
macro compiles to nothing... It is really ONLY for MSVC Debug mode...</div>=
<div><br></div><div>So am back on track ;=3D)) and the test cases run all s=
ome 230 without a problem.</div>
<div><br></div><div>I wanted to do a compare of the test cases when using t=
he 2009 release, and this tidy5, but that proved quite difficult. First it =
is very difficult to compare the output message text files, since in tidy5 =
the &#39;information&#39; strings have been changed/enhanced, so that means=
 they are ALL different.</div>
<div><br></div><div>And comparing the tidied html output can also be quite =
difficult, since there are some indented &#39;space&#39; changes. Well ok, =
a lot of that can be avoided by using the diff -w switch, but there is stil=
l the question of why was the indented &#39;space&#39; changed?</div>
<div><br></div><div>Also it is made difficult on tests where there is no do=
ctype was given on the input. It seems tidy5 will always &#39;prefer&#39; t=
he simple HTML5 &lt;!DOCTPE html&gt;, rather than the 4.01 or 4.01 Transiti=
onal used by Tidy 2009...</div>
<div><br></div><div>But still of the 220 compares of the tidied HTML 103 we=
re the SAME, at least when using the -w to ignore space differences...</div=
><div><br></div><div>And of the 117 different, about 56 were just different=
 in the doctype as mentioned, and a good many others were only indent space=
 difference, but still to analyse and understand the others, and decide whe=
ther it is a good or bad difference...=C2=A0</div>
<div><br></div><div>And still to look at the 4 or so missing cases, where t=
idy5 output a file while tidy 2009 did not. Or is that reversed?</div><div>=
<br></div><div>It would be nice if others could do a clone of my -</div>
<div>=C2=A0<a href=3D"https://github.com/geoffmcl/tidy-fork">https://github=
.com/geoffmcl/tidy-fork</a></div><div>compile it, and provide some feed bac=
k...</div><div><br></div><div>First I hope people will use, and LIKE the cm=
ake build system. There are some cmake enhancements that could be added if =
needed, desired...=C2=A0</div>
<div><br></div><div>In unix/linux systems cmake supports, can generate Unix=
 Makefiles, Ninja, CodeBlocks, Eclipse, Kdevelop3, Sublime builds...=C2=A0<=
/div><div><br></div><div>In windows it supports ALL versions of MSVC from 6=
 up to 12+, and where available both 32-bit and 64-bit, plus NMake Makefile=
s, Watcom, MSYS, MingGW, as well as the previously mentioned unix build sys=
tems...=C2=A0</div>
<div><br></div><div>All through one easily maintainable CMakeLists.txt file=
. If this can be generally agreed then the multiple other build systems can=
 be removed. =C2=A0</div><div><br></div><div>Anyway, in general it seems th=
ings are LOOKING TIDY ;=3D))</div>
<div><br></div><div>Regards,</div><div>Geoff.</div><div><br></div></div>

--047d7b5dbac0bcb52c04ffe6228c--


--===============6512398554305029893==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
--===============6512398554305029893==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tidy-develop mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tidy-develop

--===============6512398554305029893==--