Pushing for full HTML5 support
Tidy Project <[email protected]> Mon, 4 Aug 2014 19:50:40 +0200
| Newsgroups | gmane.comp.web.html-tidy.devel |
|---|---|
| Message-ID | <CAPuVx4Niy04KHjJKE=shVGZ0V6w2-H4Pw40hoX5i6s9UbJFVUQ@mail.gmail.com> |
--===============0677752760447303104==
Content-Type: multipart/alternative; boundary=047d7b67606a8ecae004ffd160f0
--047d7b67606a8ecae004ffd160f0
Content-Type: text/plain; charset=UTF-8
Hi Arnaud, Charles, Bjoern,
Back after quite a LONG time being 'happy' with my own 'development' fork
of tidy, which, thanks to Bjoern's patch, plus some of my own fixes,
supported HTML5 quite well, but not completely ;=()
So I found the time to look more fully into completing HTML5 support, so I
started with a clone of https://github.com/w3c/tidy-html5 to my own github
account https://github.com/geoffmcl/tidy-fork ... added a CMakeLists.txt
to use a cmake/MSVC10 build, and pushed all my recent fixes...
I added a few more HTML5 tag which seemed 'missing', maybe deliberately,
not sure...
Developed a html5 test set of about 30+ files, and with some other fixes,
got it all working quite satisfactorily...
I further built a set of about a dozen HTML 4.01 files with tags that have
been removed in 5, and if you configure tidy5 using -doctype html5 tidy5
will flag these as 'warnings'... it has always 'converted' <dir> to <ul>
which is one of the 'removed' tags...
Did a clone and a build in my Ubuntu linux, applying a few more fixes,
including a build/cmake/build-me.sh to make building even easier...
All seemed good ;=)), until I got around to re-running the nearly 230 test
cases... most went well until test 1423252, and the new tidy5 blew up!!!
The test is quite a short 'ugly' one -
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<title>[1423252] missing text node, and font propagation</title>
</head>
<body>
<font color="red">a
<dl>b</dl>
<dt>c
</body>
</html>
It reports a warning:
line 10 column 1 - Warning: discarding unexpected </body>
and does TY_(FreeNode)( doc, node );
But a little later, in GetToken(...) it extracts this SAME node from
lexer->token. But this node HAS BEEN FREED so is now an invalid memory
address...
I really need some help to find a fix for this... like in this case perhaps
adjust the lexer to not have a 'pending' node, or something...
I added some debug code into FreeNode(doc,node), to check if the node being
freed was the same as that in lexer->token, like -
if (node && doc && doc->lexer) {
if (node == doc->lexer->token) {
/* WHAT TO DO NOW???
Are about to 'free' a node that is still available in
lexer->token
tried these but led to other problems... */
// doc->lexer->token = NULL; // TY_(NewNode)(
doc->lexer->allocator, doc->lexer );
}
}
while (node)
{
Node *next = node->next;
// do the freeing of attributes, children, and node, etc...
node = next;
}
This showed me that this freeing of the node that is presently in
lexer->token happens several times, but in the first few cases there is no
problem because the freed node is never extracted from lexer->token and
used anywhere...
Of course running this test on my own tidydev and there is no such
problem... and also there is no problem with the sourceforge 2009 release
code... Both pass ALL tests...
So the point is that while the w3c/tidy-html5 repo is a great start to
HTML5 handling, it has now introduced a bug that is VERY hard to track
down...
It also fails on a second test, 1773932 I think, but have yet to look
deeper into that...
As stated, any help appreciated with this freed node being used later...
I am posting this here where I know people are 'interested' in tidy as I am
not sure anyone is monitoring the w3c/tidy-html5 'issues' page, and nothing
has been commit in the last 2 years, but will also try to post it there...
hmmm, had some trouble doing that in 'issues' but will try again later...
If I can get past this 'bug' I would be prepared to try to continue to push
forward to another release of HTML Tidy (with full HTML5 support), and try
to address some of the pending issues ;=))
Best regards,
Geoff.
PS: This is my 3rd attempt to post a message here... it has been so long
that I had 'forgotten' which email address I should use... I hope it is
good this time ;=))
--047d7b67606a8ecae004ffd160f0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>Hi Arnaud, Charles, Bjoern,</div><div><br></div><div>=
Back after quite a LONG time being 'happy' with my own 'develop=
ment' fork of tidy, which, thanks to Bjoern's patch, plus some of m=
y own fixes, supported HTML5 quite well, but not completely ;=3D()</div>
<div><br></div><div>So I found the time to look more fully into completing =
HTML5 support, so I started with a clone of <a href=3D"https://github.com/w=
3c/tidy-html5">https://github.com/w3c/tidy-html5</a> to my own github accou=
nt <a href=3D"https://github.com/geoffmcl/tidy-fork">https://github.com/geo=
ffmcl/tidy-fork</a> ... added a CMakeLists.txt =C2=A0to use a cmake/MSVC10 =
build, and pushed all my recent fixes...</div>
<div><br></div><div>I added a few more HTML5 tag which seemed 'missing&=
#39;, maybe deliberately, not sure...</div><div><br></div><div>Developed a =
html5 test set of about 30+ files, and with some other fixes, got it all wo=
rking quite =C2=A0 satisfactorily...</div>
<div><br></div><div>I further built a set of about a dozen HTML 4.01 files =
with tags that have been removed in 5, and if you configure tidy5 using -do=
ctype html5 tidy5 will flag these as 'warnings'... it has always &#=
39;converted' <dir> to <ul> which is one of the 'remove=
d' tags...</div>
<div><br></div><div>Did a clone and a build in my Ubuntu linux, applying a =
few more fixes, including a build/cmake/build-me.sh to make building even e=
asier...</div><div><br></div><div>All seemed good ;=3D)), until I got aroun=
d to re-running the nearly 230 test cases... most went well until test 1423=
252, and the new tidy5 blew up!!!</div>
<div><br></div><div>The test is quite a short 'ugly' one -</div><di=
v><!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"></div><=
div><html></div><div><head></div><div><title>[1423252] mi=
ssing text node, and font propagation</title></div>
<div></head></div><div><body></div><div><font color=3D"=
red">a</div><div><dl>b</dl></div><div><dt>c</div>=
<div></body></div><div></html></div><div><br></div><div>It repo=
rts a warning:</div>
<div>line 10 column 1 - Warning: discarding unexpected </body></div><=
div>and does =C2=A0TY_(FreeNode)( doc, node );</div><div><br></div><div>But=
a little later, in GetToken(...) =C2=A0it extracts this SAME node from lex=
er->token. But this node HAS BEEN FREED so is now an invalid memory addr=
ess...</div>
<div><br></div><div>I really need some help to find a fix for this... like =
in this case perhaps adjust the lexer to not have a 'pending' node,=
or something...</div><div><br></div><div>I added some debug code into Free=
Node(doc,node), to check if the node being freed was the same as that in le=
xer->token, like -</div>
<div><br></div><div>=C2=A0 =C2=A0 if (node && doc && doc-&g=
t;lexer) {</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (node =3D=3D doc->le=
xer->token) {</div><div><span class=3D"" style=3D"white-space:pre"> =
</span>/* WHAT TO DO NOW???=C2=A0</div>
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Are about to &=
#39;free' a node that is still available in lexer->token=C2=A0</div>=
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tried th=
ese but led to other problems... */</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0// doc->lexer->token =3D NULL; // TY_(NewNode)( d=
oc->lexer->allocator, doc->lexer );</div>
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 }</div><div>=C2=A0 =C2=A0 }</div><div>=C2=
=A0 =C2=A0 while (node)</div><div>=C2=A0 =C2=A0{</div><div>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0Node *next =3D node->next;</div><div>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0// do the freeing of attributes, children, and node, et=
c...</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0node =3D next;</div>
<div>=C2=A0 =C2=A0}</div><div><br></div><div>This showed me that this freei=
ng of the node that is presently in lexer->token happens several times, =
but in the first few cases there is no problem because the freed node is ne=
ver extracted from lexer->token and used anywhere...</div>
<div><br></div><div>Of course running this test on my own tidydev and there=
is no such problem... and also there is no problem with the sourceforge 20=
09 release code... Both pass ALL tests...</div><div><br></div><div>So the p=
oint is that while the =C2=A0w3c/tidy-html5 repo is a great start to HTML5 =
handling, it has now introduced a bug that is VERY hard to track down...</d=
iv>
<div><br></div><div>It also fails on a second test, 1773932 I think, but ha=
ve yet to look deeper into that...</div><div><br></div><div>As stated, any =
help appreciated with this freed node being used later...</div><div><br>
</div><div>I am posting this here where I know people are 'interested&#=
39; in tidy as I am not sure anyone is monitoring the w3c/tidy-html5 'i=
ssues' page, and nothing has been commit in the last 2 years, but will =
also try to post it there... hmmm, had some trouble doing that in 'issu=
es' but will try again later...</div>
<div><br></div><div>If I can get past this 'bug' I would be prepare=
d to try to continue to push forward to another release of HTML Tidy (with =
full HTML5 support), and try to address some of the pending issues ;=3D))</=
div>
<div><br></div><div>Best regards,</div><div>Geoff.</div><div><br></div><div=
>PS: This is my 3rd attempt to post a message here... it has been so long t=
hat I had 'forgotten' which email address I should use... I hope it=
is good this time ;=3D))</div>
<div><br></div></div>
--047d7b67606a8ecae004ffd160f0--
--===============0677752760447303104==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
--===============0677752760447303104==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Tidy-develop mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tidy-develop
--===============0677752760447303104==--