Re: Test suite

On Fri, May 06, 2005 at 02:07:55PM -0400, David Malcolm wrote:
> On Fri, 2005-05-06 at 10:03 +0200, Geert Stappers wrote:

 [ fixed crasher file back into CVS ]

> > > Are there any other tests for old crashers we should add into CVS?
> > > 
> > > Should we ship these files in the tarball?
> > 
> > My first thought: no.
> > 
> > On second thought:
> > 
> > It can be done. What is needed is an explanation why we ship it.
> > Currently is this the content of examples/crash/README
> > 
> >  Files that cause conglomerate to crash.
> 
> Maybe should read:
> "This is a regression test suite for Conglomerate.  The files here
> trigger bugs in Conglomerate.  The aim is that Conglomerate should be
> able to load all of these files successfully."
> 
--- README      15 Sep 2003 14:16:11 -0000      1.1
+++ README      6 May 2005 20:05:08 -0000
@@ -1,4 +1,7 @@

- Files that cause conglomerate to crash.
+ This is a regression test suite for Conglomerate.
+ The files here trigger bugs in Conglomerate.
+ The aim is that Conglomerate should be able
+ to load all of these files successfully.

  It is here for easy reproduction and should _not_ go into distribution.

> > 
> >  It is here for easy reproduction and should _not_ go into distribution.
> 
> I think the easiest way to do that is to keep them in CVS but not have
> them in the tarball.

Done
(a long time ago :-p )

> > > Anyone got thoughts on automating the process?  A shell script that
> > > looped through all the test files and launched Conglomerate on each in
> > > turn and somehow reported if Conglomerate crashed on any would be cool!
> > 
> > I made this
> 
> This looks great!  Thanks!  Can we get the script in CVS please?

Euh, it allready is ...

> Ideally we'd have this set up so it gets run if you do a "make test"; it
> would run the built conglomerate executable upon all of the test cases.

mmmm, that is a tricky request

examples/crash is not in the tarball,
so the toplevel Makefile will complain about it when it is missing.

request is IMHO too hard to honour

> 
> > <screenshot>
 [ tester.batch ]
> > </screenshot>
> > 
> > It produces this output
> > 
> > <screenshot>
> > stappers@bahrain:/usr/src/conglomerate/examples/crash
> > $ ./tester.batch
> > ./tester.batch: line 32:  3803 Terminated  conglomerate $f >$f.log 2>&1
> > ./tester.batch: line 32:  3809 Terminated  conglomerate $f >$f.log 2>&1
> > ./tester.batch: line 33:  3815 Terminated  conglomerate $f >$f.log 2>&1
Those are avoided in the new version.

> > ****************
> > rivet.xml.log
> > ****************
> > ** Message: Found display spec based on matching public ID of DTD:
> >     -//OASIS//DTD DocBook XML V4.1.2//EN
> > 
> > ** Message: Trying to load
> > "/usr/share/pixmaps/conglomerate/cong-note-48.png"
> > ** Message: Trying to load
> > "/usr/share/pixmaps/conglomerate/cong-note-48.png"
    ** Message: file now completely load
> > ****************
> > septentrio.xml.log
> > ****************
> > ** Message: coverage of Website Layout = 0.000000
> > ** Message: coverage of TEI Lite = 0.500000
> > ** Message: coverage of Webpage = 0.500000
> > ** Message: coverage of XHTML 1.0 (strict) = 0.000000
> > ** Message: coverage of XSL stylesheet = 0.000000
    ** Message: file now completely load
> > ****************
> > testcase-150368.xml.log
> > ****************
> >     -//OASIS//DTD DocBook XML V4.1.2//EN
> > 
> > 
> > ** ERROR **: file cong-document-traversal.c: line 594
> > (recursive_create_traversal_nodes): assertion failed:
> > (should_have_traversal_node (xml_node))
> > aborting...
> > some real report
> > stappers@bahrain:/usr/src/conglomerate/examples/crash
> > </screenshot>
> 

So what I like to see, is the addtion of this line of code
    g_message ("file now completely load\n");
at the right place.


Cheers
Geert Stappers

_______________________________________________
Conglomerate-devel mailing list
[email protected]
http://lists.copyleft.no/mailman/listinfo/conglomerate-devel

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFCe9NWOSINbgwa/7sRAlN5AJ9yF3G8/lEDm5coTuYURpdJoYk0MACguttR
uDdhlYEFYAIO4TP5sqiQXIo=
=VjX9
-----END PGP SIGNATURE-----