Re: testing

On Fri, Jan 30, 2015 at 10:20:35PM -0500, al davis wrote:
> On Friday 30 January 2015, Felix Salfelder wrote:
> > there's something odd with the new tests from the 'testing'
> > branch.
> .......
> > 
> > apparently, the "resistor" line eats the probes on X1.q1. is
> > this a bug or a feature?
> 
> It's a bug, that I forgot about but really knew about for a long 
> time.

thought so. weird enough, the bug is not present in gnucap-uf (i
would not have noticed otherwise).

to the point. a test that witnesses a bug is valuable and should be
identifiable. a comment might be sufficient. other projects with test
suites permit tests that are flagged 'expected to fail'. this is a great
way to maintain a todo list...
(surprise: this is provided by autotools [1]).

> If you want a number ...  the number I come up with is 86% of 
> the "branches".  The way I got that number is to grep for 
> untested(), grep for {, grep for case, and compare.  It's a 
> little off because there are other uses of {, but this error is 
> less than 1%.

about coverage... you might find gcov/lcov useful. it does not require
code manipulation or manual bookkeeping and the results are much more
detailed. roughly, (from my notes, not recently tested) the following
once created a bunch of interesting .html files for me.

"""
mkdir cov
cd cov
../configure
make -j10 CXXFLAGS="-g -O0 --coverage" CFLAGS="-g -O0 --coverage" check
lcov --directory . -c -o libbash_test.info
lcov --remove libbash_test.info "/usr*" -o libbash_test.info
genhtml -o ../test_coverage -t "libbash test coverage" --num-spaces 8 libbash_test.info
"""

cheers
felix

[1] http://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html