Re: Gnucap test

On Thursday 31 October 2002 10:45 pm, [email protected] wrote:
> Hello all,
>
>   First, this is version 0.32
>
>   The script, test, in the test subdirectory contains
> an error on the diff line.  The original diff line was:
>
> diff $4/$ii.out $3/$ii.out >>$3/$2.diffs || echo "****** $ii
> fails ******"
>
> with the suggested invocation line for test:
>
> ./test ../src/O/gnucap "" 0001 ==
>
> the result is a single diff-file called .diffs.
> The following seems to work much better.
>
> diff $4/$ii.out $3/$ii.out >>$3/$ii.diffs || echo "****** $ii
> fails ******"
>
> But, now I get many diff-files of zero length.
> I also get corresponding "fails" messages.
> Has anyone sorted this out.

The test directory is rather brutal, and the one in 0.32 was 
not updated from 0.31.  This is a known bug, known just after 
the posting.

One unfortunate fact about floating point math is that there is 
some variation depending on how it is compiled, what CPU, what 
compiler, etc.  If everything is ok, these differences will be 
small and can be ignored.  You need to look at the diff files 
to decide.

Here are some known causes of differences:

gcc 2.95 vs. 2.96 vs. 3.01 vs. 3.10, etc.
optimization on or off.
Processor - even AMD vs. Intel.

As an example .... what is sin(10*pi) ?

We all know it is zero, but the computer might give you 
something like 3.34223e-54 on one, and 7.23423e-38 on another.  
I hope you can see that this difference can safely be ignored.

The tests are deliberately sensitive enough that these things 
show.

There are changes between 0.31 and 0.32 in the initial steps of 
transient analysis that have a further effect on the results.  
In all known cases, the 0.32 results are as good or better than 
the 0.31 results, even though the published test results show 
failure.  

There is one case, in the test suite, where the step control 
was deliberately set to produce known incorrect answers, but 
0.32 finds it and produces correct answers.

There is another case where the test prints every iteration.  
Since the initial time step is computed different, the initial 
guess is different, so the iteration trajectory is different 
but both produce the same correct end result.

If this is all you find, it is ok.

I have the current correct files, and had them before the 
release.  Due to non-gnucap pressure, they did not get updated 
in the release tarball.