Re: Problem with spuriously failing gcc tests with dg-output.

On 2 June 2017 at 11:35, Georg-Johann Lay <[email protected]> wrote:
> Hi, when I am running gcc test suite
>
> $ make check-gcc RUNTESTFLAGS='-all ubsan.exp=float-cast-overflow-2.c'
>
>                 === gcc tests ===
>
> Schedule of variations:
>     unix
>
> Running target unix
> Using /usr/share/dejagnu/baseboards/unix.exp as board description file for
> target.
> Using /usr/share/dejagnu/config/unix.exp as generic interface file for
> target.
> Using /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/config/default.exp as
> tool-and-target-specific interface file.
> Running
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/gcc.dg/ubsan/ubsan.exp ...
> PASS: c-c++-common/ubsan/float-cast-overflow-2.c   -O0  (test for excess
> errors)
> PASS: c-c++-common/ubsan/float-cast-overflow-2.c   -O0  execution test
> FAIL: c-c++-common/ubsan/float-cast-overflow-2.c   -O0  output pattern test
> PASS: c-c++-common/ubsan/float-cast-overflow-2.c   -O1  (test for
> ...
>
> I am getting spurious FAILs.  Rerunning the same tests might come up with
> more or less FAILs / PASSes.
>
> The tests have "lots" of dg-output directives
>
> $ cat gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c | tail -n 3
> /* { dg-output "\[^\n\r]*runtime error: value -?nan is outside the range of
> representable values of type '__int128 unsigned'\[^\n\r]*(\n|\r\n|\r)" } */
> /* { dg-output "\[^\n\r]*runtime error: value inf is outside the range of
> representable values of type '__int128 unsigned'\[^\n\r]*(\n|\r\n|\r)" } */
> /* { dg-output "\[^\n\r]*runtime error: value -inf is outside the range of
> representable values of type '__int128 unsigned'" } */
>
> https://gcc.gnu.org/viewcvs/gcc/trunk/gcc/testsuite/c-c%2B%2B-common/ubsan/float-cast-overflow-2.c?revision=212755&view=markup#l1
>
> In the according log file, some of the "PASS..." output is messed up with
> the output from dg-output, making the test fail:
>
> $ cat testsuite/gcc/gcc.log
> PASS: c-c++-common/ubsan/float-cast-overflow-2.c   -O0  (test for excess
> errors)
> Setting LD_LIBRARY_PATH to : [cut]
> spawn [open ...]
>
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c:20:3:
> runtime error: value 1.70141e+38 is outside the range of representable
> values of type '__int128'
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c:20:3:
> runtime error: value 1.70141e+38 is outside the range of representable
> values of type '__int128'
> [...]
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c:25:3:
> runtime error: value 3.40282e+38 is outside the range of representable
> values of type '__int128 unsigned'
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c:25:3:
> runtime error: value 3.40282e+38 is outside the range of representable
> values of type '__int128 unsigned'
> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-2.c:25:3:
> runtime error: PASS: c-c++-common/ubsan/float-cast-overflow-2.c   -O0
> execution test
>
> BUMMER! The PASS output comes at the wrong place and messes up the dg-output
>
> FAIL: c-c++-common/ubsan/float-cast-overflow-2.c   -O0  output pattern test
> Output was:
> c-c++-common/ubsan/float-cast-overflow-2.c:20:3: runtime error: value
> 1.70141e+38 is outside the range of representable values of type '__int128'
> c-c++-common/ubsan/float-cast-overflow-2.c:20:3: runtime error: value
> 1.70141e+38 is outside the range of representable values of type '__int128'
> [...]
> c-c++-common/ubsan/float-cast-overflow-2.c:25:3: runtime error: value
> 3.40282e+38 is outside the range of representable values of type '__int128
> unsigned'
> c-c++-common/ubsan/float-cast-overflow-2.c:25:3: runtime error: value
> 3.40282e+38 is outside the range of representable values of type '__int128
> unsigned'
> c-c++-common/ubsan/float-cast-overflow-2.c:25:3: runtime error:
> Should match:
>
> Output is missing after "runtime error: " which is the place where the
> "PASS" message was emit.
>
> Any ideas what can cause such fails?  It's on a 8-core x86_64 machine, but
> the testsuite is not started in parallel mode.
>
> $ runtest -version
> Expect version is       5.45
> Tcl version is          8.6
> Framework version is    1.5.3
>
> Does this ring a bell somewhere?  What could shred the output?
>

Maybe it's similar to what we discussed in:
https://lists.gnu.org/archive/html/dejagnu/2016-03/msg00034.html

Christophe

> Thanks in advance,
>
> Johann
>
>
> _______________________________________________
> DejaGnu mailing list
> [email protected]
> https://lists.gnu.org/mailman/listinfo/dejagnu