Re: RFE: Allow for execution output to influence test result

DejaGnu already has the requested feature as far as I can tell.

Rainer Orth wrote:
> In the GCC testsuite, there's quite a number of tests (mostly in
> gcc/testsuite/gcc.target/i386, but also on powerpc and x86_64) that
> check if the host CPU supports some ISA extension at runtime and
> currently just exit (0) if that support is missing, producing a PASS
> result.
>
> That's not only highly confusing, but completely against the spirit of
> IEEE 1001.3.
>
> Some of those tests (rather some *-check.h headers that do the actual
> legwork of calling __cpuid or __builtin_cpu_supports) have provisions to
> print either PASSED or SKIPPED depending on the outcome of that test,
> but only in DEBUG mode.
>   

These are problems with the GCC testsuite, not the DejaGnu framework.

> It occured to me that it would be useful and reasonably simple to always
> prints these (or similarly stilyzed) comments and evaluate those from
> DejaGnu (respectively gcc-dg-prune).
>
> While there's already support for this for the compile step of a test
> (matching specific patterns and emitting e.g. ::unsupported::message
> which are then converted into an UNSUPPORTED result), the execution step
> only looks at the execution output to handle dg-output.
>   

Looking at lib/dg.exp:dg-test, the execution step is performed by 
calling ${tool}_load, which is expected to produce a list of two 
elements, STATUS and OUTPUT.  If STATUS is "pass", the framework can 
then further examine OUTPUT for expected execution results.  If STATUS 
is "fail", the framework emits a verbose message reporting the state of 
the Tcl "errorCode" variable.  Otherwise, STATUS is taken as the name of 
a test-reporting procedure and called.

The ${tool}_load (here probably "gcc_load") callback is supplied by the 
testsuite.  All you should need to do is improve those tests to report a 
different status if the required ISA support is not available (maybe 
"exit (42)" instead of "exit (0)"?) and arrange for gcc_load to 
recognize that status and return a list {unsupported ""}, which will 
cause the execution test to be reported as UNSUPPORTED, which should be 
the correct status if it depends on CPU features lacking on the current 
machine.


-- Jacob