Fail verbosely when test fails in process_test
Tom de Vries <[email protected]>
| Newsgroups | gmane.comp.sysutils.dejagnu.general |
|---|---|
| Message-ID | <[email protected]> |
[ was: Re: Check if test has reportable output in process_test ] On 06/03/2017 05:59 PM, Tom de Vries wrote: > Hi, > > I found a problem in the dejagnu testsuite, with process_test. > > If there is a problem running a .test file, then process_test is silent > about it. > > This minimal demonstrator patch adds basic reporting if there is a > problem, by checking that at least a single clause in the expect command > triggered. Using this patch we're able to demonstrate that there are > problems in target.test and remote.test. > This patch tries to be verbose about what exactly goes wrong when running a .test file fails. [ As we now can see, what currently goes wrong in both target.test and remote.test, is that load_lib is called, but not available. ] OK? Thanks, - Tom _______________________________________________ DejaGnu mailing list [email protected] https://lists.gnu.org/mailman/listinfo/dejagnu
0001-Fail-verbosely-when-test-fails-in-process_test.patch
(text/x-patch, 3.2 KB)
Fail verbosely when test fails in process_test
Result:
--------------------------------------------------------
Running ../src/testsuite/runtest.all/libs.exp ...
ERROR: ../src/testsuite/runtest.all/target.test failed
../src/testsuite/runtest.all/target.test stdout: ...
ERROR: (DejaGnu) proc "load_lib telnet.exp" does not exist.
=== foobar Summary ===
... done
../src/testsuite/runtest.all/target.test stderr: ...
The error code is TCL LOOKUP COMMAND load_lib
The info on the error is:
invalid command name "load_lib"
while executing
"::tcl_unknown load_lib telnet.exp"
("uplevel" body line 1)
invoked from within
"uplevel 1 ::tcl_unknown $args"
can't read "test_counts(PASS,total)": no such variable
while executing
"set val $test_counts($x,$which)"
(procedure "log_summary" line 59)
invoked from within
"log_summary total"
(procedure "log_and_exit" line 5)
invoked from within
"log_and_exit"
(procedure "::unknown" line 15)
invoked from within
"load_lib "telnet.exp""
(file "../src/testsuite/../lib/remote.exp" line 23)
invoked from within
"source $file"
invoked from within
"if [ file exists $file] {
source $file
} else {
puts "ERROR: $file doesn't exist"
}"
(file "../src/testsuite/runtest.all/target.test" line 31)
... done
ERROR: ../src/testsuite/runtest.all/remote.test failed
../src/testsuite/runtest.all/remote.test stdout: ...
... done
../src/testsuite/runtest.all/remote.test stderr: ...
invalid command name "load_lib"
while executing
"load_lib "telnet.exp""
(file "../src/testsuite/../lib/remote.exp" line 23)
invoked from within
"source $file"
invoked from within
"if [ file exists $file] {
source $file
} else {
puts "ERROR: $file doesn't exist"
}"
(file "../src/testsuite/runtest.all/remote.test" line 17)
... done
--------------------------------------------------------
2017-06-03 Tom de Vries <[email protected]>
* testsuite/runtest.all/libs.exp (process_test): Dump entire output when
there's no reportable output.
---
testsuite/runtest.all/libs.exp | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/testsuite/runtest.all/libs.exp b/testsuite/runtest.all/libs.exp
index 7520e12..b716007 100644
--- a/testsuite/runtest.all/libs.exp
+++ b/testsuite/runtest.all/libs.exp
@@ -31,7 +31,31 @@ proc process_test { test } {
if [file exists $test] {
verbose "Processing test $test" 2
- spawn -open [open "|$EXPECT $test $srcdir $subdir [pwd]" r]
+
+ set res [catch {
+ exec -ignorestderr \
+ $EXPECT $test $srcdir $subdir [pwd] \
+ > OUTPUT 2>ERROR
+ }]
+ if { $res } {
+ perror "$test failed" 0
+
+ set fp [open "OUTPUT" r]
+ set output [read $fp]
+ close $fp
+ puts "$test stdout: ..."
+ puts "$output"
+ puts "... done"
+
+ set fp [open "ERROR" r]
+ set error [read $fp]
+ close $fp
+ puts "$test stderr: ..."
+ puts "$error"
+ puts "... done"
+ }
+
+ spawn -open [open "|cat OUTPUT" r]
expect {
"No such file or directory" {
perror "$test wouldn't run" 0
@@ -68,6 +92,7 @@ proc process_test { test } {
verbose "All Done" 3
}
}
+ file delete OUTPUT ERROR
} else {
perror "$test doesn't exist" 0
}