Re: Excluding FAILs from UNSUPPORTED test cases

Arsen Arsenović <[email protected]> Fri, 23 Sep 2022 11:42:11 +0200
Newsgroups	gmane.comp.sysutils.dejagnu.general
Message-ID	<12990700.Ys9jVhoPhe@bstg>
Hi,

On Friday, 23 September 2022 05:33:42 CEST Jacob Bachmeyer wrote:
> I have not yet had the occasion to dive into the dg.exp code, but a
> quick look at the documentation (such as it is) in the header comment
> block suggests that you need code in your  "libstdc++-dg-prune"
> procedure to recognize the unsupported cases and return the
> "::unsupported::freestanding" string.
Jonathan implemented this solution in the meanwhile, it did work as 
expected, though, it also produced more PASSes than anticipated, by 
going from:
FAIL: 30_threads/promise/cons/assign_neg.cc  (test for errors, line 29)
UNSUPPORTED: 30_threads/promise/cons/assign_neg.cc: hosted C++ headers 
not supported

... to:
PASS: 30_threads/promise/cons/assign_neg.cc  (test for errors, line 29)
UNSUPPORTED: 30_threads/promise/cons/assign_neg.cc: hosted C++ headers 
not supported

Which is (IMO) good enough, though, I can't quite tell why that
happens.

> Digging briefly into the code, the comp_output contents are only
> checked for those tags (at line 756 in lib/dg.exp on current Git
> master) if the ${tool}-dg-prune procedure is defined.  Do you have
> such a procedure? If not, try "proc libstdc++-dg-prune { target text
> } { return text }" instead of your workaround below as the simplest
> option.  I may be able to suggest a better solution after looking at
> the rest of the testsuite.
This was defined, but the ::unsupported:: string was being returned from 
${tool}-dg-test rather than ${tool}-dg-prune, changing that to be 
returned from ${tool}-dg-prune had the effect described above.

> Where is the Git repository this patch applies to?  I will take at
> least a brief look at the testsuite.
The diff is a subset of this commit:
https://github.com/ArsenArsen/gcc/commit/bd18c507663727d6961c37e23a2be86a4c47e927

> Another option might be to add a target selector to the "dg-do" line
> that will match all targets except freestanding.  Unfortunately there
> is no simple way to say "except these" in the pattern match rules
> used here, so a ${tool}-dg-prune procedure is probably your best
> option, based on what I have seen thus far.
Indeed, that is what we wanted to avoid.

Thanks!
-- 
Arsen Arsenović
signature.asc (application/pgp-signature, 358 B)