Re: Excluding FAILs from UNSUPPORTED test cases

Arsen Arsenović wrote:
> Hi,
>
> While working on freestanding libstdc++, we came up with the idea to use 
> a special error message to detect tests that use headers that don't work 
> in freestanding, and mark them as unsupported by emitting 
> ::unsupported:: from libstdc++-dg-test.
>   

I have not yet had the occasion to dive into the dg.exp code, but a 
quick look at the documentation (such as it is) in the header comment 
block suggests that you need code in your  "libstdc++-dg-prune" 
procedure to recognize the unsupported cases and return the 
"::unsupported::freestanding" string.

Digging briefly into the code, the comp_output contents are only checked 
for those tags (at line 756 in lib/dg.exp on current Git master) if the 
${tool}-dg-prune procedure is defined.  Do you have such a procedure?  
If not, try "proc libstdc++-dg-prune { target text } { return text }" 
instead of your workaround below as the simplest option.  I may be able 
to suggest a better solution after looking at the rest of the testsuite.

> However, in combination with dg-warn and dg-error, this does not behave 
> as intended: the errors those expect will not be emitted, and extra 
> errors will be emitted, hence files that should be UNSUPPORTED are 
> counted as FAILs (test for excess errors/test for errors, line NN) as 
> well as UNSUPPORTED.
>
> I have a workaround for this issue:
> diff --git a/libstdc++-v3/testsuite/lib/libstdc++.exp b/libstdc++-v3/
> testsuite/lib/libstdc++.exp
> index f9a165c3d61..526f0fbab73 100644
> --- a/libstdc++-v3/testsuite/lib/libstdc++.exp
> +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp
> @@ -428,6 +428,10 @@ proc libstdc++-dg-test { prog do_what 
> extra_tool_flags } {
>  
>      set unsupported_message [libstdc++_check_unsupported_p 
> $comp_output]
>      if { $unsupported_message != "" } {
> +	upvar dg-messages messages
> +	upvar dg-excess-errors-flag excess-errors-flag
> +	set messages [list]
> +	set excess-errors-flag 1
>  	set comp_output "::unsupported::$unsupported_message"
>      }
>  
>
> ... however, it feels prone to breakage (since it depends on how dg-test 
> is implemented).
>   

Where is the Git repository this patch applies to?  I will take at least 
a brief look at the testsuite.

> Is there a better way to solve this?
>   

Another option might be to add a target selector to the "dg-do" line 
that will match all targets except freestanding.  Unfortunately there is 
no simple way to say "except these" in the pattern match rules used 
here, so a ${tool}-dg-prune procedure is probably your best option, 
based on what I have seen thus far.

> Thanks in advance, and have a most wonderful day,
>   
You are welcome.


-- Jacob