Re: TAP Directives for XPASS and XFAIL tests with lib/tap-driver.sh

On 6/30/25 16:41, Soham wrote:
> Hello,
>
> I recently was using the tap driver provided by automake, and noticed 
> that there currently is no way to declare a test as xfail or xpass 
> apart from marking it as "todo". The test ouput however does produce 
> rows for XPASS and XFAIL, and the existing TRS format already supports 
> both.
>
> Since the TAP specification allows for custom directives [1], I was 
> wondering if there was interest in a patch adding these to 
> `lib/tap-driver.sh`. I would add "XPASS" and "XFAIL" directives, and 
> if the test did not pass/fail as expected, it would result in an ERROR.

Perhaps Autotest uses ERROR differently, but the better way to think 
about it is expected-vs.-unexpected results.  A test marked XFAIL that 
fails is expected to fail, but a test marked XFAIL that passes is a 
surprise XPASS.

A better solution is to add a directive that declares that certain tests 
are expected to fail.  The TAP driver then matches the actual results 
with the expected results.


-- Jacob