Re: TAP Directives for XPASS and XFAIL tests with lib/tap-driver.sh

On 7/2/25 13:00, Soham wrote:
> [...]
>
> Well, I forgot to clarify in the original email, but my particular target
> use-case is is "unit tests" written in code, say for a particular 
> function or
> class. In that case it gets cumbersome to have one binary per test 
> (atleast IMO).
> My idea is to add support for a TAP output like so:
>
> [not] ok 5 - Call sys_open with negative file descriptor # XFAIL
>
>
> Where the trailing XFAIL directive would put it in the output for XFAIL,
> and consider the test "passing". If it were to be an "ok" it would 
> count as a
> failing test.
>
>
> [...]
>
> Essentially its encoding the expected-fail information in the TAP 
> message itself
> rather than in the surrounding runner script or harness.
>
This is a good idea---code in the test script can determine if the test 
is expected to fail and mark it accordingly.  Note that expected 
failures should *not* mean an expected error return:  in your example of 
passing invalid arguments to a system call, the test *passes* (TAP "ok") 
when the system returns an error.

XFAIL is for tests that demonstrate a bug that has not yet been fixed or 
that expose a bug in some underlying layer.  The test really does 
fail---the result obtained is *not* what it should be, but that failure 
has come to be expected.


-- Jacob