FW: Why are Bad Tests Yellow?

I just noticed that I accidentally took this discussion off-list earlier and
some of you might find it interesting.

Jeff. 

-----Original Message-----
From: Jeff Brown [mailto:[email protected]] 
Sent: Wednesday, February 27, 2008 11:14 PM
To: 'Charlie Poole'
Subject: RE: [nunit-developer] Why are Bad Tests Yellow?

It depends on the reason.
Gallio has four primary status codes: Passed, Failed, Inconclusive, Skipped.

These codes can be further refined down (in arbitrary ways) using
categories.
For example, "error" is a category of Failed.  "pending" and "ignored" are
category of Skipped.

The idea is that frameworks that plug into Gallio can define new categories
to represent more specific outcome variations.  These categories are
displayed in reports.  No advance configuration is required for the creation
of a new category.  So it's fully dynamic.

Jeff.

-----Original Message-----
From: Charlie Poole [mailto:[email protected]]
Sent: Wednesday, February 27, 2008 11:27 AM
To: 'Jeff Brown'
Subject: RE: [nunit-developer] Why are Bad Tests Yellow?

Do you report them as errors or as not run?

Charlie 

> -----Original Message-----
> From: Jeff Brown [mailto:[email protected]]
> Sent: Wednesday, February 27, 2008 11:02 AM
> To: 'Charlie Poole'
> Subject: RE: [nunit-developer] Why are Bad Tests Yellow?
> 
> I use Red for such tests.  I reserve yellow for tests that were 
> inconclusive or skipped.
> 
> Jeff. 
> 
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of 
> Charlie Poole
> Sent: Wednesday, February 27, 2008 9:24 AM
> To: [email protected]
> Subject: [nunit-developer] Why are Bad Tests Yellow?
> 
> Hi All,
> 
> Historically, NUnit marks non-runnable tests as yellow in the Gui. 
> This includes methods with args, methods returning a value, abstract 
> methods and classes and classes with no default constructor.
> 
> If we take Yellow to mean "Not Run" this is logical - the test could 
> not be run because it is ill-formed.
> 
> If we take it to mean "Not quite as bad as red" then it doesn't make 
> much sense - obviously, something else was intended and an "error"
> was made in expressing that intent.
> 
> In NUnitLite, such tests are created in a way that they fail as soon 
> as they are run, with an error message showing why they cannot be run.
> They show up as red when run from the NUnit gui.
> 
> The "Red" approach makes sense to me. We currently use red to show 
> run-time failures and unexpected exceptions. I think we should also 
> use it to show badly formed tests.
> 
> What do you all think?
> 
> Charlie
> 
> 
> 
> --------------------------------------------------------------
> -----------
> This SF.net email is sponsored by: Microsoft Defy all challenges.
> Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> nunit-developer mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-developer
> 
> 



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/