Re: Obfuscation & unit testing

Hi Brad, 

> > This is actually a new argument for something I have talked about
> > before: don't do acceptance tests in NUnit.
> 
> I'm just starting to set up a FitNesse server now, so maybe 
> that will lead me to agree with you, but I'm curious, what 
> leads you to say this? 
> I've used it for that task, with some success.

I've also done it when I had to, with "some" success. It's
just not as good as it could be, IMO, for several reasons.

1) Acceptance/Customer tests are supposed to be understandable
by the customer. NUnit displays a tree of class and method
names, marked with a green (we hope) icon. No matter how clearly
we name the tests, its still just a report of names. For some
shops, I've created html reports of the test results, but again,
it's still just a list of names.

2) The exercise of sitting with a customer to define and
refine a FIT test is so valuable that it the interaction
suddenly became unnecessary due to advances in technology,
we would have to invent another reason to do it.

3) In some domains, tests need to be transparent, in the sense
that given enough time one could take manually verify that the
inputs match the outputs. Even where this is not required, I 
have found that customers tend to like this approach. FIT 
allows me to set up a display in that fashion.

4) Changing to a different tool helps programmers to change
their mindset - which they should do when they are working
on customer tests.

5) Most acceptance-level tests require execution of a continuous
script, in which each step depends on the previous. By design,
NUnit does not support this, although some people have 
created extensions to enable it.

6) Because low-level programmer testing requires a special
environment, platforms like NUnit provide it. There are 
many aspects of that environment which cause the code
being tested to work differently than it does in the 
actual app.(*) The more sophisticated a framework 
like NUnit becomes, the more differences there will be.
Acceptance tests require an environment identical to that
in which the tests will be run.

Charlie

* I know you'll ask me for an example. :-) Here's one:
There are APIs in .NET which only work in the primary
AppDomain. NUnit runs tests in a secondary AppDomain.
Q.E.D.



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/