Re: Data Driven Addins

Hi Kelly, 

> > Another issue to bear in mind is that there's more to it than the 
> > underlying implementation similarities. What I mean is...
> > even if it's /possible/ to solve very different sorts of 
> problems with 
> > a common implementation, that commonality may not be 
> obvious to users, 
> > who may require a different syntax so they know what they are doing.
> 
> And, I suppose that some users might not want a big solution 
> when a smaller one would do.

That's my experience on the other (user) list and in the forums.
 
> > Examples of what I see as different problems are:
> >
> > 1) I want a shorthand way to specify the same test method 
> with several 
> > different arguments, so I don't have to type it many times.
> 
> So here, you could have a RowTest type approach, or an 
> IterativeTest type approach, or a Theory approach that 
> combines the two into one syntax.

I see this as a RowTest or ParmData or other approach that
actually puts the data in an attribute. See below for where
I see the others fitting.
 
> > 2) I want to run the same test in different contexts - with a 
> > different database, under a different culture, etc.
> 
> This starts to feel like Gary's solutions, where the Fixture 
> is run multiple times within different contexts...

That's one way. Then there are very specific solutions, like
using SetCulture to specify the culture.
 
> > 3) I want to be able to supply an open-ended list of 
> parameters to a 
> > test and run all of them as if they were separate tests.
> 
> I'm not sure exactly what you're getting at here... do you mean a
> func(a,b,c,...) parameter list? If so, I'm not sure how 
> that's helpful.

I see this as what IterativeTest is doing. NUnit cannot know
the number of test cases until it evaluates the iterator.
I'd rather see this happen at run time, but that cannot
be done with the current release of NUnit.
 
> > 4) I want to test multiple implementations of the same 
> interface using 
> > a standard set of tests.
> 
> Yes, that is where Gary's approach is good.

Agreed
 
> > 5) I want to specify general statements about how a method 
> should work 
> > and provide data to test those general statements.
> 
> This is a bit vague to me... can you come up with an example 
> of what you might be thinking?

This is what Theories can do.
 
> > 6) I want to specify general statements about how a method 
> should work 
> > and have the framework generate data sets to test those statements.
> 
> I think this is the Pex approach, or the Agitator approach, 
> right? I guess this is part of the same general set of 
> problems, but unlike the other elements in this list, I think 
> to do that in the best way requires that you analyze the 
> source code. Starts to sound like a pretty big job. And then 
> how do you repeat a failed test run?

I think it's a variant on Theory as well. If you have a way
of stating the conditions for which the test should pass,
then analysis of the code is not required.

Anyway, I was just listing things that we might want to do,
not worrying about how hard they would be. :-)
> 
> > Add your own...
> 
> I think that about covers everything we've been talking about...
> 
> > Many of the above can be solved by a data-driven 
> implementation, but 
> > I'm not convinced that it serves the user best to make them 
> all /look 
> > like/ the same thing.
> 
> NUnit's style is to make one thing do lots. Assert does lots.
> TestFixture does lots. If we added Assume, that would do just 
> about the same amount of work as Assert. I would try to 
> persuade the group not to create a bunch of different new 
> things, but to see if one new thing would cover most or all 
> of what we want. Maybe we could even do some of this stuff by 
> just adding new parameters to the Test and TextFixture 
> attributes, then we don't have to make anything "new" at all. 
> That's the style of NUnit as I have seen it. Changing the 
> style of something as successful as NUnit seems risky to me.

Assert is a façade and it's not the only façade. We have
CollectionAssert, StringAssert, FileAssert as well. We have
the Is/Has syntax helpers and the AssertionHelper.

TestFixture OTOH does one and only one thing that I know of.
Maybe I'm missing something - what did you mean?

Charlie

> -Kelly
> 
> --------------------------------------------------------------
> -----------
> This SF.net email is sponsored by: Microsoft Defy all 
> challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> nunit-developer mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-developer
> 



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/