Re: TestDataProvider Extension Point

Hi Andreas,

Nice summary. A few comments inline... 

> Charlie pointed out to me that there is a issue with the 
> current design of the extension system in some cases. The 
> cause of this is that NUnit
> 2.4 does not organize extension points in a tree structure 
> like Mono.Addin does.

Exactly...
 
> NUnit 2.4 has only one extension point for all test case 
> builders and all test fixture builders use it. On the other 
> hand it does not make sense to build NUnitTestMethods when a 
> CSUnitTestFixture is its parent. 
> But other addins also have this issue. For example 
> NUnitTestFixtureBuilder uses NUnitTestCaseBuilder to create 
> the test cases but NUnitTestCaseBuilder cannot be registered 
> with the TestCaseBuilder extension point to prevent that 
> NUnitTestMethod instances are built for test fixtures other 
> than NUnitTestFixture. 
> Therefore NUnitTestFixtureBuilder has its own instance of 
> TestCaseBuilderCollection containing the 
> NUnitTestCaseBuilder. If you have a look at Charlie's csUnit 
> addin you'll see that CSUnitTestFixtureBuilder registers 
> CSUnitTestCaseBuilder before it creates the test cases and 
> removes it afterwards. This is another workaround for this issue.
> 
> I don't know if there is a solution to this issue with the 
> current architecture of the extension infrastructure. The 
> extension point for test case builders used by 
> NUnitTestFixtureBuilder has to be different from the ones 
> used by other test fixture builders.

The solution that comes to mind is to develop a tree-based extension
structure. I was thinking of doing that, following the model of
mono-develop, sharpdevelop and eclipse, when Lluis announced that
mono.addins was being packaged as a separate assembly, for use
by other programs. It makes sense to use it, leveraging all of
the experience of other projects with it and participate in
improving it, rather than starting from scratch.

> I'm not sure if the same issue applies to the 
> TestDataProvider extension point. It's supposed to be used 
> with test case builders creating parameterized tests and I 
> see no reason why a test data provider which provides test 
> data for a ParameterizedNUnitTestCaseBuilder would work 
> differently when it provides test data for another test case 
> builder. In both cases it would get a MethodInfo object  and 
> return the test data. 

Or a source of data - see below.

> The same applies for parameterized test fixtures. Maybe it 
> makes sense to have two extension points - one for test cases 
> and one for test fixtures.

I see another distinction that seems important: that between data 
that can be provided at load time and data provided at run time.

In many cases, the data is constant and it doesn't matter when
it is created. But some data may be generated fresh for each
test run. In that case, we should only provide the test with
a delegate or interface from which to get the data, not with
the data itself.

Charlie

> Andreas
> 
> 
> --------------------------------------------------------------
> -----------
> This SF.net email is sponsored by: Microsoft Defy all 
> challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> nunit-developer mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-developer
> 



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/