Re: Extension Points in NUnit 2.5

Hi Kelly, 

> >  We'll try to focus this on architecture rather than syntax.
> 
> Both are important... but I will try to focus.

To clarify - syntax is important in the sense that our architecture
should enable implementors using a wide variety of syntaxes. So it
would be very helpful to ask whether a given design would facilitate
creation of some syntax or other. I'd just like to avoid debates about
which competing syntaxes are best, since we aim to enable multiple
syntaxes.
 
> >  Exactly... trading off future usefulness versus effort.
> 
> Getting it done right is, IMHO, better than getting it done quickly.

I don't want to beat this to death, but "getting it done right" implies
NUnit 3.0 for me. For 2.5, the choice is between "getting it done easily"
and not leaving it for later.
 
> >  > In general, this extension point sounds like it might be very  > 
> > useful in terms of declarative programming tests, such as xml  > 
> > (XAML, etc.) testing.
> >
> >  I believe you're thinking of something like an xml file that  
> > expresses the tests declaratively as well. That's a really  
> good idea.
> 
> Precisely.

</snip>
 
> >  However, in 3.0, we planned for an extension that will 
> take  a single 
> > file and construct a test suite from it, as NUnit  
> currently does for 
> > assemblies. That's where your declarative  test 
> specification - if I 
> > guessed right - would fit in.
> >  Other use cases for that, off the top of my head, include...
> >   pNUnit test specifications (which are xml)
> >   static checking of source using a rules file
> >   running junit tests (jar files)
> 
> Right, that sounds like what I was looking for.

Yup, this is where the plugin-based architecture really comes
into its own - much more so than anything we've done so far.
 
> >
> >    [assembly:RequiredExtension("IterativeTestExtension")]
> 
> Oh, that. Right, that's really good.
> 
> >  If it's not available, the whole thing fails.
> 
> Could we make it so that only the tests that depend on the 
> extension fail, and the other tests run to their natural 
> conclusion? I don't like it when one problem keeps the whole 
> things from working. In contracts, I think they call it 
> severability. Just because one part of the contract is judged 
> to be unenforcable, the rest of the contract is still in force.

That would mean you would have to mark each one of them, so it
doesn't seem as if it would be worth the trouble. However, it
might be possible to allow the attribute to apply on individual
tests, just in case someone wanted to use it that way.

> >  The reason this is
> >  of value is that some tests will be silently ignored or even  
> > silently succeed if the extension is not available. 
> However,  we will 
> > still use any extensions, even if they are not declared.
> >  That was the general consensus in the earlier discussion.
> 
> I see. Can we determine the dependence and mark all dependent 
> tests red? Or just put up some overall red output? Even 
> having the summation be red seems like it would be enough if 
> there were feedback about the result.

There's no general way to determine the dependence. How an
extension decides to apply itself is completely opaque to
NUnit and there is no necessity that the test have any
sort of reference to the extension.

We discussed this at some length (somewhere) and I was thinking
that the optional assembly attribute was a pretty good compromise.
It's the same level of error you get if your test references an
assembly that is missing: nothing loads, nothing works.

Charlie

> >  > >  Please let me know what you think of this as a plan - 
> subject  
> > to  > > change as we go, of course.
> >  >
> >  > Isn't that always the case? :-)
> >
> >  It's a good thing for us, of course, because we keep 
> getting smarter. 
> > :-)
> 
> And if not smarter, at least more well informed. :-)
> 
> -Kelly
> 
> --------------------------------------------------------------
> -----------
> This SF.net email is sponsored by: Microsoft Defy all 
> challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> nunit-developer mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-developer
> 



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/