Re: Extension Points in NUnit 2.5

On Thu, Feb 28, 2008 at 11:48 AM, Charlie Poole
<[email protected]> wrote:
>  > I don't have a strong opinion on this... would it change the
>  > way current things work? What are the trade offs, pro and con
>  > for putting them all together? Would it help in creating new variants?
>
>  Pro: It would be easier to create an extension that can work in
>      multiple contexts - like on a type and on a method.
>
>  Con: Most such extensions would likely be decorators or some other
>      modifier for tests, not test builders.
>
>  Con: If it added more methods, every extenson would have to implemet
>      them, even if they just threw an NYI exception, and would have
>      to deal with further tests in their CanBuildFrom method.
>
>  Con: The same thing can be accomplished with a single class that
>      implements several interfaces.
>
>  Con: More extensions would have to be examined for each type and
>      method found in a test assembly.
>
>  I think I talked myself out of it. :-)

Sometimes, I get lucky and ask the right question. :-)

>  > >  TestDataProvider - A new extension point to encapsulate  a
>  >
>  > Can we please have this discussion out on the list...
>
>  Yes... Andreas has volunteered to work in this area and is drafting
>  up some ideas for the list. I'll throw my ideas in at the same time
>  as I'm sure you and others will.
>
>  We'll try to focus this on architecture rather than syntax.

Both are important... but I will try to focus.

>  > >  ProjectConverter - An extension point for addins that know  how to
>  > >
>  > >   My Conclusion: See how much work it is.
>  >
>  > If you looked at it as a prototype for 3.0, and thought about
>  > what we might learn about the problem domain, that might tip
>  > the decision point to some extent.
>
>  Exactly... trading off future usefulness versus effort.

Getting it done right is, IMHO, better than getting it done quickly.

>  > In general, this extension point sounds like it might be very
>  > useful in terms of declarative programming tests, such as xml
>  > (XAML, etc.) testing.
>
>  I believe you're thinking of something like an xml file that
>  expresses the tests declaratively as well. That's a really
>  good idea.

Precisely.

>  However, it's not what this is for. It's only to construct
>  NUnit projects, which are essentially lists of files. At this
>  time, those files are all assemblies.

Ah. Too bad.

>  However, in 3.0, we planned for an extension that will take
>  a single file and construct a test suite from it, as NUnit
>  currently does for assemblies. That's where your declarative
>  test specification - if I guessed right - would fit in.
>  Other use cases for that, off the top of my head, include...
>   pNUnit test specifications (which are xml)
>   static checking of source using a rules file
>   running junit tests (jar files)

Right, that sounds like what I was looking for.

>  > >  GENERAL:
>  > >
>  > >   I'd like to implement the feature whereby addins needed
>  > >   by a test are declared using an assembly attribute. If
>  > >   there is no declaration, things work just like now. If
>  > >   an addin is declared but is not installed, the entire
>  > >   test fails.
>  >
>  > I'm not sure what this means exactly... but it doesn't sound bad. :-)
>
>  We had some discussion about this on one of the lists - I'm not
>  sure which. Essentially, you can optionally provide a sort of
>  manifest of addins needed to run a test. For example...
>
>    [assembly:RequiredExtension("IterativeTestExtension")]

Oh, that. Right, that's really good.

>  If it's not available, the whole thing fails.

Could we make it so that only the tests that depend on the extension
fail, and the other tests run to their natural conclusion? I don't
like it when one problem keeps the whole things from working. In
contracts, I think they call it severability. Just because one part of
the contract is judged to be unenforcable, the rest of the contract is
still in force.

>  The reason this is
>  of value is that some tests will be silently ignored or even
>  silently succeed if the extension is not available. However,
>  we will still use any extensions, even if they are not declared.
>  That was the general consensus in the earlier discussion.

I see. Can we determine the dependence and mark all dependent tests
red? Or just put up some overall red output? Even having the summation
be red seems like it would be enough if there were feedback about the
result.

>  > >  Please let me know what you think of this as a plan - subject  to
>  > > change as we go, of course.
>  >
>  > Isn't that always the case? :-)
>
>  It's a good thing for us, of course, because we keep getting smarter. :-)

And if not smarter, at least more well informed. :-)

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/