Re: NUnit 2.5: What's In It Now

On Wed, Apr 9, 2008 at 7:28 PM, Charlie Poole
<[email protected]> wrote:
> Hi Kelly,
>
>  As I see it, having a group of test types that act the same,
>  implies the ability to add extensions to a single test type,
>  aka extensions on extensions. Therefore, 3.0 not 2.5.
>
>  Being able to do this sort of thing is exactly why I want
>  to move to a more sophisticated addin architecture rather
>  than waste time trying to kluge it using our current one.

Ok, I think I see your point now.

>  > >  I'm pretty sure that MbUnit has precisely the degree of
>  > orthogonality
>  > > that the authors intended - no more no less. Same for NUnit.
>  >
>  > Their intention apparently was to have more orthogonality
>  > than NUnit does.
>
>  That's hard to say, since NUnit has never had the data sources
>  before - orthogonal or not. We agreed that it should be possible
>  to use any data source with any test in any combination. We now
>  have that - in 2.5.

The data sources changes the nature of the framework in interesting
ways, for sure. It pushes for orthogonality, but is that always right?
I don't know.

>  > How do I, as a casual user, learn that I can't call
>  > Assume.That inside of a Test? I could assume that it would in
>  > a particular way, and it wouldn't. How do I discover that
>  > fact as a casual user?
>
>  The test would give an error due to the unexpected exception.

That would help, IF the error were specific enough... something like

"Assume is not supported within [Test], it is for use with [Theory] only."

Or something to that extent. I don't like the dependence of Test on
Theory even in a message, but how else do you get them on the right
track quickly?

>  > So, do we want to create an MbUnit test runner as an
>  > extension to NUnit?
>
>  I think we want to have something that gives the functionality,
>  but I'm not sure we want to go as far as a test runner.

If we're doing it for csUnit and MSTest, why would MbUnit be a second
class citizen? Just curious. Is it because it's hard, or confusing, or
what?

>  There
>  are three levels we could do this at:
>
>  1) Have attributes and tests that look like MbUnit's
>  2) Simulate loading of mbUnit tests, execute their actual
>    asertions and interpret the results.
>  3) Write some sort of adapter that runs the tests under
>    the actual mbUnit runner and interpret the resultss.

I was thinking along the lines of 3.

>  (1) is how Andreas' RowTest works - as well as MbUnit's
>     emulation of NUnit.

Kind of weird to have NUnit run MbUnit to simulate running NUnit... :-)

>  (2) is how my csUnit and (in process) vsts addins work.
>  (3) is how Gallio works
>
>  For now, I'm sticking to (1) and (2) but we may go farther
>  in NUnit 3.0.

I wouldn't expect anything on this line until 3.0. Adding some
data-driven stuff in 2.5 is really good, but not emulation of MbUnit.

>  > Ok, that's a new idea. I figured you would want to know the
>  > specific data that made the Theory fail so that you could do
>  > something about it. I'm not sure how to present that if
>  > running the Theory is an atomic operation, but it does make
>  > me think about things in a potentially different way.
>
>  It seems to be how David Saff implemented it and it was my
>  assumption. The fact that you want more info if it fails
>  does't work against it. The single test would just return
>  an array of results.

Yes, it's just a bit different than I was thinking.

>  > I would agree that I am pragmatic, and trying to get things
>  > done. You are trying to preserve the integrity of NUnit over
>  > the long haul.
>
>  Actually, I'm doing both at different moments. Perhaps its
>  not always clear which mode I'm in.

Sometimes. :-)

>  > These are not entirely compatible ways to look at the world,
>  > and that leads to lively discussion. Neither is bad, just
>  > different. I do mix up the 2.5 and 3.0 stuff fairly freely,
>  > and that causes some confusion. The point though is that if
>  > we know where we want 3.0 to go, it can help in figuring out
>  > what can and cannot be put into 2.5.
>
>  Have you ever been a release manager? It's a job that requires
>  you to be very tough about putting things in that may destabilize
>  the release. That's where we are now. We have a release close
>  to going beta, which must go out this month or wait till
>  mid-summer, due to my schedule. Which do you pick? :-)

Let's get it out. :-)   Then Gary and I can play around with Theory in
Addin land.

>  > Ok. I'd be interested to hear the alternative you may have in mind.
>
>  "Reporting test results without exceptions" has been on the work plan
>  for a while. If I knew how to do it the workplan would say something
>  specific. I'm not above putting something on a plan that I have not
>  yet figured out how to do.

Fair enough.

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone