Re: NUnit 2.5: What's In It Now

Hi Kelly, 

> > Would it make  sense to say that NUnit 
> > should "understand" the MS exception?
> >  I don't think so.
> 
> No, probably not. But why impose that level of independence 
> upon our own set of tests? I can see when we are trying to 
> emulate another testing framework that it needs to be kept 
> independent. But within our own Test, IterativeTest, etc. why 
> do we need that same level of independence?

As I see it, having a group of test types that act the same, 
implies the ability to add extensions to a single test type,
aka extensions on extensions. Therefore, 3.0 not 2.5.

Being able to do this sort of thing is exactly why I want
to move to a more sophisticated addin architecture rather
than waste time trying to kluge it using our current one.

> >  I'm pretty sure that MbUnit has precisely the degree of 
> orthogonality  
> > that the authors intended - no more no less. Same for NUnit.
> 
> Their intention apparently was to have more orthogonality 
> than NUnit does.

That's hard to say, since NUnit has never had the data sources
before - orthogonal or not. We agreed that it should be possible
to use any data source with any test in any combination. We now
have that - in 2.5.

> >  There is a difference between providing a facility for orthogonal  
> > application of a feature and requiring it. What you are 
> proposing  is 
> > to require all future extensions to accept your extension and  I 
> > disagree with that philosophically.
> 
> I agree with you now that I understand things better.

OK.
 
> How do I, as a casual user, learn that I can't call 
> Assume.That inside of a Test? I could assume that it would in 
> a particular way, and it wouldn't. How do I discover that 
> fact as a casual user?

The test would give an error due to the unexpected exception.

> When you say 
> > that something should be pushed down  into the core of 
> NUnit, then you 
> > are saying it is common to  NUnit, csUnit and VSTS tests today and 
> > possibly JUnit, xUnit.net  and others in the future. At 
> that level of 
> > generality, there  is commonality, but it's pretty abstract.
> 
> Ok, I get that and agree.

OK.
 
> >  Because mbUnit has never defined extensibility to include 
> "able  to 
> > run tests from other frameworks" they are able to get away  with a 
> > simpler implementation. THere is complexity here, but  it isn't 
> > accidental - it's rooted in the problem being solved.
> 
> So, do we want to create an MbUnit test runner as an 
> extension to NUnit?

I think we want to have something that gives the functionality,
but I'm not sure we want to go as far as a test runner. There
are three levels we could do this at:

1) Have attributes and tests that look like MbUnit's
2) Simulate loading of mbUnit tests, execute their actual 
   asertions and interpret the results.
3) Write some sort of adapter that runs the tests under
   the actual mbUnit runner and interpret the resultss.

(1) is how Andreas' RowTest works - as well as MbUnit's
    emulation of NUnit.
(2) is how my csUnit and (in process) vsts addins work.
(3) is how Gallio works

For now, I'm sticking to (1) and (2) but we may go farther
in NUnit 3.0.

> >  Of course, the test method would be called many times, but  the 
> > Theory isn't the test method. I believe the original  
> implementation 
> > simply runs a Theory and gets a single  result.
> 
> Ok, that's a new idea. I figured you would want to know the 
> specific data that made the Theory fail so that you could do 
> something about it. I'm not sure how to present that if 
> running the Theory is an atomic operation, but it does make 
> me think about things in a potentially different way.

It seems to be how David Saff implemented it and it was my
assumption. The fact that you want more info if it fails
does't work against it. The single test would just return
an array of results.
 
> I would agree that I am pragmatic, and trying to get things 
> done. You are trying to preserve the integrity of NUnit over 
> the long haul.

Actually, I'm doing both at different moments. Perhaps its
not always clear which mode I'm in. 

> These are not entirely compatible ways to look at the world, 
> and that leads to lively discussion. Neither is bad, just 
> different. I do mix up the 2.5 and 3.0 stuff fairly freely, 
> and that causes some confusion. The point though is that if 
> we know where we want 3.0 to go, it can help in figuring out 
> what can and cannot be put into 2.5.

Have you ever been a release manager? It's a job that requires 
you to be very tough about putting things in that may destabilize
the release. That's where we are now. We have a release close
to going beta, which must go out this month or wait till
mid-summer, due to my schedule. Which do you pick? :-)

> >  > So why not let people throw all the result kinds directly?
> >  > Who knows how useful that could end up being? I don't.
> >
> >  I already said I was implementing  the ability to set the  test 
> > result to success and probably inconclusive. We don't  want to 
> > encourage people to throw exceptions, however,  because that's an 
> > implementation detail that could change.
> 
> Ok. I'd be interested to hear the alternative you may have in mind.

"Reporting test results without exceptions" has been on the work plan
for a while. If I knew how to do it the workplan would say something
specific. I'm not above putting something on a plan that I have not
yet figured out how to do.

Charlie

> -Kelly
> 
> --------------------------------------------------------------
> -----------
> This SF.net email is sponsored by the 2008 JavaOne(SM) 
> Conference Don't miss this year's exciting event. There's 
> still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java
> .sun.com/javaone
> _______________________________________________
> nunit-developer mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-developer
> 



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone