Re: NUnit 2.5: What's In It Now

On Mon, Apr 7, 2008 at 6:43 AM, Gary Evans <[email protected]> wrote:
>  Hi Kelly,
>  >
> > > 2) As I mentionned in the earlier note, it's not hard
> > > for someone to implement Theory as an extension using
> > > the existing extension points. Gary has done that, in fact.
> >
> > I'm not really talking about Theory, I'm talking about Assume. It's
> > great what Gary has done, but without Assume, it's only half the
> > story.
>
>  My extension has assumes. My syntax is a bit worse than Assume.That, but
> the functionality's there. It would take you about 5 minutes to make the
> changes that you recently proposed to the list, but to do it in my
> extension; it's just a case of putting the changes in TheoryMethod instead
> of NUnitTestMethod.

I have no problem with you putting Assume.That into your extension. If
you need assistance with it let me know, but I think it should be
pretty easy to do. If you need, I can email you another copy or
whatever.

>  My implementation differs a little bit - your assume was simply a
> filtering, but mine catches cases where none of the assumptions were
> exercised etc. Nevertheless, it'd still be simple to change it to use
> Assume.That and catch an AssumptionException, but all in the extension, and
> you could change the behaviour to your preference.

I think it's important to know when ALL runs of a Theory resulted in
AssumptionExceptions... how does it show up in the UI?

>  I know you were asking Charlie the question, but I thought I'd chip in my
> 2p on this (sorry for the diversion).

No problem. This is a mailing list, not a private conversation!! :-)

> I was a bit lazy when I wrote my prototype Theory extension, my assumptions
> were just a separate method that was called prior to the theory being
> executed, but but I could have written them to work using the Assume.That()
> syntax the same as you recently posted to the list. Your change was to be in
> the NUnitTestMethod (I think, I'm working off the top of my head) to catch
> the AssumptionException (maybe not correct name again).

I think Assume.That works better than what you're describing in that
it keeps the Assumptions closer to the Theory, increasing readability.
My opinion is that readability is one of the more important
contributions Theory makes, so this is important imho.

>  However, We've all derived from NUnitTestMethod in our paramaterized test
> extensions, to handle the case of having a test with data, in my theory
> extension I called it TheoryMethod (I think). and that would be the ideal
> place to put the Assume functionality? i.e. it doesn't need to be in the
> base class.

Ok.

>  I think how you've written the Assume.That() and the exception handling is
> good, but I think that we don't want Assumes to live in NUnitTestMethod -
> does it make sense for a user to write an Assume in a vanilla unit test
> (i.e. not even a RowTest or a DataSource test?).

I don't know what you mean here... Assume and Assert are an orthogonal
concern to different Test types. I might be missing something as my
brain is a bit hammered today (dang ole day job anyway).

> I haven't looked at the 2.5
> sources yet, but I'm assuming that there is a derived class that the RowTest
> and DataSource creates, ParameterizedTestMethod? Maybe that class, or a
> Theory class deriving from that would be the better place to put this - but
> this can be shipped with the Theory extension, and doesn't have to be in the
> core/framework of NUnit, unless there's some reporting mechanism missing
> (i.e. if we wanted a different way to report assumption failures, if we
> bothered putting that functionality in).

If you only want Assume to be caught in Theory, then maybe you can
separate it into the separate kind of test. If you want to maintain
orthogonality, then Assume catching has to go into the core. (if I
understand things right). It's the catching of the AssumeException
that needs a new extension point, not the throwing of it.
Specifically, in ProcessException:

   protected internal virtual void ProcessException(Exception
exception, TestCaseResult testResult)
               {
                     if (IsAssumeException(exception))
                     {
                               // TODO: Perhaps add text to some output stream.
                               testResult.Success();
                               return;
                     }

                       if (!ExceptionExpected)
                       {
                               RecordException(exception, testResult);
                               return;
                       }

If there were some sort of extension point where this could call an
external thing something like this:

   protected internal virtual void ProcessException(Exception
exception, TestCaseResult testResult)
               {
                     if (ExceptionHandledByExtensionPoint(exception,testResult))
                     {
                               return;
                     }

                       if (!ExceptionExpected)
                       {
                               RecordException(exception, testResult);
                               return;
                       }


Then I think we could implement Assume such that it would be
orthogonal to the Test type. I think that would be a good thing...

<example snipped>

> > =============================================
> > Now, I don't know about you, but I like the second version better. And
> > this is very clearly data testing, not Theory.
> >
>
>  I can't see that this gives us much, we're using exceptions to control
> program flow, which is slow, and we only save a line of code, i.e we could
> have
>
>  if (!TypeOfFileIsB(fi))
>    return;

I don't think it's as expressive. That's all. It also reports the test
as having succeeded, rather than just not being applicable, which is a
difference that doesn't jive with Theory... (although that's what my
Assume does too because there isn't a test result type that's really
appropriate to the result yet.)

>  and it may be confusing for the users once they are used to seeing Assumes
> for Theories? In a theory, Assumptions are powerful as they tell us the
> cases for which we know our theory is not valid, but putting them in a
> general test feels strange.
>  This is only my opinioni and gut-feeling though, it might be that users are
> totally happy with this? I don't know...

I think it's a matter of pragmatism. Is it useful to Assume in data
tests, or even regular tests? I think it may be in some rare
conditions. Others may not agree. That's fine, it's just an opinion.

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Register now and save $200. Hurry, offer ends at 11:59 p.m., 
Monday, April 7! Use priority code J8TLD2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone