Re: Addin Error Reporting Issue

On Thu, Feb 14, 2008 at 3:00 PM, Charlie Poole
<[email protected]> wrote:
> Hi Kelly,
>
> > For programming errors (and NUnit is mostly about those)
> > stopping after the first failure seems right. For data tests,
> > most often it is not the right thing.
>
> I'd put it a little differently. NUnit works in two phases:
> first it loads tests, then it runs them.

I meant it in a different way, but what you say is right.

> Generally, if you want it to be non-runnable (yellow)
> do it at load time and if you want it to fail (red)
> do it at run time.

This is good advise. I'll remember that.

> OK, at this point, I realize that some of what I told you earlier
> was off the mark. I was thinking you had a TestFixtureBuilder
> as well as a TestCaseBuilder and that the fixture builder was
> adding the test cases. Obviously, that's not how it works, so
> igore any previous advise and lets go ahead with this code.

Gary's might work that way... perhaps that's where you got thrown off the scent.

> > So, if I try to handle the situation myself... My best guess
> > is something along these lines:
> >
> >     public Test BuildFrom(MethodInfo method)
> >     {
> >       Attribute attrib = Reflect.GetAttribute(method,
> > IterativeTestAttribute, false);
> >       string functionName =
> > IterativeTestFramework.GetSourceEnumeratorName(attrib);
> >       TestSuite suite = new TestSuite(method.Name);
> >
> >       object tester = Reflect.Construct(method.DeclaringType);
> >       MethodInfo m = tester.GetType().GetMethod(functionName);
> >       IEnumerable returnValue = null;
> >       try
> >       {
> >         returnValue = (IEnumerable) m.Invoke(tester, null);
> >         if (returnValue != null)
>
> CP: I would either not test for null and let it throw or put
> in code so that a null return is marked as non-runnable. The
> former seems easiest.

Seems reasonable.

> >         {
> >           foreach (object current in returnValue)
> >           {
> >             suite.Add(new IterativeTestCase(method, current));
> >           }
> >         }
> >       }
> >       catch (Exception e)
> >       {
> >         suite.RunState = RunState.NotRunnable;
> >         suite.IgnoreReason =
> >           string.Format("{0} failed to return a valid
> > IEnumerable.\n{1}", functionName, e.Message);
> >       }
> >
> >       return suite;
> >     }
> >
> > But since RunState and IgnoreReason have no intellisense...
> > I'm not sure this is exactly the right approach, but I
> > suspect it's something close to this that I want to do.
>
> I'm not sure why the intellisense fails sometimes, but this
> is how to handle the problem.
>
> In addition, you might want to consider the special case
> where the for an individual element throws. In that case,
> you can either not add the element or add a special element
> that will fail when executed.

I think that's a run time thing, not a build the test list thing. I
think it would show up red at runtime, and that's what I think you
want to have happen. Thanks for your help.

Now, how to write a test to test this code???? Hmmm.

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/