Fwd: Addin Error Reporting Issue

On Feb 13, 2008 10:34 AM, Charlie Poole <[email protected]> wrote:
> Hi Kelly,
> FWIW, you could do the same thing like this, without the
> extension...
>
>     [Test]
>     public void AllObjectsMeetCondition()
>     {
>        ArrayList rv = new ArrayList();
>        DirectoryInfo di = new DirectoryInfo(@"..\..\DataFiles");
>        FileInfo[] fi = di.GetFiles("*.*");
>        foreach (FileInfo info in fi)
>        {
>          MyObject s = new MyObject(info.FullName);
>          rv.Add(s);
>        }
>
>        Assert.That( rv, Has.All.Property( "Condition", true ) );
>     }

Yes, you can do this. The question is how does it report a failure.
When Has.All.Property fails, does it output separate messages for each
Property in the collection that doesn't meet the condition? I really
need it to tell me which pieces of data fail in one pass, not a simple
all passed or something failed. That's why I did the addin. Granted, I
didn't look into Has.All.Property at the time so it could do reporting
the way I want, but I didn't know about it.

> My point is not that the addin is useless - this is a simple, contrived
> example after all - but that you may need to do a bit more than what
> NUnit already does here. The substitute code above has exactly the
> same problem as your code... the test will fail without any indication
> of which data element caused the failure and none of the other tests
> will be run.

This is only a problem in my code if it fails in building the list. It
does do proper reporting if one of the files doesn't meet the
condition inside the [IterativeTest]...

> > Which would allow me to debug the creation of the FileObjects
> > independently of my AddIn. I don't like the fact that because
> > FileObjects threw an exception none of the other tests are
> > successfully run, or even shown in the NUnit GUI. I admit to
> > still being a user of the Debugger, so the second test helps
> > me out a lot, if it can be run.
> >
> > I think this could be solved by putting a try-catch around
> > the creation of the tests, and if the creation of the test
> > throws an exception, then the test is somehow created in an
> > already failing state, and doesn't actually try to run later,
> > but the entire load doesn't fail. Does that make sense? Am I
> > stating the problem clearly enough? This seems like a fairly
> > general problem for all Addins that call user code during
> > test creation.
> >
> > Thanks for any suggestions!
>
> From NUnit's point of view, here's what is happening... barring
> any false memory of how your code works since it's not in front
> of me...
>
> 1) Here is a Type... lets see if someone CanBuildFrom it.
> 2) OK, this guy says he can, so
> 3) Let's call his BuildFrom...
> 4) Whoops... BuildFrom threw an exception, the whole thing
> is in error.

Pretty much.

> Since the FixtureBuilder is responsible for calling the
> TestCaseBuilder, it has to intercept any exceptions. If
> they are passed back to NUnit, it appears that the whole
> fixture is in error.
>
> Remember, when you are writing an addin, you are becoming
> a part of NUnit in a certain sense, so you can no longer
> completely rely on "NUnit" to catch the exceptions.

Sure, that's fine. The question though is how can I have it show up as
merely a failed test rather than a complete failure to load the entire
set of tests? I'm willing to do some work to make this behave better,
but the question is whether it can be done without messing with the
core of NUnit, or would I have to go into the Core?

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/