Re: Tearing down when SetUp fails?

Hi Hans Christian,

We should discuss this behavior and try to get it right
(whatever we decide that is) for 2.5. 

But before we get into it, because of the name of your
test, I'm wondering if you actually ran under NUnit.
When you run under resharper, no matter what version
of the NUnit framework your test references, you 
get the semantics that resharper defines. That's 
because this particular behavior is not implemented
in nunit.framework, but in nunit.core, which is
not used in this situation.

I'll run my own tests as well, but I thought I
should ask.


Charlie.

> -----Original Message-----
> From: [email protected] 
> [mailto:[email protected]] On 
> Behalf Of Hans Christian Falkenberg
> Sent: Friday, September 12, 2008 5:33 AM
> To: [email protected]
> Subject: [nunit-developer] Tearing down when SetUp fails?
> 
> Hi,
> 
> I was about to post a bug about this - but then I read the 
> documentation and realized this might be something up for 
> discussion. Please let me know if it's there's a previous 
> conclusion on this and I'll just submit the bug report...
> 
> Issues:
> 1. TearDown behavior is inconsistent between
>     [TearDown] and [TestFixtureTearDown] 2. The documentation 
> specifies that it should
>     do the Wrong Thing (imho) :p
> 
> What I think should happen:
>    If there is an exception in any [SetUp] method, still run
>    all [TearDown] methods.
>    If there is an exception in any [TestFixtureSetUp] method,
>    still run all [TestFixtureTearDown] methods.
> (I realize only one of each method is allowed, but the doc says "any")
> 
> What the documentation says (in both 2.4.8 and 2.5 alpha 3):
>    So long as any SetUp method runs without error, the 
> TearDown method is
>    guaranteed to run. It will not run if a SetUp method fails 
> or throws an
>    exception.
> and
>    So long as any TestFixtureSetUp method runs without error, the
>    TestFixtureTearDown method is guaranteed to run. It will 
> not run if a
>    TestFixtureSetUp method fails or throws an exception.
> 
> What actually happens (in both 2.4.8 and 2.5 alpha 3):
> Running attached code:
>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>      /nologo bin\Debug\Dummy.dll
>    Fixture Setting up
>    .Setting up
>    Tearing down
>    FFixture Tearing down
> 
>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds 
> Uncommenting FixtureSetUp exception:
>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>     /nologo bin\Debug\Dummy.dll
>    Fixture Setting up
>    .F
>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
> 
> 
> So [TearDown] doesn't behave as specified when a [SetUp] 
> method throws an exception, that's a bug report, right?
> 
> Except I think it *should* behave that way, so I was going to 
> submit a bug report on [TestFixtureTearDown] :)
> 
> Here's my reasoning:
> Say you are acquiring two resources in the SetUp method:
> [SetUp] public void SetUp() {
>    r1 = Resources.Acquire1();
>    r2 = Resources.Acquire2();
> }
> 
> And of course the must be released:
> [TearDown] public void TearDown() {
>    Resources.Release(r1);
>    r1 = null;
>    Resources.Release(r2);
>    r2 = null;
> }
> 
> Now, since Resource.Release will just return if its argument 
> is null and never throws, everything is fine. Except if the 
> documentation is to believed: If an exception can occur while 
> acquiring r2 (and it can), we'll have to write this:
> 
> [SetUp] public void SetUp() {
>    r1 = Resources.Acquire1();
>    try {
>      r2 = Resources.Acquire2();
>    } catch {
>      Resources.Release(r1);
>      throw;
>    }
> }
> 
> Which I think is bad, but I guess not *that* bad... until you 
> have 4-5 resources. Then there's suddenly a crazy amount of 
> TearDown code duplication in SetUp. So here is how I decided 
> to work around this problem:
> 
> [SetUp] public void SetUp() {
>    setUpCalled = true;
>    r1 = Resources.Acquire1();
>    r2 = Resources.Acquire2();
> }
> 
> [TearDown] public void TearDown() {
>    if (!setUpCalled) return;
>    setUpCalled = false;
>    Resources.Release(r1);
>    r1 = null;
>    Resources.Release(r2);
>    r2 = null;
> }
> 
> Not the worst of solution's, but now I'm basically doing 
> NUnit's work myself. And when the methods in TearDown *can* 
> throw Exceptions they will now be reported as if they were 
> thrown in SetUp, which is just to ask for someone being 
> confused later.
> But at least the code will release resources properly in both 
> NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which 
> behaves like the NUnit doc says it should) and future NUnit 
> versions which might adhere to the documentation.
> 
> So... could we instead please change the NUnit behavior so 
> that other people don't have to write workarounds like this?
> 
> I know junit doesn't run TearDown (or didn't when I last used 
> it a year back) when SetUp throws, but junit didn't report 
> test exceptions when TearDown throwed either, so not much 
> cause to let them set the standard here...
> 
> A better argument against might be that people didn't write 
> code to check for null in their [TearDown] and 
> [TestFixtureTearDown] methods. But then their [TearDown] 
> methods are already failing, and they'll just have to fix 
> their [TestFixtureTearDown] methods when upgrading to 2.5, won't they?
> 
> + Hans Christian
> 
> 
> 
> PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5 
> resources before a unit tests... Well, the resources are 
> files (with Acquire==Create and Release==Delete) and it would 
> hardly be proper to use mocked files for unit testing when 
> testing writing to the OS - that would kind of void the 
> entire testing.
> 
> PPS: Why should a unit test need files? Well - it does.
> 



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/