Re: Tearing down when SetUp fails?

Err, well sorry about the name of the test - I originally used
it to submit a bug report to ReSharper because I saw different
behavior wrt. whether the [TearDown] method was invoked when
I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)

So the answer is most likely that I used both :)

After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and [TestFixtureTearDown]
so I read the documentation and then I wrote this mail.

+ Hans Christian

PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that
I started a discussion on what the behavior should be in NUnit.

On Fri, 12 Sep 2008, Charlie Poole wrote:

> Hi Hans Christian,
>
> We should discuss this behavior and try to get it right
> (whatever we decide that is) for 2.5.
>
> But before we get into it, because of the name of your
> test, I'm wondering if you actually ran under NUnit.
> When you run under resharper, no matter what version
> of the NUnit framework your test references, you
> get the semantics that resharper defines. That's
> because this particular behavior is not implemented
> in nunit.framework, but in nunit.core, which is
> not used in this situation.
>
> I'll run my own tests as well, but I thought I
> should ask.
>
>
> Charlie.
>
>> -----Original Message-----
>> From: [email protected]
>> [mailto:[email protected]] On
>> Behalf Of Hans Christian Falkenberg
>> Sent: Friday, September 12, 2008 5:33 AM
>> To: [email protected]
>> Subject: [nunit-developer] Tearing down when SetUp fails?
>>
>> Hi,
>>
>> I was about to post a bug about this - but then I read the
>> documentation and realized this might be something up for
>> discussion. Please let me know if it's there's a previous
>> conclusion on this and I'll just submit the bug report...
>>
>> Issues:
>> 1. TearDown behavior is inconsistent between
>>     [TearDown] and [TestFixtureTearDown] 2. The documentation
>> specifies that it should
>>     do the Wrong Thing (imho) :p
>>
>> What I think should happen:
>>    If there is an exception in any [SetUp] method, still run
>>    all [TearDown] methods.
>>    If there is an exception in any [TestFixtureSetUp] method,
>>    still run all [TestFixtureTearDown] methods.
>> (I realize only one of each method is allowed, but the doc says "any")
>>
>> What the documentation says (in both 2.4.8 and 2.5 alpha 3):
>>    So long as any SetUp method runs without error, the
>> TearDown method is
>>    guaranteed to run. It will not run if a SetUp method fails
>> or throws an
>>    exception.
>> and
>>    So long as any TestFixtureSetUp method runs without error, the
>>    TestFixtureTearDown method is guaranteed to run. It will
>> not run if a
>>    TestFixtureSetUp method fails or throws an exception.
>>
>> What actually happens (in both 2.4.8 and 2.5 alpha 3):
>> Running attached code:
>>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>>      /nologo bin\Debug\Dummy.dll
>>    Fixture Setting up
>>    .Setting up
>>    Tearing down
>>    FFixture Tearing down
>>
>>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
>> Uncommenting FixtureSetUp exception:
>>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>>     /nologo bin\Debug\Dummy.dll
>>    Fixture Setting up
>>    .F
>>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
>>
>>
>> So [TearDown] doesn't behave as specified when a [SetUp]
>> method throws an exception, that's a bug report, right?
>>
>> Except I think it *should* behave that way, so I was going to
>> submit a bug report on [TestFixtureTearDown] :)
>>
>> Here's my reasoning:
>> Say you are acquiring two resources in the SetUp method:
>> [SetUp] public void SetUp() {
>>    r1 = Resources.Acquire1();
>>    r2 = Resources.Acquire2();
>> }
>>
>> And of course the must be released:
>> [TearDown] public void TearDown() {
>>    Resources.Release(r1);
>>    r1 = null;
>>    Resources.Release(r2);
>>    r2 = null;
>> }
>>
>> Now, since Resource.Release will just return if its argument
>> is null and never throws, everything is fine. Except if the
>> documentation is to believed: If an exception can occur while
>> acquiring r2 (and it can), we'll have to write this:
>>
>> [SetUp] public void SetUp() {
>>    r1 = Resources.Acquire1();
>>    try {
>>      r2 = Resources.Acquire2();
>>    } catch {
>>      Resources.Release(r1);
>>      throw;
>>    }
>> }
>>
>> Which I think is bad, but I guess not *that* bad... until you
>> have 4-5 resources. Then there's suddenly a crazy amount of
>> TearDown code duplication in SetUp. So here is how I decided
>> to work around this problem:
>>
>> [SetUp] public void SetUp() {
>>    setUpCalled = true;
>>    r1 = Resources.Acquire1();
>>    r2 = Resources.Acquire2();
>> }
>>
>> [TearDown] public void TearDown() {
>>    if (!setUpCalled) return;
>>    setUpCalled = false;
>>    Resources.Release(r1);
>>    r1 = null;
>>    Resources.Release(r2);
>>    r2 = null;
>> }
>>
>> Not the worst of solution's, but now I'm basically doing
>> NUnit's work myself. And when the methods in TearDown *can*
>> throw Exceptions they will now be reported as if they were
>> thrown in SetUp, which is just to ask for someone being
>> confused later.
>> But at least the code will release resources properly in both
>> NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
>> behaves like the NUnit doc says it should) and future NUnit
>> versions which might adhere to the documentation.
>>
>> So... could we instead please change the NUnit behavior so
>> that other people don't have to write workarounds like this?
>>
>> I know junit doesn't run TearDown (or didn't when I last used
>> it a year back) when SetUp throws, but junit didn't report
>> test exceptions when TearDown throwed either, so not much
>> cause to let them set the standard here...
>>
>> A better argument against might be that people didn't write
>> code to check for null in their [TearDown] and
>> [TestFixtureTearDown] methods. But then their [TearDown]
>> methods are already failing, and they'll just have to fix
>> their [TestFixtureTearDown] methods when upgrading to 2.5, won't they?
>>
>> + Hans Christian
>>
>>
>>
>> PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
>> resources before a unit tests... Well, the resources are
>> files (with Acquire==Create and Release==Delete) and it would
>> hardly be proper to use mocked files for unit testing when
>> testing writing to the OS - that would kind of void the
>> entire testing.
>>
>> PPS: Why should a unit test need files? Well - it does.
>>
>
>
>

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/