Re: Tearing down when SetUp fails?

Ah, I thought I had read somewhere that
,

On Fri, 12 Sep 2008, Charlie Poole wrote:

> OK, I re-read your first mail and I now see you did make a
> distinction betweeen running resharper and nunit-console.
>
> I want to do some research before I get back to you on the
> "how it should be" part. I have a vague sense that we may
> have changed the behavior from what the docs say as the
> result of a user request and if so I'd like to get the
> reasons for that request into the discussion.
>
> One part I can explain: in the 2.4 docs I meant the phrase
> "any setup method" to mean "either SetUp or TestFixtureSetUp."
> It's still not working as described, but I should have
> spelled it out for clarity.
>
> In 2.5, there can be multiple SetUp methods, and the logic
> of which TearDowns should and should not be run still
> has to be determined. Since there is not likely to be
> another 2.4 release, it's 2.5 that is the key to this
> discussion.
>
> Charlie
>
> Charlie
>
>> -----Original Message-----
>> From: Hans Christian Falkenberg [mailto:[email protected]]
>> Sent: Friday, September 12, 2008 1:12 PM
>> To: Charlie Poole
>> Cc: [email protected]
>> Subject: RE: [nunit-developer] Tearing down when SetUp fails?
>>
>> Err, well sorry about the name of the test - I originally
>> used it to submit a bug report to ReSharper because I saw
>> different behavior wrt. whether the [TearDown] method was
>> invoked when I used
>> 1) nunit-console.exe (see my exact command in orig mail) and
>> 2) Right click -> Run Unit Tests (with ReSharper in VS)
>>
>> So the answer is most likely that I used both :)
>>
>> After submitting that bug report I noticed that even NUnit
>> does it different between [TearDown] and
>> [TestFixtureTearDown] so I read the documentation and then I
>> wrote this mail.
>>
>> + Hans Christian
>>
>> PS! I also told the ReSharper guys (who replied just an hour
>> after my bug report that they would look on the issue) that I
>> started a discussion on what the behavior should be in NUnit.
>>
>> On Fri, 12 Sep 2008, Charlie Poole wrote:
>>
>>> Hi Hans Christian,
>>>
>>> We should discuss this behavior and try to get it right
>> (whatever we
>>> decide that is) for 2.5.
>>>
>>> But before we get into it, because of the name of your test, I'm
>>> wondering if you actually ran under NUnit.
>>> When you run under resharper, no matter what version of the NUnit
>>> framework your test references, you get the semantics that
>> resharper
>>> defines. That's because this particular behavior is not
>> implemented in
>>> nunit.framework, but in nunit.core, which is not used in this
>>> situation.
>>>
>>> I'll run my own tests as well, but I thought I should ask.
>>>
>>>
>>> Charlie.
>>>
>>>> -----Original Message-----
>>>> From: [email protected]
>>>> [mailto:[email protected]] On
>>>> Behalf Of Hans Christian Falkenberg
>>>> Sent: Friday, September 12, 2008 5:33 AM
>>>> To: [email protected]
>>>> Subject: [nunit-developer] Tearing down when SetUp fails?
>>>>
>>>> Hi,
>>>>
>>>> I was about to post a bug about this - but then I read the
>>>> documentation and realized this might be something up for
>>>> discussion. Please let me know if it's there's a previous
>>>> conclusion on this and I'll just submit the bug report...
>>>>
>>>> Issues:
>>>> 1. TearDown behavior is inconsistent between
>>>>     [TearDown] and [TestFixtureTearDown] 2. The documentation
>>>> specifies that it should
>>>>     do the Wrong Thing (imho) :p
>>>>
>>>> What I think should happen:
>>>>    If there is an exception in any [SetUp] method, still run
>>>>    all [TearDown] methods.
>>>>    If there is an exception in any [TestFixtureSetUp] method,
>>>>    still run all [TestFixtureTearDown] methods.
>>>> (I realize only one of each method is allowed, but the doc
>> says "any")
>>>>
>>>> What the documentation says (in both 2.4.8 and 2.5 alpha 3):
>>>>    So long as any SetUp method runs without error, the
>>>> TearDown method is
>>>>    guaranteed to run. It will not run if a SetUp method fails
>>>> or throws an
>>>>    exception.
>>>> and
>>>>    So long as any TestFixtureSetUp method runs without error, the
>>>>    TestFixtureTearDown method is guaranteed to run. It will
>>>> not run if a
>>>>    TestFixtureSetUp method fails or throws an exception.
>>>>
>>>> What actually happens (in both 2.4.8 and 2.5 alpha 3):
>>>> Running attached code:
>>>>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>>>>      /nologo bin\Debug\Dummy.dll
>>>>    Fixture Setting up
>>>>    .Setting up
>>>>    Tearing down
>>>>    FFixture Tearing down
>>>>
>>>>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
>>>> Uncommenting FixtureSetUp exception:
>>>>    c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
>>>>     /nologo bin\Debug\Dummy.dll
>>>>    Fixture Setting up
>>>>    .F
>>>>    Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
>>>>
>>>>
>>>> So [TearDown] doesn't behave as specified when a [SetUp]
>>>> method throws an exception, that's a bug report, right?
>>>>
>>>> Except I think it *should* behave that way, so I was going to
>>>> submit a bug report on [TestFixtureTearDown] :)
>>>>
>>>> Here's my reasoning:
>>>> Say you are acquiring two resources in the SetUp method:
>>>> [SetUp] public void SetUp() {
>>>>    r1 = Resources.Acquire1();
>>>>    r2 = Resources.Acquire2();
>>>> }
>>>>
>>>> And of course the must be released:
>>>> [TearDown] public void TearDown() {
>>>>    Resources.Release(r1);
>>>>    r1 = null;
>>>>    Resources.Release(r2);
>>>>    r2 = null;
>>>> }
>>>>
>>>> Now, since Resource.Release will just return if its argument
>>>> is null and never throws, everything is fine. Except if the
>>>> documentation is to believed: If an exception can occur while
>>>> acquiring r2 (and it can), we'll have to write this:
>>>>
>>>> [SetUp] public void SetUp() {
>>>>    r1 = Resources.Acquire1();
>>>>    try {
>>>>      r2 = Resources.Acquire2();
>>>>    } catch {
>>>>      Resources.Release(r1);
>>>>      throw;
>>>>    }
>>>> }
>>>>
>>>> Which I think is bad, but I guess not *that* bad... until you
>>>> have 4-5 resources. Then there's suddenly a crazy amount of
>>>> TearDown code duplication in SetUp. So here is how I decided
>>>> to work around this problem:
>>>>
>>>> [SetUp] public void SetUp() {
>>>>    setUpCalled = true;
>>>>    r1 = Resources.Acquire1();
>>>>    r2 = Resources.Acquire2();
>>>> }
>>>>
>>>> [TearDown] public void TearDown() {
>>>>    if (!setUpCalled) return;
>>>>    setUpCalled = false;
>>>>    Resources.Release(r1);
>>>>    r1 = null;
>>>>    Resources.Release(r2);
>>>>    r2 = null;
>>>> }
>>>>
>>>> Not the worst of solution's, but now I'm basically doing
>>>> NUnit's work myself. And when the methods in TearDown *can*
>>>> throw Exceptions they will now be reported as if they were
>>>> thrown in SetUp, which is just to ask for someone being
>>>> confused later.
>>>> But at least the code will release resources properly in both
>>>> NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
>>>> behaves like the NUnit doc says it should) and future NUnit
>>>> versions which might adhere to the documentation.
>>>>
>>>> So... could we instead please change the NUnit behavior so
>>>> that other people don't have to write workarounds like this?
>>>>
>>>> I know junit doesn't run TearDown (or didn't when I last used
>>>> it a year back) when SetUp throws, but junit didn't report
>>>> test exceptions when TearDown throwed either, so not much
>>>> cause to let them set the standard here...
>>>>
>>>> A better argument against might be that people didn't write
>>>> code to check for null in their [TearDown] and
>>>> [TestFixtureTearDown] methods. But then their [TearDown]
>>>> methods are already failing, and they'll just have to fix
>>>> their [TestFixtureTearDown] methods when upgrading to 2.5,
>> won't they?
>>>>
>>>> + Hans Christian
>>>>
>>>>
>>>>
>>>> PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
>>>> resources before a unit tests... Well, the resources are
>>>> files (with Acquire==Create and Release==Delete) and it would
>>>> hardly be proper to use mocked files for unit testing when
>>>> testing writing to the OS - that would kind of void the
>>>> entire testing.
>>>>
>>>> PPS: Why should a unit test need files? Well - it does.
>>>>
>>>
>>>
>>>
>>
>
>
>

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/