Re: establishing logical test order?

However, if you wanted to group tests-which seems like your goal-you
have two options.  You can use namespaces to hierarchically organize
your tests into collapsible, related groups.  Alternatively, you can
join your four tests together (here, T00-T03) into one test.  In this
second option, you will see  a failure in T00 without running T01
because T00's code will be run before T01 and therefore cancel out
execution of T01.


I'm not quite sure I understand. I already use classes and namespaces to
organize my tests. The trouble is, when they come up alphabetically,
they are not organized with any kind of meaning. The namespace
"Utilities" is near the bottom while the "ApplicationTests" is near the
top, with a bunch of other labeled stuff all mixed around them.

This is causing me to have to name my tests with these weird ordering
numbers, but this is not really scalable to large projects, or projects
that include multiple components.

David,

 

I misunderstood you.  I assumed that the grouping was more important
than the ordering: in other words, that having a collapsible group of
four tests would be "good enough".  Clearly, I took your example too
literally.

 

Sorry about that.

 

Rocco

 

From: David Jeske [mailto:[email protected]] 
Sent: Monday, June 23, 2008 5:14 PM
To: Pigneri, Rocco
Cc: [email protected]
Subject: Re: [Nunit-users] establishing logical test order?

 


On Mon, Jun 23, 2008 at 11:53 AM, Pigneri, Rocco
<[email protected]> wrote:

	As you mentioned, that is not a natural feature of unit tests:
unit tests are by definition unrelated to each other and can be run in
any order without introducing failures. 


Yes, my tests can be run in any order, they are all independent tests of
a single piece of functionality. 

However, all tests depend on the proper functioning of "upstream code".
I like to see the failures in order from smallest to largest number of
dependencies, because that's where the core problem usually lies. For
example, if something broke the core .NET string class, wading through
all kinds of other tests is not necessary if your basic-core set of
tests right at the top said strings are somehow broken.
 

	However, if you wanted to group tests-which seems like your
goal-you have two options.  You can use namespaces to hierarchically
organize your tests into collapsible, related groups.  Alternatively,
you can join your four tests together (here, T00-T03) into one test.  In
this second option, you will see  a failure in T00 without running T01
because T00's code will be run before T01 and therefore cancel out
execution of T01.


I'm not quite sure I understand. I already use classes and namespaces to
organize my tests. The trouble is, when they come up alphabetically,
they are not organized with any kind of meaning. The namespace
"Utilities" is near the bottom while the "ApplicationTests" is near the
top, with a bunch of other labeled stuff all mixed around them.

This is causing me to have to name my tests with these weird ordering
numbers, but this is not really scalable to large projects, or projects
that include multiple components.

It seems like we could do some simple assembly dependency analysis to
order the tests into dependency groups. This way tests of bottom-level
stuff would naturally be grouped together, and levels would get
increasingly complicated (with more dependencies) as they went down. A
project which had to many cross-dependencies that it's just one big soup
is probably not layered correctly.  When doing my own tests pre-NUnit, I
build this manually in the makefile test order. However, if we're
automating all of this, this seems like the way to do it. Thoughts?

- David

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
Nunit-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nunit-users