Re: Roadmap Updated

On Tue, Mar 11, 2008 at 3:54 PM, Charlie Poole
<[email protected]> wrote:
> Hi Kelly,
>
>  > I think we can talk about it... :-)  How else will we
>  > discover what it should be?
>
>  Then I'm in favor of the first talks being about definitions.

Ok, that sounds fine. What would you like to begin defining?

>  > I can appreciate the difficulties involved with that. I
>  > wonder if Lambda expressions might offer a solution to this.
>  > They are not evaluated until needed, which in this case would
>  > be later. You could pass in a Lambda expression creating the
>  > set of Tests to theorize about, but it would not be executed
>  > at Load time, but only when the expression is actually
>  > evaluated. It's a thought anyway.
>
>  There are lots of simpler ways than that. Otherwise, we couldn't
>  be doing it now - even under .NET 1.0.

Sure...

>  Lambda expressions could provide users a way to create arbitrary
>  code, which NUnit would then execute inside itself - but why
>  would we want that?

Because they can be assigned to variables. I would guess you could
even do it in an attribute, like this:
[Test code={dosomething()}]
Although I haven't tried that...

Just a brainstorming idea... I like backwards compatibility when
possible of course.

>  As I said, NUnit /already/ does what you want provided your
>  extension - which is not "user code" by the way - doesn't
>  defeat it by creating things in a non-lazy way or making
>  it appear to NUnit that things need to be repeatedly created.

I pretty easily got running my test cases from three minutes down to
13 seconds once I thought about it... so it was my bad approach within
my tests... so I'm agreeing with what you're saying here.

>  > Ah, then the practical approach it is to be eh? Well then,
>  > let me add my story...
>  >
>  > I have a set of XML files containing data that need to be
>  > checked against a set of rules. I need to have multiple Tests
>  > or Theories run against these multiple files (so it's a two
>  > dimensional problem) to check them for internal consistency.
>
>  So, is the set of files a static list or is it somehow
>  figured out dynamically?

Let's just say it's all the files in a directory that match a certain
regular expression.

That's a pretty normal use case, I think.

>  > I guess it's back to the stories then.
>
>  I may be misreading you, but I get the idea you want to pick a best
>  way to implement data-provision. For design, that makes sense. But
>  for requirements, it doesn't. I think there is a requirement for
>  static data providers and dynamic data providers.

I agree. I think the Theory concept combines the RowTest and
IteratorTest approaches nicely, and I've been in favor of combining
those two approaches for some time now.

I get the idea that you don't think Theory is comprehensive enough
somehow. Is that a proper read of your position? Or do you just want
to think about it more before committing to a particular API?

>  > That's an interesting user story... however, I'm thinking
>  > that one of the things that you want out of a testing
>  > framework is repeatability and you would certainly lose that
>  > in this case. Of course, it's not nice to be judgmental about
>  > how people use tools. They are just tools anyway.
>
>  I want both. I'd like to have a different set of random values
>  use every time I run the test but the same values used when
>  I need to re-run it under the debugger. I'm a demanding user.

Sure thing. As long as you can repeat the last run, I'm not against
random variables. I just like repeatability. I guess I'm used to
computers being predictable. Maybe I'm demanding too.. ;-)

>  > That's interesting. Then why go to the effort of using
>  > Mono.Addins if not to make this type of thing easier? If
>  > you're saying "general" user in the sense of people writing
>  > tests on a day to day basis, then I think I see your point.
>  > However, what would change? Would it make us less likely to
>  > provide good documentation for extension points?
>
>  By "general user" I mean someone who writes tests and doesn't
>  particularly care how NUnit works. When he needs to use a
>  new feature, he goes to some local guru and asks about it.
>  That guru might very well be writing extensions - just like
>  he might be creating user controls for others in his group
>  to use. He's the target for the addin architecture.

Ok, then we're on the same page. Perhaps we could differentiate
between "general user" and "NUnit guru" for purposes of conversation.
I think that there should be lots of NUnit gurus out there, and that
they should have better extension points and documentation to work
from.

>  The new architecture will have more extension points and
>  a better design - for example extensions on extensions -
>  that makes it easier to do things that are hard now, and
>  possible to do things that are impossible. But it isn't
>  intended to lower the bar for changing how NUnit works.

I always like to make things easier to do, even if they are hard
things. I don't want to diminish in any way how easy NUnit is to use
for the general user, and I don't think we have to in order for it to
be easier for the NUnit guru to extend.

>  > >  I think we would mess up the plugin architecture if we
>  > attempted  to
>  > > make it suitable for general use. OTOH, we could easily
>  > have  a plugin
>  > > - something like the addin Roy wrote - that makes it  easy
>  > for users
>  > > to do test-level extensions through inheritance.
>  >
>  > Are you referring to XtUnit? What sorts of test-level
>  > extension might be able to be done in a system such as you
>  > are describing?
>
>  Yes... take a look at how he uses a special attribute as
>  something a user can inherit from.

I will do so. Sounds very interesting.

-Kelly

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/