Re: TDD for numerical algorithms

Well, I don't have much to offer in the way of specifics, but maybe some of
these thoughts will be helpful:

Certain specialized cases need specialized tests. Randomness is one.
Performance can be another (Security, Load, ...) - I've had some recent
discussions about a binary search code kata I recorded and how to cope with
that case. As a general rule, tests shouldn't really be concerned about
implementation, but there's no way to really force a binary search into a
method when a simple linear loop through all elements works well enough and
passes tests. So perhaps that's an exception to the rule about knowing the
internals.

I heard once the Microsoft Excel team tests its spreadsheet by having
essentially 2 spreadsheet engines: one is the production engine that's
optimized for performance, the second is a completely separate
implementation whose is clearer to work with and understand, but slower, and
is used to verify the production code. (I need a reference for that - not
sure it's true).

Recently I ran across a paper about some interesting visual comparison tools
used on the Shrek movies to look for rendering bugs. They had to come up
with some creative ways to use automation to detect problems that would
probably be noticeable to humans while ignoring issues too small to pay
attention to.

With a randomness algorithm perhaps you can run a bunch of numbers through
and measure the distribution. While it may be hard, at some point the
authors of the algorithm had some qualities they were shooting for in the
numbers being output and they had to measure that themselves to see if they
were successful or not. These sorts of tests may be unconventional when
compared to the usual TDD fare (fair? faire? phair?)

Occasionally I'll write unit tests for timing issues that could fail if the
suite run happens while the machine is otherwise bogged down. It's risky
because those tests could be very fragile and cause a lot of maintenance
hassels ... but sometimes I've gotten away with a crufty test that does its
job well enough.


-- 
Chris
http://improvingenterprises.com
http://clabs.org


[Non-text portions of this message have been removed]



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/refactoring/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/refactoring/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/