Re: An attack on a mint

On Mar 4, 2008, at 18:29, David Wagner wrote:

> Testing seems good when you want to ensure that desired  
> functionality is indeed present (that the code does do the good  
> things you want it to do) but not so good when you want to ensure  
> that undesired behavior is absent (that the code doesn't do bad  
> things you don't want it ever to do).

Indeed. I noticed this just recently: in my CapTP implementation, I  
could not think of any way to test that SturdyRefs are opaque to  
unprivileged clients. (On further thought, it /might/ be possibe with  
an information-flow-examining auditor.)

There's another form of testing that might help more though:  
Haskell's QuickCheck and its descendants. They check that an  
arbitrary predicate is true for a randomly selected subset of all  
possible inputs to it.

Since the test cases are machine-generated, they are not limited to  
the programmer's consideration of what needs to be tested. However,  
they might well check many simple cases and miss the edge cases where  
failures occur, since they ignore the structure of the algorithm  
being tested.

http://www.cs.chalmers.se/~rjmh/QuickCheck/

-- 
Kevin Reid                            <http://homepage.mac.com/kpreid/>