Suggestion: generate_random method of validators for fuzz testing

Hi,

I've had an idea for FormEncode, I wonder if there's any interest in
seeing this implemented.

The idea is this:  A validator can be seen not just as a function to
check for valid input, it is also a specification of the data that is
valid input for a function.  If validators were able to generate data as
well as check data, then you could easily generate test cases for your
functions, in a style similar to QuickCheck in Haskell[1].  This is an
example of Fuzz testing.

Some example use cases:

+ I have a TurboGears controller method that takes input from a form.
This method should always return an HTML page regardless of the input,
The HTML page should either be a response or an error message, but not a
500 error (as a result of an unhandled exception).  It is impossible to
exhaustively test this, so I'm tempted just to write a couple of unit
tests, and leave it at that.  I almost certainly haven't considered some
corner cases.  If I could pass it a thousand random data points, I could
be much more certain that it really worked.

+ I am writing a new validator that subclasses another validator.  A
sensible test for this validator would be that any valid input to the
base class, when given to the child should result in an appropriate
python object, or an appropriate error message.  If I could generate a
lot of sample data, I could more easily test this.

+ I have a project that requires me to import a huge amount of sensitive
data to an SQLObject driven database (I really do).  The data is a dump
from another database in CSV format.  The import function is really a
complicated validator (it isn't because I was less wise when I wrote the
thing, but it really should be!).  I want to generate some very large,
realistic looking sample CSV files, to test uploading data to the DBMS,
without having to look at real CSV files.  Doing this by hand is quite
time consuming, and not very effective because I tend to take shortcuts,
so I'm not likely to exercise all of the code.  My upload function
provides a specification for the format of the data, so I feel like I'm
really doing the same thing twice when I write my function to generate
test data.

What do people think?  Personally I think this could make formencode
validators far more useful.

[1] http://www.cs.chalmers.se/~rjmh/QuickCheck/
-- 
Peter S Russell <[email protected]>
Qustom


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV