Re: Suggestion: generate_random method of validators for fuzz testing

On Fri, 2006-10-06 at 09:06 -0300, Jorge Godoy wrote:
> Peter S Russell <[email protected]>
> writes:
> 
> > + I have a TurboGears controller method that takes input from a form.
> > This method should always return an HTML page regardless of the input,
> > The HTML page should either be a response or an error message, but not a
> > 500 error (as a result of an unhandled exception).  It is impossible to
> 
> Not just on formencode side, but TurboGears has a decorator named
> "exception_handler" that pairs with "error_handler" on doing that.
> "error_handler" is activated with bad input, "exception_handler" with
> exceptions from your code.

Perhaps that wasn't a great example. 

I know that I can have exception handlers (and I do), but that doesn't
help me to make my methods stronger in the face of malicious input.
Ideally any bad input should be thrown out by my validators (that's what
they're there for), and never get as far as my method.  The flip side of
this is that if my method is working right, then it should never fail on
any input that does get through the validators.

If my exception handler gets called it's because my method isn't
correct, or my validators aren't restrictive enough.  I don't think I
should ever get an exception once the site is in production.  Even with
exception handlers I risk leaving something in an inconsistent state.

I'm talking about a way of using the validators to test the method more
thoroughly.  Writing unit tests by hand isn't a good way of testing that
a method behaves sensibly in the face of bad input, I'm neither stupid
enough nor ingenious enough to come up with enough of it.

There's more about Fuzz testing here: http://lwn.net/Articles/199429/

-- 
Peter S Russell <[email protected]>
Qustom


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV