Re: RFC: A Test For Every Patch

Michael G Schwern <[email protected]> writes:

> Problems
> --------
> 
> If we have 5000 patches does that mean 5000 tests?  Yes.  

Probably more than that, depending on the functionality that a patch
adds.

> Won't that take a long time to run? Probably. Won't that make
> building Perl really annoying? Probably. Solution? Break up the
> tests into mandatory and secondary. The mandatory tests would be
> similar to what t/ is now and would be distributed with Perl and run
> at every build. The complete set of secondary tests would only need
> to be run periodically by the perl6 developers and not necessarily
> with every build. "make more tests" or something. If the set of
> secondary tests grows too large, it could be distributed seperately
> from the main distribution.

Note that for any testing build with new patches added then the
entire test suite should be run, mandatory and advisory. Even tests
that *COULDN'T POSSIBLY* be tripped by this new patch should be run.
After all, if they couldn't possibly be tripped then they won't be
tripped and all will be well...

> It is encouraged, but not necessary, that every patch come with a test.  It
> is simply necessary that the patch eventually aquire a test sometime before
> its being accepted.  As a result if someone is good at writing code, but not
> tests its okay.  Somebody will (hopefully) pick up the slack.  Writing tests
> also provides a "way in" for people who do not yet understand the perl
> internals yet wish to help out with development.

Note that for bugfix patches it should be the case that the test
*already* exists. I'm not sure if this should be a seperate RFC but 

Every Bug Should Come With A Test. Or should acquire a test as soon as
possible. The bugfix patch will then make the bug test succeed (as
well as possibly defining more tests...)

-- 
Piers