Re: preforking prove

Now on cpan. A much simpler solution than what I suggested :) and apparently still works with parallel testing. Thanks miyagawa!

   https://metacpan.org/module/forkprove

On Nov 6, 2012, at 9:58 AM, Jonathan Swartz wrote:

> miyagawa sent me this proof of concept - using fork, not starman. personally I like the opportunity to combine with parallel testing. :)
> -----
> proof of concept - https://gist.github.com/4024197
> 
> * using Shotgun loader (plackup -L Shotgun) would make a clean environment for perl to run test for each request, while still preloading modules with -M. Combined with Starman, you'll get a benefit of running multiple workers for parallel testing.
> 
> * But if you just need to preload modules to run tests, using fork() like above is much easier and involves no HTTP server etc :)
> 
> * My POC above seems to work fine (I will probably make a github repo later), but it fails in some tests (with Moose, only a couple of tests fail) and it runs almost twice as fast. It is important to remember NOT to preload Test::More (and Test::Builder) in the parent process because doing so will mess up things.
> 
> On Nov 5, 2012, at 5:03 PM, Jonathan Swartz wrote:
> 
>> We have a large slow test suite at work (Test::Class, 225 classes, about 45 minutes run time). Many of the tests start by loading a bunch of the same modules. Obviously we could speed things up if we could share that loading cost.
>> 
>> I'm aware of Test::Aggregate and Test::Aggregate::Nested, but a number of our tests run into the caveats (e.g. they change singletons -- yes, this is not ideal, but also not easily changeable).
>> 
>> I thought of an alternative to Test::Aggregate called "preforking prove", or pfprove, which does the following:
>> 
>> * Accepts the same arguments as prove
>> * Preloads the module(s) in -M
>> * Launches a Starman server in the background
>> * For each test file, makes a request to the Starman server. The Starman child runs the  test and exits.
>> 
>> The idea is that you preload all the common stuff in the Starman parent, so that forking each child and running each test is fast (at least on Linux).
>> 
>> Potential advantages over Test::Aggregate:
>> * runs one test per process, so avoids some caveats
>> * keeps the TAP in traditional form (one TAP stream per file)
>> * works well with parallel prove
>> 
>> Potential disadvantages:
>> * lots of extra complexity (requires Starman or similar, need to make sure it shuts down, need to handle errors, etc.)
>> 
>> Curious what you think. Is there something like this out there already? Potential problems?
>> 
>> If I do this I might call it Test::Aggregate::Preforking, just to keep it in the same category.
>> 
>> Thanks!
>> Jon
>> 
>