Re: [PATCH] for review: docs for the undocumented --rules option for'prove' (and related TAP::* bits)

On 08/20/2012 07:29 AM, Mark Stosberg wrote:
> 
>> However, I see that 'rules' is the subject of testing in t/scheduler.t.
>>   Do the individual tests in that file give you any clue as to how to
>> proceed?
> 
> Thanks for the feedback, Jim.
> 
> I believe I found what I needed over the weekend.
> 
> First, I made a list of what needed to be done here as some raw notes:
> https://github.com/markstos/Test-Harness/wiki/Missing-documentation-for-'rules'-and-'scheduler'-in-Test::Harness---App::Prove
> 
> 
> This morning I've submitted a "pull request" with proposed docs here.
> There is more feedback in this Github pull request:
> 
> https://github.com/Perl-Toolchain-Gang/Test-Harness/pull/5
> 
> I would be interested in a peer-review of my work.


When testing this on my large test suite, I believe I've found bug.
Here's my understanding of what appears to be happening:

A correct "schedule" is being created, with most tests set to run in a
parallel. The few exceptions I've sent are clearly being put in sequence
at the end.

Yet, a number of the exceptions still fail in these runs, but pass if
run by themselves.

Using a well-timed capture of activity with "ps", I was able to confirm
that my "exceptions" are running at the same time as some of the other
parallel tests.

I suspect there's a bug that works as follows, but I haven't isolated it
yet. Here's my suspected trigger:

- First start some tests that are parallel ready, but some of them are slow.
- Next in the schedule have some tests which much be run in sequence.

I think the "sequence" tests are in fact being run in sequence, but some
of the "slow" parallel tests are still running.

I'll try to mock-up this situation and see what I find.

   Mark