TAP::Parser::Multiplexer isn't parallel on Windows, solutions?

[email protected] (bulk88) Sat, 6 Jul 2013 18:28:45 -0400
Newsgroups perl.qa
Message-ID <[email protected]>
When running /t/harness on Windows, no matter how many test jobs I 
select in the env var, never more than 2 of the left most % progress 
things change, the others % things never get updated and remain as "0/?" 
always. Also my CPU on my 8 core never goes above 10%. I think this has 
to do with TAP::Parser::Multiplexer. Usually the harness perl process 
has 1 child process open in Process Explorer, but sometimes when there 
is rapid process turn (where the .t took less than 1 sec to run) it 
might be parallel.

I've tried stepping through _aggregate_parallel in TAP::Harness and 
TAP::Parser::Multiplexer and Multiplexer uses select() somehow on *nix 
but I dont understand how to implement a Windows specific fix to allow 
parallel testing to work, or the overall OOP heavy API design of TAP. 
I'd like to try to get parallel testing working on Windows, but I need 
help understanding how TAP the module's API works.

I did some hacking in Multiplexer with
____________________________________________________
=head2 Instance Methods

=head3 C<add>

  $mux->add( $parser, $stash );

Add a TAP::Parser to the multiplexer. C<$stash> is an optional opaque
reference that will be returned from C<next> along with the parser and
the next result.

=cut

sub add {
    my ( $self, $parser, $stash ) = @_;

    my @handles = $parser->get_select_handles;
        use Devel::Peek;
        Dump($handles[0]);
        use Win32API::File;
        use Win32::API;
        Win32::API::More->Import('kernel32.dll', 'BOOL WINAPI 
GetNamedPipeInfo(HANDLE hNamedPipe,
LPDWORD lpFlags,
LPDWORD lpOutBufferSize,
LPDWORD lpInBufferSize,
LPDWORD lpMaxInstances
)');
        my %h;
(        $h{Flags},
        $h{OutBufferSize},
        $h{InBufferSize},
        $h{MaxInstances}) = (0,0,0,0);
        $h{ret} = GetNamedPipeInfo(Win32API::File::GetOsFHandle($handles[0])
        $h{Flags},
        $h{OutBufferSize},
        $h{InBufferSize},
        $h{MaxInstances}
                                   );
        use Data::Dumper;
        print Dumper(\%h);
        print 
Win32API::File::GetFileType(Win32API::File::GetOsFHandle($handles[0]));
______________________________________________________

and $handles[0] was a Win32 Named Pipe, so in theory a non-blocking or 
asynchronous (read() returns control back to you immediately, but keeps 
the memory buffer around until the read transaction is done, you find 
out another way when the transaction is done with the buffer) or polling 
( 
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365779%28v=vs.85%29.aspx 
) IO is available.

Another idea is to open() multiple child perl test procs, and <> the 
oldest child perl test proc in TAP::Parser::Iterator::Process in FILO. 
With this idea it will sometimes and erratically be parallel, but it is 
better than 100% serial.

3 related commits to the problem, 
https://github.com/Perl-Toolchain-Gang/Test-Harness/commit/6d27f808824c3592e7089e7438f1ca18c860d307 
https://github.com/Perl-Toolchain-Gang/Test-Harness/commit/0da860a22df5ca50b9702a70d81fd960604755bd 
https://github.com/Perl-Toolchain-Gang/Test-Harness/commit/ccffd0283e1820165ec307ffbbc4d3ee16fca6b3 
by AndyA