Re: PIPE performance and calling 'time()' at the wrong time.

On Apr 29, 2010, at 9:32 AM, Hein, Nashua NH wrote:

> John Gillings asked me about an interesting behavior with a major
> performance difference using pipes.

Lots of interesting stuff snipped, but basically Perl reading from its  
own pipe is 10 times worse than reading from a file and Perl reading  
from a DCL pipe is 100 times worse.  Ouch.

> Also tried large settings for DECC$PIPE_BUFFER_QUOTA  / SIZE 'just in
> case'

Likely irrelevant for Perl piping in from a DCL command using qx().   
Try fiddling with PERL_MBX_SIZE, which is only 512 by default (finally  
bumped to 8192 in Perl 5.12.0).  Perl has its own pipe implementation  
which, like the CRTL's, is based on mailboxes.  But it had the ability  
to set the size of the mailbox buffer before the CRTL did.  We should  
probably make it honor the latecomer DECC$PIPE_BUFFER_SIZE as an alias.

Also likely irrelevant for your DCL PIPE command example.  At least I  
thought the pipe driver implemented its own device class and not a  
mailbox.  Even if it's a mailbox, the buffer size settings for the  
CRTL only apply to pipes created by the CRTL, and Perl's to pipes  
created by Perl.  This one is created by DCL.  Probably only Forrest  
Kenney or the source listings could reveal what goes on in there.

> Some reading
>
> http://h71000.www7.hp.com/openvms/journal/v9/mailboxes.pdf (Bruce
> Ellis)


This is a great article.  I think this is what finally made me realize  
that mailboxes can partially emulate stream behavior on read  
operations only but not on write.  Which makes them totally unsuitable  
for implementing pipes, but that's a different beef only tangentially  
related.

On the performance question, it would be interesting to know, when  
Perl is piping in from a command spawned under its control, how much  
of the slow-down is the set-up time of creating the process and how  
much is actually reading through the mailbox.

When reading from a DCL pipe in Perl, it might be worth experimenting  
with sysread(), which according to its docs, "bypasses buffered I/O."   
It rather sounds like you have a reader lazily waiting for the writer  
to do something interesting, but when DCL is the writer, it's even  
lazier and stalls out waiting for the reader to go ahead and do  
something with the small change already tossed its way.

________________________________________
Craig A. Berry
mailto:[email protected]

"... getting out of a sonnet is much more
  difficult than getting in."
                  Brad Leithauser