Re: what was VMS do here? (was [perl #79960] Setting $/ to read fixed records can corrupt valid UTF-8 input)

On Mar 2, 2012, at 3:07 AM, Eric Brine wrote:

> On Thu, Mar 1, 2012 at 6:17 PM, Craig A. Berry <[email protected]> wrote:
> What happens on Unix when you have a pipe buffer that is 8192 bytes and you set $/ to 8193 and read a record containing UTF-8 data through the pipe?
> 
> Perl requests 8K (formerly 4K) chunks until it has received enough. It requests 8K even if it only needs 1 byte.

I think you're thinking of the PerlIO buffer that I increased from 4K to the larger of 8K and BUFSIZ in 5.14, and which only applies to the perlio layer.  But S_sv_gets_read_record calls PerlIO_read, which just retrieves the base layer (formerly stdio, currently unix) and calls its Read method, which is just read().  So there is no buffering under Perl's control.

I was thinking of a situation where something external to Perl limits how much data you can get in one read and thus gives you less than the full amount requested by $/.  I'm pretty sure you'll get mangled UTF-8 if you happen to be mid-character when you hit the end of the device buffer.  To test this, you'd need to know something about the internals of your system's pipe implementation (or other device with a fixed buffer).

________________________________________
Craig A. Berry
mailto:[email protected]

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser