[perl #132885] .next-handle seems to change handles but doesn't

On Mon, 19 Feb 2018 16:15:37 -0800, comdog wrote:
> #!/Users/brian/bin/perl6s/perl6-latest
> 
> I'm playing with .next-handle from IO::CatHandle. I'm
> trying to create a situation where I can read just the first five
> lines from each command line argument:
> 
>     quietly {
>         my $limit = 5;
>         for lines() {
>             state $lines = 1;
>             FIRST { $*ARGFILES.on-switch = { put "NEW FILE"; $lines = 1 } }
>             if $lines > $limit {
>                 $*ARGFILES.next-handle;
>                 next;
>                 }
>             put "{$*ARGFILES.path}:{$lines++} $_";
>             }
>         }
> 
> Here's a test file:
> 
>     First
>     Second
>     Third
>     Fourth
>     Fifth
>     Six
>     Seventh
> 
> With one or more command-line arguments I get this odd behavior (and
> lots of warnings that I suppressed):
> 
>     test.txt:1 First
>     test.txt:2 Second
>     test.txt:3 Third
>     test.txt:4 Fourth
>     test.txt:5 Fifth
>     NEW FILE
>     :1 Seventh
>     read bytes requires an object with REPR MVMOSHandle (got VMNull
> with REPR Null)
>       in block <unit> at lines-method.p6 line 5
> 
> It does "switch" after five lines, but then it keeps reading from the
> same handle while losing a line. Then there's a strange error at the
> end that kills the whole thing.
> 
> I expected that it would close the current handle, open a new one, and
> continue. If it gets to the end, it would simply not provide any more
> lines() and things would end normally.
> 
> The docs for .next-handle lets you keep changing it as long as you
> like no matter how many
> 
> -----
> 
> This is Rakudo Star version 2018.01 built on MoarVM version 2018.01
> implementing Perl 6.c.


This is because current implementation of IO::CatHandle.lines simply flattens
a gather/taken .lines Seq from each handle, so it never expects the handles to
get switched before each .lines Seq is fully-consumed. The .next-handle closes
the previous handle, which is where the weird error comes from (it tries to use
the now-nulled $!PIO to .read/.eof from). Pretty sure .words is similarly affected.

I think if we give IO::CatHandle!LINES and IO::CatHandle!WORDS their own
iterators that basically do the same thing as IO::Handle ones and then make
.next-handle also toss the remaining data in the $!decoder, then the OP code
would work.

In the meantime, you can use `.get` instead:

    my $limit := 3;
    my $lines = 1;
    with $*ARGFILES -> $af {
        while ($_ := $af.get) !=:= Nil {
            $lines == 1 and put "NEWFILE:";
            put "$af.path():{$lines++} $_";
            next unless $lines > $limit;
            $af.next-handle and $lines = 1;
        }
    }

I also noticed another deficiency: when trying to set .on-switch on $*ARGFILES,
you always miss its very first call that's done by IO::CatHandle.new, since to give
you $*ARGFILES, .new has already been called. 

Not sure what can be done about it, but I filed it as 
https://github.com/rakudo/rakudo/issues/1545 IO::CatHandle is not yet part of any
language, so we have the ability to polish the rough edges.