Re: Mail::Box significant reducation in memory usage

Mark Overmeer wrote:

>* Andy Maas ([email protected]) [040921 00:32]:
>  
>
>>I have some modification to Mail::Box files to help reduce memory usage 
>>significantly without sacrificing performance (speed). In fact with 
>>reduced footprint, processing time is reduced significantly on large msgs.
>>    
>>
>
>Very nice!
>
>By the way, the idea was to release Mail::Box::FastScalar as replacement
>for IO::Scalar.  In 5.8 perl, it is even possible to get rit of it as
>a whole, because you simply can open any scalar as a filehandle.  That
>would be even faster, I presume.  But not backwards compatible...
>
>In 5.8.1, perl complains about
>    {   # File without separators.
>        $lines = $file->getlines;
>
>--> Can't call $io->getlines in a scalar context, use $io->getline
>
>    {   # File without separators.
>        $lines = [ $file->getlines ];
>
>On the other hand: did you have a look at Mail::Box::Parser::C?  That will
>give some gains as well... to get your patch in, I need a fix there as
>well.
>
>I think there is much more to gain in performance ;-)
>  
>
Let me try address the two items above:

1. For getlines compatibility (non FastScalar getlines doesn't work in 
scalar context). Replace this line

    {   # File without separators.
        $lines = $file->getlines;

   with

        {   # File without separators.
            eval {
                $lines = $file->getlines;
            };
            if ($@) {
                if ($@ =~ /scalar context/) {
                    $lines = [$file->getlines];
                }
                else {
                    die $@;
                }
            }
        }

2. On Mail::Box::Parser::C

The aim is to make bodyAsList in that module be the same as in Perl.pm 
and at the same time avoid copying if possible. Two alternatives: 1) do 
it in C.pm (perl) or 2) in C.xs (C).

I'm new to this Perl XS thingy so won't even attempt at touching C.xs. 
As for C.pm, here is how it can be modified (but won't save extra 
copying which can probably be done in C.xs).

    sub bodyAsList(;$$)
    {   my ($self, $exp_chars, $exp_lines) = @_;
        $exp_chars = -1 unless defined $exp_chars;
        $exp_lines = -1 unless defined $exp_lines;
    /    # body_as_list $self->{MBPC_boxnr}, $exp_chars, $exp_lines; OLD
    /
    *    # NEW
        my ($begin, $end, @lines) = **body_as_list $self->{MBPC_boxnr},
    $exp_chars, $exp_lines;
        ($begin, $end, \@lines);
    *}