memory usage when saving attachments

Hi. 

I wondered if there is a way to save attachments to disk without
keeping all of it in memory. Mail::Message tends to use a lot of
memory with large attachments, compared to MIME::Parser.

with MIME::Parser:
> time ./decode.pl -type=mimeparser 1071580436.22782.ren.runbox.com,S=8556823
Before: 1978368
Message size: 8556770 bytes
MEM: Loading MIME::Parser: 3170304 bytes
MEM: Creating MIME::Parser object: 12288 bytes
MEM: Decoding message: 401408 bytes
After: 5566464
0.610u 0.110s 0:00.71 101.4%    0+0k 0+0io 364pf+0w

with Mail::Message:
> time ./decode.pl -type=mailmessage 1071580436.22782.ren.runbox.com,S=8556823
Before: 1978368
Message size: 8556770 bytes
MEM: Loading Mail::Message: 2854912 bytes
MEM: Creating Mail::Message object: 40919040 bytes
MEM: Decoding message: 27582464 bytes
After: 72777728
2.590u 0.140s 0:02.75 99.2%     0+0k 0+0io 373pf+0w

So, MIME::Parser uses 400kB to decode an 8MB mail, and Mail::Message
uses 27.5MB. In addition, Mail::Message uses 41MB to read the message
from a filehandle.

These numbers surprise me, since Mail::Box uses a lot of lazy loading
techniques to save memory. But it doesn't seem like this is the
default with Mail::Message objects. Am I doing something horribly
wrong to get these results?

One other thing - The attachments that are decoded from Mail::Message
seem to be consistently 1 byte larger than they are when decoded by
MIME::Parser. It seems that they somehow attract an extra newline at
the end.

I'm using these versions:

MIME::Parser: 5.406 
Mail::Message: 2.051


Attached: decode.pl

-- 
Trond Michelsen

#!/usr/bin/perl -s

use GTop;

my $dir = $::dir || "$ENV{HOME}/mime";

print "Before: ", mem_used(), "\n";
foreach my $msg (@ARGV) {
    open my $fh, $msg or next;
    print "Message size: ", -s $fh, " bytes\n";
    $FUNC{$::type}->($fh, $dir);
}
print "After: ", mem_used(), "\n";

BEGIN {
    our %FUNC = (mimeparser => sub {
		     my ($fh, $dir) = @_;
		     
		     mem();
		     require MIME::Parser;
		     mem("Loading MIME::Parser");
		     my $parser = MIME::Parser->new;
		     $parser->extract_nested_messages(1);
		     $parser->decode_headers(1);
		     $parser->output_to_core(0);
		     $parser->tmp_recycling(1);
		     $parser->output_dir( $dir );
		     mem("Creating MIME::Parser object");
		     
		     my $entity = $parser->parse( $fh );

		     mem("Decoding message");
		     return;
		 },

		 mailmessage => sub {
		     my ($fh, $dir) = @_;
		     mem();
		     require Mail::Message;
		     mem("Loading Mail::Message");
		     my $msg = Mail::Message->read($fh);
		     mem("Creating Mail::Message object");
		     foreach my $part ($msg->parts) {
			 my $fn  = $part->body->dispositionFilename($dir);
			 $part->body->decoded->write(filename => $fn)
			   or die "Couldn't write to $fn: $!\n";
			 $part->delete;
		     }
		     mem("Decoding message");
		     return;
		 },
		);
}

sub mem_used { GTop->new->proc_mem($$)->size }

sub mem {
    my $string = shift;
    our $MEM_USED;
    my $used = mem_used();
    printf "MEM: %s: %d bytes\n", $string, $used - $MEM_USED
	if $string;
    $MEM_USED = $used;
    return;
}