Re: Mail::Box::IMAP updates

Trond Michelsen wrote:
> On Sun, Dec 19, 2004 at 09:02:17PM -0500, Tom Allison wrote:
> 
>>What I'm really after is trying to use the Mail::Box architecture to 
>>build an IMAP webclient that runs with HTML::Mason.  From what I've done 
> 
> 
> I was trying this a while ago, but had to abandon it due to memory
> issues. A message with a 30MB attachment would eat up something like
> 150MB of memory. Which is rather unacceptable in a mod_perl
> environment. I meant to have a look at the Mail::Message modules to
> see if I could get it to not read the entire message into memory,
> unless the user requests it, but I got a new job before I had the
> time. I'm not sure if anything's changed with Mail::Box by now, though.
> 

I think this could be addressed by checking the size first.  If to 
large, then save it to /tmp/?

> This could also be a problem with the IMAP-module. Even in a local
> network, you can easily get performance issues if you download 30MB
> unneccessarily, fairly often.
> 
> If you're sure you'll never encounter emails this large, then you can
> probably ignore the issue, though :)
> 
> Another problem is the number of mails. If you have a mailbox with
> 10,000 entries, and you're only interested in displaying 100 of them,
> you'd want to make sure that you're only pulling 100 headers from the
> IMAP-server. Even if the objects are fairly small and fast to create,
> it can slow down things a lot if you're only using 1% of the objects
> you've created.
> 

I did a lot of work with this for IMAPClient and Mason already.
At the time, I wasn't having much success with Mail::Box::IMAP4 and so 
started playing with Mail::IMAPClient directly.  One of the advantagest 
that IMAPClient ($imap) presented was the ability to retrieve three 
groups of UIDS in short order.

For a directory tree I used the
$imap->recent_count($folder);
$imap->unseen_count($folder);
$imap->message_count($folder);

To display the individual headers, I built an array of UIDS and used 
those in spliced groupings to show groups of 0-20/21-40/41-60....
These are all managed by the IMAP server and doesn't require that each 
mail message be opened up and read (like MessageIds would).
These arrays can get me to a list of email in the format of 
Subject/Date/From with a reference to the actual message by UID, but the 
process is pretty ugly and it is here that I might lean towards using 
Mail::Messsage for some of the process.  Especially for actually reading 
the message.

I cheat a lot with cache objects in mason with ages of 1 to 5 minutes to 
minimize thrashing through the entire folder trees.  This prevents 
rebuilding the display any more often than every minute.

At least with this approach, I might not have to create too much memory 
overhead until I actually open up the email itself.  But I really can't 
imagine a 30MB email BODY.  Attachments are different, but that comes later.

> 
> Well, MIME::Entity (the object returned from MIME::Parser) goes pretty
> well together with Template::Toolkit. I expect it to work equally well
> with HTML::Mason.

<snip>

Thanks.
I'll have to play with this for a bit to understand what exactly I'm 
doing here.