Abnormal huge allocations happening when "scanning new messages"

The function imap_rescan (or a nearby function) allocates +60 MB of ram
to fetch 14,558 headers and then it suddenly clears everything.

I find it "very" hard to believe that you really need to allocate 60MB
of ram, just to write a summary file of ~1MB. Is there something not
being freed while processing happens?

Can't this be implemented with a simple .. ?

while (stuff = get_next_from_imap ())
{
	add_to_summary (stuff)
	free_that_next_stuff (stuff)
}

How can such an implementation ever need 60MB ram? I also don't think my
IMAP service is "sending" me more than a megabyte or two of information.

The only way how I *can* implement having to allocate 60MB just to store
a 1MB summary, is to do something like:

all_stuff = get_everything_from_imap()

copy (allstuff, extraallstuff)
copy (extraallstuff, extraextraallstuff)
copy (extraextraallstuff, extraextraextraallstuff)

foreach (stuff in extraextraextraallstuff)
{
	add_to_summary (stuff)
}

free (allstuff)
free (extraallstuff)
free (extraextraallstuff)
free (extraextraextraallstuff)

I'm probably soon going to checkout what is really happening here.
Because allocating 60MB for fetching the headers of a small 14,558
messages folder is ... simply insane in my opinion. Note that the
allocation happens linear (in valgrind, you'll see a perfect triangle
with a 90 degrees angle when the deallocation happens).

I also noticed that by far not everything is getting freed after the
procedure. So it's also leaking very much (like, a 3 MB leak at least).

By quick-looking at the current code. I find it hard to believe that
somebody actually *really* used his common sense to implement this: the
code is really a mess. And while saying that, I don't care which famous
great hacker implemented it. It's not well-done imo.

The imap4 code at least *looks* nicer. But I haven't yet in depth
checked out that one (and it's often not really working).


ps. I have this grey feeling that I'm going to find a lot dead bodies in
camel, now that I started really looking into its code. Is libspruce
ready Fejj? ;-)


-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be