Re: How to manage c-client memory management?

Mark Crispin wrote:
> On Tue, 14 Jul 2009, Shawn Walker wrote:
>> I'm trying OP_SHORTCACHE, but this seems to slow the download messages 
>> since now each message has to be fetched from the server.
> 
> Indeed it does.  If you want to use early 1990s memory models, you must 
> suffer early 1990s performance problems.
> 
>> there are countries that have very low bandwidth (Australia has 180k) 
>> and to download huge files can take a long time.
> 
> That is why you should cache.  Aggressively.  Memory is cheaper than 
> bandwidth.

There are companies that will not upgrade the computer regardless how cheap memory is today.  I 
can't force the customers to buy more memory.  Hence your quote above about 1990s memory models.

> 
>> The issue with calling mail_free_cache is that the "nmsgs" get reset 
>> back to 0 when I'm trying to get the header, envelope, body or 
>> attachment as the message is being downloaded or the user took some 
>> action that I need to get the text of the message or attachment.
> 
> I did not see that you were using mail_free_cache().  That is an 
> internal function that you should never call.  The appropriate function, 
> which is documented in internal.txt, is mail_gc().

I'll try that out.

> 
>> The issue is how the users is storing their messages on the server, 
>> some users has 30,000, 100,000 or some ridiculous amount of messages 
>> in a folder. Caching all of those messages consume the memory.
> 
> Memory is cheaper than bandwidth.  I routinely deal with mailboxes of 
> that size.  I would not think of slowing things down to save a few 
> pennies of bandwidth.

There are companies that will not upgrade the computer regardless how cheap memory is today.  I 
can't force the customers to buy more memory.  Hence your quote above about 1990s memory models.

> 
>> And not everybody has a computer that has 4 GB of memory, some people 
>> in eastern Europe is still using computers from the mid nineties 
>> running Windows 95!
> 
> 14 year old computers are suitable for tasks of 14 years ago.  They are 
> not suitable for modern tasks.  At today's netbook prices, there is no 
> excuse to continue using an obsolete, power-wasting, dinosaur.

Try saying that to the customers that won't budge on upgrading.  We are only forced to have to deal 
with what the customers are using.  Nothing can change that.

> 
> Try running Outlook on that 100,000 message mailbox on that Windows 95 
> machine and see how well it does.
> 
> 4GB of memory costs less than an espresso at Starbucks.  However, you 
> should not need anywhere near this.  I regularly play with large 
> mailboxes on a Nokia N800 which only has 128MB.

I would say that Nokia N800 does not store all of you message on your phone.  I can access my 15,000 
IMAP folder with my iPhone with 16 GB of memory, but the iPhone does not does download all 15,000 
messages, just the first 100 or whatever I have it configured it for.

> 
>  From your use of the word "download", you are not using the c-client 
> library effectively.  If you were, nobody would ever download 30,000 
> messages in a session, much less 100,000 messages.

The users that we are dealing with will and always will download all 30,000, 100,000 1,000,000 
messages to their computer.  We cannot control how the user want to use the product.  Unless we 
really cripple how the product work.

> 
> Look at the Alpine source code for an example of how it is done properly.
> 

Using Alpine source to how we need to use IMAP is comparing apples and oranges.  If we could control 
how the application is running then we could model after Alpine.

> -- Mark --
> 
> http://panda.com/mrc
> Science does not emerge from voting, party politics, or public debate.
> Si vis pacem, para bellum.