IMAP writes takes forever
Raul Dias <[email protected]> Fri, 03 Jun 2005 16:16:34 -0300
| Newsgroups | gmane.comp.lang.perl.modules.mail-box |
|---|---|
| Message-ID | <[email protected]> |
Hi, I have not really looked at the source code to findout why, so this is most a report of my expirience. I have being migrating large MBoxes files to a IMAP server. So in this expiriece, the MBox file was 1.6 GB in size and had about 16500 messages. So the procedure was -open the mbox -open the imap folder -looped every message in the mbox -copyTo() the message to the imap folder. -delete() the message. -close imap folder -close the mbox folder. In the first run the machine run out of memory after hours. It is a dual Athlon MD with 3.5GB of RAM. In the second run I limited the work to a max of 10000 messages than close and reopen the mailboxes. This kept the memory usage to a max of 79% (2.8GB). It took 2 days of processing (at 90% CPU usage) before I decided to kill it. Doing a strace in the script process I saw that it did thousands of FETCH( <TONS of UID> ) FLAGS and SEARCH to the IMAP server. Then <TONS of UID> seems to be one UID foreach message in the IMAP folder. So if there is 40000 messages in the folder it will keep doing FETCH for every UID. As I said I have not look into the source to findout if this is really happening into Mail::Box or Mail::IMAPClient. But I suspect this is happening at Mail::Transport::IMAP4. Raul Dias