Re: max mails per folder
Alan Hicks <[email protected]> Tue, 10 Nov 2015 10:44:37 +0000
| Newsgroups | gmane.mail.imap.dbmail |
|---|---|
| Message-ID | <[email protected]> |
On 09/11/2015 16:44, Jure Pečar wrote: > On Mon, 09 Nov 2015 12:38:06 +0100 > Andrea Brancatelli <[email protected]> wrote: > >> >> >> Well, obviously it the depends on the mail amount. >> >> We have about 1000 mailboxes with about 350GB of online mail. On spike >> hours we have a full recycle of the logs in about 10 minutes especially >> because of circular replication increasing the server activities. > > I ran a separate dbmail imapd instance in order to capture what's happening at append time. This is the point I found lots of time spent: > > Nov 09 17:24:54 email-prod-01.a dbmail-imapd[16154]: [0x7f398a864e30] Database:[db] db_stmt_prepare(+419): [0x7f398a82cb00] [SELECT seen_flag, answered_flag, deleted_flag, flagged_flag, draft_flag, recent_flag, DATE_FORMAT(internal_date, GET_FORMAT(DATETIME,'ISO')), rfcsize, message_idnr FROM dbmail_messages m LEFT JOIN dbmail_physmessage p ON p.id = m.physmessage_id WHERE m.mailbox_idnr = ? AND m.status IN (0,1) ORDER BY message_idnr ASC] > Nov 09 17:24:54 email-prod-01.a dbmail-imapd[16154]: [0x7f398a864e30] Database:[db] db_stmt_set_u64(+439): [0x7f396400ad70] 1:[27520] > Nov 09 17:25:06 email-prod-01.a dbmail-imapd[16154]: [0x7f398a864e30] Database:[db] db_con_clear(+298): [0x7f398a82cb00] connection cleared > > It's 12 seconds in this case. > > Looking at the code I find this statement in dm_mailboxstate.c, function state_load_messages. Immediately after the query is done, msginfo tree is filled with the data from the query resut. > > It appears this tree insert is the cause of slowness we're observing. Since this is in GLib code, I'll dig further there. I'll also have to dust off some trees, insertion and sorting CS theory ;) > > Hi, This slowing down might be able to be refactored out as I'm unsure why, on inserting a message, a tree of messages is needed. I haven't looked at the code recently but didn't think dbmail retrieved the messages before or after an insert. It's normal for databases to prefer a table scan rather than use an index when it thinks the number of rows exceeds the benefit of using an index. Alan -- Persistent Objects Ltd 128 Lilleshall Road London SM4 6DR Lifting brand value by using all means at my disposal including technological, motivational and best practice. Proud sponsor of TEDx Wandsworth 2015 Registered in England and Wales 03538717 +44/0 79 3030 5004 +44/0 20 8544 5292 http://p-o.co.uk https://plus.google.com/+AlanHicksLondon _______________________________________________ DBmail mailing list [email protected] http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail