Re: LMTPD=>Postfix tune up

Sandino Araico Sánchez <[email protected]> Tue, 11 Nov 2014 22:06:47 -0600
Newsgroups	gmane.mail.imap.dbmail
Message-ID	<[email protected]>
On 03/11/14 11:21, Reindl Harald wrote:
>
>
> Am 03.11.2014 um 18:12 schrieb Jorge Bastos:
>> In fact it has a more solid behavior, I've set it to 2 processes.
>
> too much - Paul *explicitly* recommends 1 and in fact it can't handle
> more than one at the same time
It can.
You just need to run two or more (single-threaded) dbmail-lmtp processes
listening on a different IP address each.
You should be able to run as many dbmail-lmtpd processes as CPU cores in
your RDBMS for optimal performance.
>
>> But delivery is slowwww in this way, at least for the traffic that I
>> have,
>> I have around ~200/250 inbound emails/min
Parallelizing dbmail-lmtpd might help in performance.
You might also be interested in applying this patch for an extra 400%
performance gain in message insertion, no mather if the message is
received by dbmail-dmtp, dbmail-lmtpd or dbmail-imapd.

Only in dbmail-3.0.2/src: .dm_db.c.swp
Only in dbmail-3.0.2/src: 3.0-header_cache_perf.patch
diff -uri dbmail-3.0.2.orig/src/dbmail-message.c dbmail-3.0.2/src/dbmail-message.c
--- dbmail-3.0.2.orig/src/dbmail-message.c	2012-03-11 02:02:30.000000000 -0600
+++ dbmail-3.0.2/src/dbmail-message.c	2013-07-16 02:15:28.000000000 -0500
@@ -1621,7 +1621,7 @@
 	unsigned char *raw;
 	unsigned i;
 	time_t date;
-	volatile gboolean isaddr = 0, isdate = 0, issubject = 0;
+	volatile gboolean isaddr = 0, isdate = 0, issubject = 0, isblacklist = 0;
 	const char *charset = dbmail_message_get_charset(self);
 	gchar *sortfield = NULL, *datefield = NULL;
 	InternetAddressList *emaillist;
@@ -1652,6 +1652,20 @@
 		issubject=1;
 	else if (g_ascii_strcasecmp(header,"Date")==0)
 		isdate=1;
+	else if (g_ascii_strcasecmp(header,"received")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"content-type")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"x-spam-status")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"x-spam-score")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"dkim-signature")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"domainkey-signature")==0)
+		isblacklist=1;
+	else if (g_ascii_strcasecmp(header,"authentication-results")==0)
+		isblacklist=1;
 
 	values = g_relation_select(self->headers,header,0);
 
@@ -1666,6 +1682,11 @@
 			if (value) g_free(value);
 			continue;
 		}
+
+		if (isblacklist == 1) {
+			if (value) g_free(value);
+			continue;
+		}
 
 		// Generate additional fields for SORT optimization
 		if(isaddr) {
diff -uri dbmail-3.0.2.orig/src/dm_db.c dbmail-3.0.2/src/dm_db.c
--- dbmail-3.0.2.orig/src/dm_db.c	2012-03-11 02:02:30.000000000 -0600
+++ dbmail-3.0.2/src/dm_db.c	2013-07-16 02:29:17.000000000 -0500
@@ -1480,7 +1480,8 @@
 		r = db_query(c, "SELECT p.id "
 			"FROM %sphysmessage p "
 			"LEFT JOIN %sheader h ON p.id = h.physmessage_id "
-			"WHERE h.physmessage_id IS NULL", DBPFX, DBPFX);
+			"WHERE h.physmessage_id IS NULL "
+			"ORDER BY p.id DESC", DBPFX, DBPFX);
 		while (db_result_next(r)) {
 			id = g_new0(u64_t,1);
 			*id = db_result_get_u64(r, 0);

Another extra performance gain can be achieved by cleaning unneeded
tuples from dbmail_header and dbmail_header_value tables.

>
> then you need more then once instance on different ports so that each
> domain can have it's own lmtpd-daemon in transport maps or find a
> proxy solution between the MTA and dbmail spread incoming mail to the
> different instances

> frankly at such a high amount one has typically not only a single MTA
> but DNS round robin and each of them has it's own lmtpd-instance
> connecting to the same database over network
You only need to resolve your LMTP server to more than one IP address
using DNS round robin, then have all your domains be delivered to all
your dbmail-lmtpd instances.
>
> just because with that mail traffic you hardly have a single machine
> able to process the load of a contentscanner and with a contentscanner
> and sensible filtering (postscreen with dnsbl scoring...) i doubt the
> lmtpd will see 250 messages per minute :-)
>
>
>
>
>
>
>
> _______________________________________________
> DBmail mailing list
> [email protected]
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


-- 
Sandino Araico Sánchez 
http://sandino.net

_______________________________________________
DBmail mailing list
[email protected]
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
signature.asc (application/pgp-signature, 884 B)