Re: Qmail-Specific?

Ignore Reply-To, one-time avoidance of autoresponders on DJB's lists.

On 11 Feb 2010, at 20:36, Bruce Guenter wrote:
On Thu, Feb 11, 2010 at 01:13:08PM +0000, Sabahattin Gucukoglu wrote:
>> Can a wrapper script do this to pass the necessary environment to =
ezmlm programs, given that it can parse out the extension part?
>=20
> If the wrapper script could figure out the right variables, sure.  =
You'd
> have to adjust the ezmlmrc as well to build the right links and such.

I'll take a closer look later on, but I thought it'd be much harder than =
it looks.

> Now, I'm not exactly sure where the need for alternative mailing list =
managers came from.  Here's postfix's ancient VERP support:
>> http://www.postfix.org/VERP_README.html
>=20
> If I'm reading that right, this would seem to require that ezmlm send =
to
> Postfix via QMQP (or SMTP).  I don't know that the sendmail interface =
is
> adequate to the needs of ezmlm.  What happens if there are more
> recipients than can be passed on the command line, for example?

Yes, QMQP or ESMTP with PIPELINING would do it best for Postfix since =
that only requires starting up the server process once to queue the =
delivery, and not much overhead to fork an already initialised instance.

>> Ezmlm-idx's database and MIME support are attractive features.  I'd =
love it if (ever I find the time, and) it worked with other MTAs like =
this.
>=20
> I am willing to help.  I would love to be able to say that ezmlm-idx
> works with more than just qmail.

I'm still evaluating all the other MTAs to find "The right one for me".  =
So far I've found two (Courier and XMail) which have built-in list =
distribution features: Courier is closest to Qmail and probably needs =
not very much change to get ezmlm going ... and indeed Google shows this =
to be the case, at least for inbound delivery.  Courier supports a =
facsimile of the .qmail-* support, and indeed can be compiled and =
configured for this.  It has subscriber management using its mail =
processor, maildrop, called courier-mlm, which does similar things (but =
not as flexibly) to ezmlm-*.  XMail has distribution but no (as yet) =
subscriber management.

I'd be willing to write an SMTP client that injected QMQP-received mails =
to accommodate QMQP, if that is preferable to having EZMLM do this =
itself.  I'd be more inclined to do this in Tcl than in C though, since =
that would give the possibility to make it completely event-driven with =
no fork and no I/O and not much effort.  Also, the needs of my list are =
neither mission critical nor particularly demanding.

>> If nothing else, connection caching can be done by an MTA
>> holding deliveries for multiple recipients at any given time.
>=20
> This presumes that connection caching is a net benefit for a MTA.
>=20
By itself, not much, but when your queueing strategy includes sorting =
deliveries for optimal delivery to remote hosts (group by exchanger), =
and when you have clients completely independent of the queue as with =
Postfix and Qmail (as opposed to Exim and Sendmail which only use =
caching when the process pumping the queue happens to be in charge of =
the connections, which unless you use a single process, is not often =
enough for optimal delivery in a list case and results in long cache =
times and no parallelism), it should make a difference.  Qmail seems to =
be the absolute worst case since it doesn't even try to consolidate =
deliveries in the same transaction, however this doesn't matter with a =
VERP'd list.

>> The right way is to use RFC 1893 DSNs, rather than VER{P,H}.
>=20
> DSNs can't inject the sender's email address into the header or =
footer,
> giving customized unsubscribe instructions (among other things).
>=20
DSN is certainly appropriate for bounce handling, however unsubscription =
can take advantage of the sender's return address when s/he makes an =
unsubscription request.  I know this isn't the ezmlm way, but it is a =
solution used by most other MLMs (Mailman, Ecartis, etc).  It gains you =
speed of delivery in exchange for the inflexibility of foolproof =
unsubscription and bounce handling.  Most of the time a subscriber on a =
forward destination subscribed at the source can simply examine the =
headers to find out which address forwarded, and what for.  He can still =
ask the MLM to unsubscribe the source address, and confirm it at the =
destination.  And you can still use VERP, say on a monthly basis, or in =
newsletter-style or reminder-style messages sent by the managers.

>> But doing message personalisation would make ezmlm more =
MTA-compatible, with the small
>> downside that for maximum efficiency you would need to know how best
>> to put mailers into queue-only mode to avoid the deficiencies that
>> might be otherwise caused by eventual message submission over the =
time
>> of a delivery, for the purposes of exploiting connection caches.
>=20
> Actually, the biggest problem with doing message personalization in
> ezmlm is not connection caching or other delivery issues.  How do you
> reliably queue up large numbers of messages without starving other =
users
> of the disk and/or the queue (depending on how the MTA dequeues the
> messages)?  And what happens if the process is interrupted?  The
> complete set of messages needs to be effectively queued as a single
> transaction, and that can't happen when the messages are personalized
> outside of the MTA without some alternate kind of queueing process.
>=20
I would expect the implementation details of the MTA to cover your first =
two questions.  The advantage of doing VERP is simply that less state =
has to be maintained for a given list message, rather than a message for =
each recipient of that message, namely, who the last person you sent =
mail for was.  To make a many-message list distribution into a single =
transaction that the MTA exploits to best potential, you (depending on =
the MTA) either submit messages faster than the sorting and submission =
process so that there is little wasted session time, or simply tell the =
MTA to put the messages into the queue for the next queue run (sendmail, =
DeliveryMode=3Dq and then either wait for it or ask for it).  You could =
probably turn this into an SMTP extension, if you wanted; when a client =
connects, it requests that all mails sent during a connection are queued =
and not sent until the client quits.  This could roll into the work =
being done now to get transaction checkpoint standardised:
=
http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc=
1845bis.html

Cheers,
Sabahattin