Re: ezmlm on courier?

>> Thanks for this pointer, it lead the way to a better solution: fake
>> qmail-queue.
>
> That doesn't really sound better, but it is certainly less invasive  
> on the part of ezmlm-idx.

Ok, I agree. Maybe "easier" would have been a better word choice.


>> I spent a few hours today and have something in place
>> that emulates the qmail-queue binary and turns around and calls
>> Courier's sendmail bin, which includes a "-verp" flag suitable for
>> accepting a large number of recipients. Brief testing shows that it
>> seems to work for subscribing and posting, so we may be good to go.
>
> Cool.  I would be curious to see this too.

Attached below.

This is not my finest code -- it is a first-past draft at just  
attempting to see if it'd work. It could use some additional error  
checking (say, bad input given). As I said, if it does work, I'd be  
delighted if someone wanted to clean it up, re-write it in C for  
efficiency, and include it in the standard ezmlm-idx distribution.

Some of the exit codes don't line up as well as ideal. For example,  
it's possible that the first few runs of sendmail would work, but  
subsequent ones would fail (say, disk full). In other words,  
submission to the MTA is not an atomic operation because of the 100- 
max-recipients restriction. The script gives a permanent failure if  
any run of sendmail fails -- so, some users may actually receive the  
message. This is a pretty unlikely scenario, but it's possible -- if  
there are better suggestions on how to better marry ezmlm-idx to  
Courier, I'd love to know.


>> Courier-MTA's verp encoding, by the way, encodes some characters that
>> qmail doesn't -- I noticed that '-' in a VERPed address on qmail is
>> unmodified, on Courier-MTA it gets converted to +2D. Neither here nor
>> there, but thought I'd mention it for posterity's sake.
>
> Does Courier's sendmail program include a mode where it doesn't  
> rewrite
> the header?  qmail's sendmail invokes qmail-inject, which does  
> rewriting
> on email addresses in the header.

As far as I know, the "-verp" flag causes Courier to send the message  
unmodified, but with the envelope from being VERPed. Leaving that  
flag off, Courier will not do VERP on the envelope-from.

If there are conditions in which qmail-inject is actually modifying  
the message, then the script would need to explode the message out to  
every recipient and do the verping itself. For low volume /  
infrequent messages (our case), this might be acceptable.

Under what circumstances does ezmlm generate a message where qmail- 
inject rewrites the headers of the message? (And if so, what logic is  
applied? How would I replicate it?)

 From what I could see, it looked like ezmlm itself was either  
striping the verp'ed address in (i.e. sub, unsub), or was handing the  
message to the MTA with only the need for the envelope from address  
to be VERPed.  I would guess there may be an issue with the "click  
here to unsub" type links in emails, where the VERPing needs to be  
included in some sort of body text, but I did not think ezmlm  
supported this.


best,
Jeff


#!/usr/bin/perl

# Accept incoming mail in the same format that qmail's qmail-queue does,
# an use the local server's MTA to deliver it.

# This is nowhere near as efficient as a real install of qmail, but  
may be suitable
# for small lists or infrequent deliveries.

# (c) 2007 Atof Inc, released under GPL v2 or any license suitable  
for inclusion with ezmlm-idx

eval {
         local $SIG{ALRM} = sub { die "alarm\n" };
         alarm 60;

         open MAIL, "<&0" || die ("can't read fd 0: $!");
         $\ = undef;
         while (<MAIL>) { $message .= $_; }
         close MAIL;

         open ADDR, "<&1" || die ("can't read fd 1: $!");
         while (<ADDR>) { $addresses .= $_; }
         close ADDR;

         $lastChar = ord(chop($addresses));
         $secondToLastChar = ord(chop($addresses));
         if ( 0 != $lastChar && 0 != $secondToLastChar) {
                 # qmail-queue expects last two chars to be null  
bytes to signal address list is complete.
                 exit(54);
         }

         @lines = split(/\0/,$addresses);
         while ($line = shift(@lines)) {
                 if (substr($line,0,1) eq 'F') {
                         $from = substr($line,1);
                 }
                 elsif (substr($line,0,1) eq 'T') {
                         push(@rcpts, substr($line,1));
                 }
                 else {
                         exit(54);
                 }
         }

         alarm 0;
};

if ($@) {
         die unless $@ eq "alarm\n";
         # Read timed out.
         exit(52);
}

# If we get here, we have $message, $from, and @rcpts.

# If $from ends with '-@[]', we need to do verping.
$useVerp = 0;
if (substr($from, -4) eq '-@[]') {
         $useVerp = 1;
         $from = substr($from, 0, length($from)-4);
}

# Set the envelope sender:
$from =~ /([^\@]+)\@(.+)/;
$local = $1;
$host = $2;

# qmail verping doesn't add a '-', courier does, though:
if (substr($local,-1) eq '-' && $useVerp) { chop $local; }
$ENV{'MAILSUSER'} = $local;
$ENV{'MAILSHOST'} = $host;


# Loop through the recipients:
$offset = 0;
$maxRcptPerMessage = 100;
while ($offset < scalar @rcpts) {
         $endPosition = $offset+$maxRcptPerMessage-1;
         if (scalar @rcpts <= $endPosition) { $endPosition = scalar  
@rcpts-1; }
         @args = @rcpts[$offset..$endPosition];
         if ($useVerp) {
                 unshift(@args, '-verp');
         }

         eval {
                 local $SIG{ALRM} = sub { die "alarm\n" };
                 alarm 60;

                 open(SENDMAIL,'|-') || exec '/usr/bin/sendmail', @args;
                 print SENDMAIL $message;
                 close SENDMAIL;

                 alarm 0;
         };

         if ($@) {
                 exit(13); # Re-use read timed out error as a hard error
         }

         $offset += $maxRcptPerMessage;
         sleep (1 + $maxRcptPerMessage / 50); # Throttle queue  
injects to the rate of 50 messages per second -- not necessary for  
this script, but sorta nice to be able to do
}