Re: How getmail deals with delivery errors and with bounce e-mails

R. Diez <[email protected]> wrote:
> 
> > You do not *ever* want to generate a bounce message after retrieving a
> > message [...]
> 
> I am a newbie and it would be nice if you could mention an example of what
> kind of problems I would get if I did generate bounce messages like that.
> That would probably help my understanding.
 
There are many, many facets to this issue, and some of them require a fairly
deep understanding of internet mail architecture and practice to fully
appreciate.  I can't possibly give an exhaustive list, so just two examples
will have to suffice.
 
Note terminology:
 
  MTA - mail (or message) transfer agent.  Software that receives email from
  other machines (and the local machine) and delivers email (to other
  machines, or to local users).  e.g. qmail, postfix, etc.

  MUA - mail user agent.  The program users use to read, write, and usually
  send mail.  Talks to an MTA to send messages (via SMTP, or via a local
  injection program) and may talk to POP or IMAP servers to retrieve messages.  

  MDA - mail delivery agent.  A program that the MTA running on a system uses
  to deliver messages to a local mailbox (user, virtual user or mailbox, etc).
  e.g. dovecot-lda, qmail-local, etc.

For others reading this in future, you also need to understand the difference
between the message envelope (and how it is/is not recorded in the message
headers seen by the recipient) and the informational addressing fields in the
message content (To:, Cc:, etc).

1. Many POP/IMAP mail providers use MTA/MDAs configured incorrectly or
unhelpfully, such that the envelope information recorded in message headers
does not actually represent the mail transaction initiated by the sender.  The
envelope sender address is usually present, but may be munged via
address-rewriting or other techniques, or may simply be incorrect.  The
envelope recipient address is frequently missing altogether - many MDAs don't
record it - and when present, is quite frequently munged by the local MTA or
MDA.  Sometimes it's an outright fabrication by some well-meaning MTA that
copies the To: header field contents to a field supposedly recording the
envelope recipient.  If the envelope recipient address has been munged or
rewritten, any bounce message you generated will confuse the sender (if it
gets to him) -- "I didn't send a message to that address at all!".
Note, for example, mailing list messages will frequently have message
envelopes that do not match up with the header informational fields at all.
 
2. The "wrong place to bounce" problem I mentioned, in more detail.
  a) sender sends message to [email protected]
  b) MX records for example.org say mail is handled by mail.example.net
  c) sender delivers message to mail.example.net, which accepts the message
  d) sender has successfully delivered message to the proper final destination
     message.  Either it will be successfully delivered, or if a late bounce is
     unavoidable, it will receive a bounce message from the MTA on
     mail.example.net
  e) you retrieve mail from imap.example.net and re-inject it into an MTA on
     your own mail-handling machine at other.domain.org.
  f) receipient address not recognized; generate a bounce message.  
  g) sender receives a bounce message from a other.domain.org, on a completely
     different network than the MX they sent a message to, and says "wtf, this
     is junk", or the spam filter rejects it because it claims to be from 
     example.org but is from a machine/domain that is NOT an MX for that 
     domain, etc.

The MTA running on the MX for a domain should try very hard to refuse messages
at SMTP time if at all possible; if a delivery error (bad recipient, whatever)
happens later, *that MTA* can generate a legitimate late bounce.  But no other
machine/MTA should be bouncing that message after the MX MTA accepts it.

> I thought that the whole point of a multidrop mailbox ...

If you have a "proper" multidrop box, where the unmunged envelope sender and
recipient address of every message is correctly recorded in message headers,
that's great.  It makes things a lot more flexible for you.

> ... is that the internal mail server would process all e-mails ...

Yes!

> ... and generate bounce messages if necessary.

No!  See above; *only* the MX MTA that accepted the message should bounce a
message.  If you retrieve it and don't want to/cannot deliver to a local
address, delete the message or send it to a pile for occasional human review,
or whatever, but do not generate an invalid bounce.

> In fact, the documentation for dovecot-lda does mention that it can generate
> bounce e-mails:

Dovecot typically runs on the MX for the domain, and can generate late bounces
when a virtual user doesn't exist.  That's okay, because *it's still being
generated on the machine that was advertised as the MX for the domain*.

It doesn't matter what your setup is; after a message has reached the end of
its SMTP delivery chain, it should not be re-injected into an MTA (or other
software) in a way that can result in a bounce.

> So at least those guys seem to think that it could somehow make sense.

Not in the situation you have.

> I wonder how else I could generate such bounce messages.

The proper way is to generate local white- or black-lists of addresses - this
can usually be done automatically and periodically - and transfer those to the
upstream mail server so it can use them to accept or bounce the messages.

> I do not really want to replicate all e-mail accounts and aliases on the ISP
> mail server where the multidrop lives.

Unfortunately, being a good internet mail administrator requires some work and
sacrifice on your part.  Spraying invalid bounces around the net just causes
pain for *other* mail administrators, and is just plain unfriendly.

> > Messages should virtually never be re-injected into an MDA via SMTP or any
> > other method (like you would be doing if you generate bounce messages).

Note that was a late-night thinko/typo; I meant MTA here.

Also note that your example of a "vacation" response - or any other
autoresponder, as they're known generically - is *not* the same as a bounce
message.  They have different purposes, process the messages and envelopes
differently, and have different usage conventions.

If you want to use an autoresponder that generates a message "Hey, you tried
to send a message to [email protected], but that's not a valid address" that's one
thing -- you'll confuse and piss off your correspondents, fill your boots --
but do *not* generate bounce messages.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------