Re: How getmail deals with delivery errors and with bounce e-mails

Again, thanks for your help. I feel that I am slowly getting to the bottom of this.


> 1. Many POP/IMAP mail providers use MTA/MDAs configured incorrectly or
 > [...]

I think I can rule out this kind of configuration problems. I have selected one random provider for test purposes, and I have set up a "catch all" 
address there. All e-mails get an extra header named X-Original-To injected with the original envelope recipient address. This seems to be standard 
practice with Postfix, and it matches what other people report. Without such a header, a catch-all mailbox is next to useless, so I guess that most 
providers get this right.

If the provider's MTA+MDA is misconfigured, you can always migrate to a different provider.

I do not have to write a script to cater for all providers. When I select a provider, I can enable the catch-all mailbox, send a couple of test 
e-mails and check the headers.

For similar reasons, I doubt that the Return-Path header would be wrong. I checked on my provider and it's fine.

There no other MTAs or MDAs involved: my provider provides one server with MX records, and that's it. I will be using getmail and internal 
mail-related servers and scripts, but they are all under my control. With your help, I am sure that I will get it 100 % right. If not, I can always 
blame you. ;-)


>    e) you retrieve mail from imap.example.net and re-inject it into an MTA on
>       your own mail-handling machine at other.domain.org.

I do not understand why I should re-inject mail into some internal MTA of mine. I am retrieving mail with getmail, and that would go to dovecot-lda 
(an MDA). There is no MTA involved at all.

I also do not understand why I should use other.domain.org internally. I purchased domain example.org , so I will be using it internally too. Why do 
you want to introduce other.domain.org in the equation?


>    f) receipient address not recognized; generate a bounce message.
>    g) sender receives a bounce message from a other.domain.org, on a completely
>       different network than the MX they sent a message to, and says "wtf, this
>       is junk", or the spam filter rejects it because it claims to be from
>       example.org but is from a machine/domain that is NOT an MX for that
>       domain, etc.

That is not going to happen in that way. When my internal server/logic/script generates a bounce message, it will state that it comes from 
[email protected] . I will have to relay it to my provider's outgoing (SMTP) server, because, due to SPF, it is in fact the only server 
allowed to send e-mails on my behalf. And that is fine by me.


> The MTA running on the MX for a domain should try very hard to refuse messages
> at SMTP time if at all possible;

You are of course right. But this increases administration work and is not really truly a must. So I am exploring the alternatives and their costs and 
risks.


> if a delivery error (bad recipient, whatever)
> happens later, *that MTA* can generate a legitimate late bounce.  But no other
> machine/MTA should be bouncing that message after the MX MTA accepts it.

That is what I still do not understand. It does not look like the exact receiving MTA must generate the bounce e-mail itself. My internal script can 
generate it on an internal computer, at a later time, and relay it to my provider's outgoing SMTP server. The bounce may not come from the same server 
pointed to by the MX DNS record, but it will come from the server pointed to by the SPF record, so I guess it should be fine. In any case, it will 
match the domain the original e-mail sender used.

Or must the bounce e-mail come truly from exactly the MX server?


> [...] or send it to a pile for occasional human review,
> or whatever, but do not generate an invalid bounce.

Manual review is not a realistic option in my opinion. I have better things to do than reading such e-mails.


> The proper way is to generate local white- or black-lists of addresses - this
> can usually be done automatically and periodically - and transfer those to the
> upstream mail server so it can use them to accept or bounce the messages.

That would be nice indeed, but is not realistic. Big providers do not listen to their customers and provide an API to upload such white- or 
black-lists for their mail servers.


> Also note that your example of a "vacation" response - or any other
> autoresponder, as they're known generically - is *not* the same as a bounce
> message.  They have different purposes, process the messages and envelopes
> differently, and have different usage conventions.
> 
> If you want to use an autoresponder that generates a message "Hey, you tried
> to send a message to [email protected], but that's not a valid address" that's one
> thing -- you'll confuse and piss off your correspondents, fill your boots --
> but do *not* generate bounce messages.

That is an interesting subject. What is the real difference between a bounce message, and one that says "this is not a real bounce e-mail, according 
to some unwritten law, but you tried to send a message to [email protected], and that's not a valid address"?

Why would I confuse my correspondents any more than with an "on vacation" response?

As far as I can tell, there is no standard for bounce e-mails to be recognised as such. In fact, relatives and friends often ask me what a bounce 
e-mail means, because it is in English and they do not understand it. If there were such a standard, the e-mail client would probably tell me in my 
locale language that this is a bounce message because I (allegedly) misspelled some e-mail address.

If someone e-mails me and does not get the address spelling right, I would rather send a mixed bounce message in English and in all other languages I 
expect e-mails in (all the languages I speak).

If you are worried that some mail admin would be annoyed because some error information or headers are missing for troubleshooting purposes, we could 
discuss how to generate a proper bounce e-mail with all that information inside. I am sure I can copy and paste enough Perl to get it right.

Best regards,
   rdiez