Re: How getmail deals with delivery errors and with bounce e-mails

On Tue, 2020-11-03 at 16:09 +0100, R. Diez wrote:
> > If I understood you and the mail terminology correctly, you are not
> actually asking the ISP's MTA, but you are connecting to the  mail 
> provider's IMAP server. Is that right? getmail only supports POP3 and
> IMAP4 after all.
> 
Yes, I'm retrieving mail via POP3. It works well and tends to be
failsafe: I had an incorrect check for acceptable characters in the
sender address (this was in the script called by getmail to pass
messages to Postfix) which returns an error code, thus preventing
getmail from deleting that message from the ISPO's outbox. Once I'd
spotted and fixed the obvious error, the affected message was
successfully retrieved.

Kudos to Charles for supporting error handling options that preserved
the affected message rather than erasing it.

> Are you polling for new mails at regular intervals, or using IMAP
> IDLE?
> 
Yes. I use a cronjob to run getmail every 10 minutes.

> I guess that's fine, but I wanted to rely on the mail provider's spam
> filtering. At least for now. That would save me work, and I trust the 
> provider more than my current mail server skills.
> 
No problem - there is a script run by getmail for every message
retrieved which you need to write. Mine does:

- a check or two on its first 'sender' parameter, which is passed to it
  by getmail
 
- runs spamc (which passes the message through Spamassassin) and passes
  the result spamkiller, a program I wrote, which moves messages marked
  as spam by SA to a quarantine bin and calls sendmail to pass
  everything else to Postfix for delivery

Seeing you don't want to do local spam checks, your version of this
script will be simpler: pass the message to Postfix so it can do its
thing by either rejecting or rejecting the message and passing accepted
messages to Dovecot for local delivery.
   
> Do you mean that getmail runs the version of sendmail that Postfix
> provides?
> 
Yes.

> If so, could you give me an example configuration? That would save me
> some research time.
> 
Dead simple. From my getmail script:

/usr/sbin/sendmail -i -f "$sender" -- MYUSER

Its not in the manpage, but IIRC the final ' -- MYUSER ' is a default
destination if for some reason the message lacks a To: header or doesn't
match any login name.
 
> How does Postfix know whether a destination address is invalid?
> 
From the aliases database set up with newaliases.

> 
> > 4) My master MTA passes inbound mail (after redirection as needed)
> 
> How does Postfix know that a redirection is needed?
> 
The aliases database lets you alias usernames and set a default
username. This has other uses too: for instance I redirect all 'root'
messages to my usual login, so the logreport and output from other
cronjobs end up in my mailbox rather than root's mailbox.

If you haven't already read the Postfix documentation, than I suggest
you do so. Its rather complete and well written. Like all MTAs, Postfix
is extremely configurable but this does mean that's a lot to get your
head round. There;s also a rather good configuration validation tool:
find it, and use it.

Fortunately, once you've got it set up as you want to it be, the config
is very stable - just make sure you have a copy of every Postfix file
you've changed - somewhere outside /etc that is regularly backed up. Its
normal to find that can do a from scratch Postfix reinstall, drop your
saved config files back in place, start it up and find it runs normally.
 
> I am missing a piece of the puzzle here. Are there different rejection
> "levels" or "stages"? I am thinking that the SMTP mail server at my 
> provider may accept a connection, start receiving an e-mail, stop in
> the middle, and tell the SMTP client that the recipient does not
> exist.
>
Possible but unlikely: The mail will normally be delivered to the final
MTA because only that is definitive for a mailbox owner name. What
happens then depends on how that MTA is configured: it may bounce the
mail (bad address, spam, mailbox full), report deferred delivery, or
accept it for the site the sort out. As I said, all MTAs are complex
beasts, so RTFM.
 
>  
> Then the SMTP client (coming from somewhere on the Internet) will be
> the one generating a bounce e-mail for the original Thunderbird user, 
> right?
> 
No - thre final MTA does it.

> Or does the SMTP mail server at my provider always take the whole e-
> mail, and then discard it (because the recipient e-mail address is 
> unknown) and finally generate a bounce e-mail itself?
> 
Depends how its configured, but more likely to bounce it as
undeleverable.

> If I am using a "catch all" mailbox at my provider, I cannot reject e-
> mails straight away (during the first SMTP connection that brought
> the mail) without duplicating all e-mail accounts on the provider's
> mail server, which I want to avoid if possible.
> 
No - by definition they are deliverable: If you only have only one
mailbox at your ISP that all mail will be put there - your ISP has no
knowledge of what mailboxes and hosts exist on your LAN.

> So I must generate bounce e-mails later on. What is the reason behind
> the "you MUST NOT reject" rule? Because that is the only way I could 
> get bounce e-mails working, as far as I can see.
> 
We've covered this: if you use getmail that you MUST NOT bounce anything
that getmail has accepted and this also means your MTA, if you're using
one behind getmail, also MUST NOT bounce anything getmail has accepted.

However, if you have and MTA that accepts mail directly from your ISP
(using port 25, say) then it CAN bounce mail that it doesn't accept.

BASIC RULE
==========
Mail received using SMTP protocol (i.e. MTA-MTA transfer) can be
accepted, deferred (because the mailbox is full) or rejected.

Mail received using POP3, IMAP etc can not be rejected because whatever
program put it in the POP3 mailbox or the IMAP message store ... 
HAS ALREADY ACCEPTED IT.

Martin