RE: The open relay problem.

> From: Neil Brown
> Sent: Wednesday, March 24, 2004 9:57 PM
>
>

<...>

> It does seem to make universal SRS more attractive, and the arguments
> against it are hard to sustain.

Thanks for saying so.  I am advocating this as a way to gain extra benefits
while reducing the rather large breakage caused by the present SPF+SRS
scheme.  I am completely in favor of the goals behind SPF, but the design
decision of doing an SPF query on the MAIL FROM: RHS domain set off an
unfortunate chain of events.  This first caused the need for SRS0 to protect
the rewritten address, which caused the need for SRS1 to avoid the second
forwarder becoming an open relay.  With all this extra baggage, the end
recipient still cannot have high confidence that an SPF pass means that the
return-path is not forged and there is no protection against forged DSN's at
the originating gateway MTA since it's MAIL FROM: is not cryptographically
protected.

Let's take a step back and examine what would happen if we made one small
change to that one particular design choice.  If we instead choose to get
the domain for the SPF query from the rDNS of the IP of the SMTP-client,
here's what would happen:

1) The MAIL FROM: is no longer the source of the domain for the SPF query,
so we _never_ need to rewrite MAIL FROM:.  This means that there is no
breakage of existing mail practice whether or not a site publishes SPF
records, does SPF checks, both or neither.  This will greatly ease the task
of convincing the majority of major carriers to adopt SPF, since no changes
are required to their forwarding practices.

2) Non-SRS sites in the chain no longer have any effect on correct
end-to-end operation.  The present scheme breaks down if any of the
forwarders do not implement SRS unless special measures are taken at
SPF-compliant sites.  These measures will probably amount to custom
whitelists of sites to avoid doing SPF checks on, which will be a royal PITA
for site administrators to keep up with.

3) Since virtually all well-configured SMTP-servers today perform an rDNS on
the IP of the SMTP-client as a first heuristic test, we have introduced no
extra overhead.  In fact, they often go further than this by requiring a
match to an A record.  Some people consider this a best commercial practice.

4) Many SMTP-servers already reject connections from IP addresses that don't
return proper FQDN's.  The SPF gurus would have to decide what SPF results
to return for each possible type of rDNS failure.

5) Since rDNS is just as reliable an indicator of domain as the RHS of MAIL
FROM:, the amount of confidence we have in the resulting SPF query and
result is unchanged from the present system.


This is a very large improvement for one very small change in usage.  I
realize that the existing SPF specs and implementations are pretty far
along.  However, if people think about this carefully, the benefits from
this change far outweigh the costs.  In fact, with the challenge of getting
the present SPF+SRS adopted, I would argue that given the option of making
this change, it does not seem reasonable to continue on the current path.


>
> I would just really like a fairly reliable "sender" address, and some
> indication of how reliable it was.  ...


That is why we are all here.  This is a second place where a particular
design decision in SPF led to a less than optimal outcome with regard to
Neill's reasonable expectation.  The particular architectural feature that I
refer to is that each SMTP-server would do an SPF query, test the
SMTP-client against the SPF policy returned by the query, and each
subsequent site trusts the SPF assertion made by the previous site.  There
was no provision for the destination site to verify the SPF credentials of
the originating gateway MTA and thus whether or not MAIL FROM: is a forgery.
This is a problem for a couple of reasons.

First, if there even one non-SPF compliant forwarder in the chain who is
whitelisted at the next hop because they are trusted as non-spammers, that
breaks the chain of SPF assertions.  Unfortunately, the destination site has
no way to know that this has occurred.  All they know is that the
SMTP-client that delivered the message to them generated an SPF pass, so the
MAIL FROM: inappropriately "inherits" the trust that would result from an
unbroken chain of proper SPF checks.  Unfortunately, this is very difficult
to work around, even after you have all the message headers.  There is no
reliable way to know if a site implements SPF.  Publishing an SPF record has
no relation to doing SPF checks.

The second problem created by the forcing the final SMTP-server to assign a
level of trust to the MAIL FROM: that it ascertained from the SPF check on
the SMTP-client is that the quality of the SPF implementations and the local
SPF policies used by intermediate sites are unknown to the final
SMTP-server.  In an environment where all sites implemented SPF _and_ they
all met minimum implementation standards _and_ they all had reasonable local
SPF policies (would we all agree on what those are?), then there would be no
problem.  Without that fictitious environment, you are left with an unknown
level of confidence that an SPF pass asserts whether or not the return-path
is forged.

Fortunately, it is possible to make another change to the SPF framework that
will allow the destination SMTP-client to verify the MAIL FROM: address
without accepting the assertions made by unknown third parties.  That change
happens to dovetail very nicely with the change that I suggested above to
obviate the need for sender rewriting.  The specific change is as follows:
all originating gateway MTA's must SRS0 sign the MAIL FROM: for all outgoing
mail that originates from their private networks, and the SRS0 signature
must additionally contain the local-part of the originating gateway MTA
address.  One possible syntax for this would be:

MAIL FROM:<SRS0=HHHH=TT=sender-local-part=sender-domain=
          MTA-local-part@MTA-domain>

where
   "HHHH" is a SHA-1 hash computed over the remainder of the
      MAIL FROM: address _including_ MTA-domain

   "TT" is a timestamp

   "sender-local-part" is the local part of the original
      sender's address

   "sender-domain" is the domain part of the original
      sender's address

   "MTA-local-part" is the local part of the originating
      gateway MTA's address

   "MTA-domain" is the domain part of the originating
      gateway MTA's address


Now let's see what happens to SPF operation as a result of this change.

1) The entire return-path is protected by the hash function.  If anyone
along the path attempts to modify the return-path, the hash won't verify.

2) It is for all practical purposes impossible to forge a completely new
return-path with a correct hash.

3) All forwarders along the message path can perform SPF checks by getting
the domain for SPF query from the rDNS of the IP of the SMTP-client.  SPF
continues to operate exactly as it does now through the message chain.

4) At the end of the message chain, the SMTP-server can do an SPF query on
"sender-domain" to verify that "MTA-local-part@MTA-domain" is a designated
sender.  Even if there are non-SPF compliant forwarders in the message path,
the final SMTP-server can still verify this key piece of information
independently.

5) At the end of the message chain, the SMTP-server can further verify that
the source of this message was "MTA-local_part@MTA-domain" by doing a CBV on
the MX for "MTA-domain".  Since the MAIL FROM: contains an SRS0 hash, if the
CBV returns "recipient OK", you know that this message originated from that
site.  This test also obviates the need to trust the assertions of unknown
third parties.  It also guarantees that if you later have to generate a DSN,
it will be accepted.

6) This mechanism does not in any way hinder the use of SPF checks at each
hop in the message path, as long as they are done based on rDNS of the IP of
the SMTP-client.  Neither does it depend on any of the forwarders actually
doing SPF checks, though that would be highly desirable.  The final
SMTP-recipient can always verify MAIL FROM: independently, according to
_its_ local SPF policy and have a very high degree of confidence in the
result.

7) DSN's are handled exactly as they are now and go straight to the
originating gateway MTA, as opposed to going to the second forwarder, where
it is relayed back to the first forwarder where it is relayed back to the
originating gateway MTA.  This does not break any existing mail practices.

8) Bogus DSN's cannot have correct hash values, so they will be rejected
immediately after MAIL FROM: with permanent errors.  This makes you immune
to joe-job bounces, bounce spam and virus spew posing as a DSN.  Even
Microsoft mail clients do not retain the MAIL FROM: SMTP information in
their headers, so it will not be possible for viruses to harvest valid SRS0
return-paths to attack.


This is a large set of benefits due to a relatively small change in
protocol.  The main benefits are allowing the final SMTP-server to validate
the MAIL FROM: information regardless of who was in the message path, giving
senders immunity from bogus DSN's and bounces from joe-jobs plus not
breaking any existing mail practices.  There is one caveat to that last
statement: as Neill pointed out, mailing lists that qualify their posts by
MAIL FROM: instead of the From: header will be broken and will have to
change their logic.

Given all of the above advantages, why would we _not_ be willing to
reconsider the current path we are on in favor of a more rational one that
causes less breakage, gives more benefits, is easier to adopt and works well
even if it is only partially adopted?

--

Seth Goodman