Re: A new SMTP "3821" [Re: FTC stuff...........]

On  November 25, 2004 4:07 PM at "David Woodhouse" <[email protected]>
asserted:
>
> On Tue, 2004-11-23 at 12:09 -0500, Andrew Newton wrote:
> > On Nov 23, 2004, at 10:22 AM, Hector Santos wrote:
> >
> > > What Alan is suggesting, for which I strongly agree, is that the focus
> > > in solving the "problem" needs a revisiting of the SMTP model or more
> > > specifically, the functional specification.
> >
> > That has been and still is the basis for all the disagreements in this
> > area.
>
> The nature of Alan and Hector's belief is such that it could only ever
> really be disproven, and never proven -- even it if _is_ in fact true.
>
> The existence of a solution which does not require the world to be
> changed wholesale would disprove their theory that such change is
> required. On the other hand, the lack of such a solution merely shows
> that _either_ there isn't one, or it just hasn't been found yet.
>
> As it happens, I believe that multiple such solutions _have_ been found.
> They just need the details to be ironed out and the deployment to pick
> up. There is a lot of merit in all of CSV, SES, BATV, DK, IIM and the
> other solutions being discussed, and very little evidence that wholesale
> changes to existing practice are required, such as the changes which
> would be required by SPF or SenderID.
>
> Thus, I believe that anyone who seriously wants widespread deployment of
> a scheme to reduce the amount of faked email should put their energy
> into assisting one of those schemes, and only if they all turn out to be
> impractical should we revisit the possibility of requiring wholesale
> changes to the existing email infrastructure.
>
> There seem to be far too many people with a personal attachment to some
> particular scheme, who don't really seem interested in pursuing which is
> _technically_ more appropriate. I'd like to see a little less emotion,
> and a little more rationality.


Very good suggestion; let's get some rationality into this debate.

Let me bring to this august forum a debate which was happening in 'another
place' and see if we can re-visit it - because we need to find some kind of
resolution for it.

The problem is related to forwarding, and its interaction with Sender-ID (and
with SPF-Classic). I'll discuss the problem w.r.t. SPF (-Classic, because I
understand that better).

Please understand, I'm not inviting a reprise of arguments which have taken
place elsewhere, I'm trying to lift the level of debate to one of process
related to the 'definition' of SMTP.

I'll first just re-state the problem and give it bounds.

The problem arises with forwarding which undertakes the following sequence of
actions...

An entity does not wish to make widely available his 'true' eMail address of
[email protected]', he employs a forwarding agency. He publishes the address
[email protected].

A message originator  sends

HELO origin.com  MAIL FROM: [email protected] RCPT TO:  [email protected]

Now forwarding.com receives the content, consults its client database or
whatever, and places the content in a new envelope, sending it as

HELO forwarding.com MAIL FROM [email protected] RCPT TO [email protected]

Note that if the second message cannot be delivered to the [email protected]
mailbox, a DSN 'bounce' generated by end.com gets sent direct to
sender:origin.com.

The bounce has details of the forwarding arrangements which 'sender' knows
nothing about, and 'forwarding.com' gets no opportunity to know that its
forwarding arrangements with the recipient are not working.

Now here's the central technical problem:

Suppose that origin.com has published an SPF policy.  That policy knows nothing
of forwarding.com and does not include any of forwarding.com's hosts among those
authorised to send on behalf of origin.com.

end.com implements SPF testing. It. too knows nothing of forwarding.com (so
cannot white-list it).

The forwarded message fails the SPF test, because the MAIL FROM claims that _the
second_ message is from origin.com, whereas inspection of its IP address shows
that origin.com has not authorised it to send message on its behalf.

Now, please don't reply by suggesting work-arounds, etc., or by saying that
scheme PQR avoids this problem. That's not what this message is about.

Let me focus on the heart of the debate, and I'm doing my best to be fair to
both sides here, and to use neutral language.

Still in the spirit of 'neutral language' , let me use the terms 'conservative'
and 'progressive' for the two positions I am aware of.

The conservatives look at the above example and say
"SPF is breaking the mail system. Forwarding has worked perfectly well in the
past; it is SPF which is changing; it is SPF which is wrong.  SPF should not be
deployed".

Again, let us ignore for now the possible work-arounds - just concentrate on SPF
in SMTP.

The progressives say:
"The world has changed, there are threats now which were never envisaged when
the SMTP protocols and practices were first deployed. Changes of some sort or
another _have_ to happen to regain the utility and trust that SMTP deserves."

The conservatives reply: "But not using SPF - SPF breaks forwarding".

To which the progressives reply:

"No. Actually, if you look at the SMTP architecture, as can be
reverse-engineered from the relevant RFCs, it is forwarding which is breaking
the SMTP architecture.

"forwarding (as per the above example) involves the transmission of two discrete
messages, with different envelope recipients. Yet what the forwarder is doing is
using the MAIL FROM of the first message when it sends the second message.

"It is wrong" (say the progressives) "for forwarding.com to use MAIL FROM
[email protected]. SPF (rightly) detects this as an unauthorised use of
'origin.con', the second message is declared a forgery.

"It is also wrong" (say the progressives) "and has always been wrong - long
before SPF - that bounces are sent direct from end.com to origin.com.
forwarding.com should handle the bounce. Only if it cannot resolve the problem
should it notify origin.com, and it should not do this by sending a 'bounce'
back to origin.com, since. in the terms of the SMTP 'contract' to
'deliver-or-report' the message was correctly delivered to its intended mailbox
of [email protected].", so no (2821-level non-delivery) report is
appropriate."

"Foul" cry the conservatives. "There was only ever one message involved. SMTP
has always supported more than one 'hop'. This 'two messages' business is just
not true.  Forwarding has always worked in the past, using the SMTP protocols.
There is nothing wrong with forwarding."

Some progressives propose to invite all forwarders to modify their practices so
as to conform to what they understand to be the SMTP architecture - and various
schemes have been suggested to help then do this in sensible, practical ways.

Conservatives accuse the progressives of mandating "wholesale changes to
existing practices" and requiring "changes which must be implemented
ubiquitously".

In this particular situation 'ubiquitous' (OED: everywhere or in an indefinite
number of places all at once) is an exaggeration.  It is only those forwarders
who re-use the originator's MAIL FROM who have to change, no-one else does; and
the progressives would claim that this is simply asking the forwarders to now
conform to the SMTP architecture.

Now, continuing in my attempt to be fair, let me withdraw any implication that
the conservatives are against any change whatsoever.  I'm using 'progressive'
and 'conservative' purely in the context of this one topic.

There are all sorts of problem-resolution schemes being proposed, and those who
oppose one may be a strong supporter of another.

But their is no ideal scheme on offer, anywhere.  All schemes will have plus and
minus points, all will require change of one sort or another. All change must be
evaluated rationally.

To return to the specifics of SPF and forwarding, the core question I wish to
raise here is this:
-------------

How, and from where, would one get an 'authoritative' ruling about the
conformance or non-conformance of  forwarding (using the origin's MAIL FROM with
a new RCPT TO) to the extant SMTP architecture?


Chris Haynes