Re: So here it is one year later...

On Wed, 2005-02-02 at 14:07 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:
>  
> > SPF may say, "Here is a comprehensive address list", when in
> > reality it can never be comprehensive.
> 
> Of course it is.  There are at most 512 IPs sending mail from
> whoever@xyzzy to somebody@elsewhere.  The other 254*256*256*256
> IPs don't do this, unless they are forwarding mail from me, and
> then it's something arranged by somebody@elsewhere.  He's free
> to do whatever he likes, but of course he shouldn't check one
> of his relay IPs against "my" 512 IPs.

Today, mail is not checked for a path registration against some
mailbox-domain, and forwarding works as expected.  Now SPF allows any
mailbox-domain to be checked against path registration, according to
Schlitt's SPF draft.  As a result, there is no assurance that a specific
mailbox-domain was checked.  Recipients would be guessing which
mailbox-domain was used to authorize the message, assuming such
authorization was required.  The benefits of this process remains highly
dubious.

On the promise of a benefit, this scheme requires administrators to
institute a program of white-listing servers that forward mail on behalf
of their customers.  Such white-listing practices permit undetected
abuse, when these servers are excluded from the SPF rule-set (see
draft).  This system goes from some mailbox-domain may have been
checked, to perhaps none, even when there is an SPF policy in place for
the sending mailbox-domains.  Of course, there is the vast number that
still do not use SPF.

Forwarding of mail involves three parties, the sender-domain, the
recipient-domain, and the forwarding-domain. 

A) You want the sender-domain to register all servers used, which is
   not in keeping with the normal scale for a DNS response.

B) The recipient domain then must white-list all forwarding-domains
   based upon information derived from their users.

Once the recipient-domain also employs the SPF rule-set for accepting
messages, these forwarded messages may become lost or rejected.  Don't
forget about the change made with respect to HELO, where now even the
bounce may be refused.  The needed feedback to know there is a problem
may have been lost due to the zone-cut change.

Do you still insist you can list all servers?  You seem to be admitting
that you are leaving some of this listing work to the recipient-domain,
in that they must white-list forwarded accounts, some mailing-lists,
perhaps some hotel and network kiosks, etc. etc.  Why not admit that it
is _impossible_ to include the entire list?

> And there at most 512 IPs who could say HELO xyzzy.claranet.de.
> In reality no MTA should do this, but for this minor detail SPF
> isn't good enough, maybe CSV could catch the remaining 512 IPs.
> BTW, can it ?

CSV does not attempt to include all the IP addresses for an entire
mailbox-domain, but instead limits a response to that of a single
server.  There is no difficulty created when more servers are used for a
very large domain, even spanning millions of addresses.  Each individual
response is limited to that for a specific server.  Even the DNS entry
does not deal with addresses, as this process is automated within the
SRV RR type. 

As a mail provider, the HELO could reflect which domain is generating
the traffic (with the cooperation of the sending domain).  If the desire
is to isolate accountability, then the HELO technique could satisfy this
need, while not breaking mail to the same extent as attempting to
constrain mail to some mailbox-domain/IP address binding.

Although this is a function provided by the sending domain policy, a bit
within the CSV record could indicate the HELO will "always" be within
the domain of the MAILFROM.  The reason not to make this assertion is
the same as why SPF should not make their assertions.  It would always
be a lie.  The difference is that CSV scales and could achieve the same
silly goal without abusing DNS.  : )    

> > This is a problem created by the sender
> 
> "v=spf1 ip4:212.82.225.0/24 ip4:195.170.96.0/24 -all"
> isn't "a problem created by the sender", it's simply the truth.

I would say it is only half of the truth.  The other half is the need
for random white-listing to keep from disrupting current widespread and
legitimate practices.

> > SPF does not prevent spoofing of the domain, as many share
> > common providers
> 
> If somebody authorized to use one of these 512 IPs spoofs "my"
> vanity domain, then I'm sure that the abuse desk of my ISP can
> solve this problem very quickly, otherwise they'd risk to lose
> a happy customer.

You may have authorized several providers to send mail on behalf of your
domain, and discovered a problem by way of a poor reputation assessment
or, even worse, your mail being filtered.  You will likely find it
difficult to learn where the problem originated and receive little
cooperation from reputation or filtering services, as they protect
sources.  The several providers may not care whether you remain a happy
customer when you ask them to do additional work.  Perhaps they only
consider Sender-ID a valid choice and you insist that logs be sorted for
SPF specifics.  After all, SPF/Sender-ID ensures the problem is yours
and not theirs.  Very likely, your recourse will be expensive and
perhaps futile.

> And SPF offers solutions for this situation where necessary,
> e.g. some existing policies use ? instead of + for shared IPs.

Providing a PTR list of CSV authenticated domains compared against a
mailbox-domain would achieve these filtering goals without hundreds of
DNS lookups, as potentially required by SPF.  There is much less effort
maintaining names than huge address lists, and this response would fit
within the constraints of DNS.

> > there is no consensus which identities are checked against
> > such records (when these records are even examined)
> 
> 2821-From and -HELO are checked.  Anything else won't work as
> expected.  Of course I can't force you to do "the right thing",
> you could check something else somewhere else, but why bother
> with DNS and SPF if you want some random rejects ?  Try BLARS,
> its cheaper than SPF for this purpose.

Schlitt's SPF draft:
   There are several possible ways that [an] authorization failure can
   be ameliorated. ... Tests against some other identity may also be 
   used to override the test against the "MAIL FROM" identity.

If MAILFROM doesn't work, then try an assortment of mailbox-domains.
You can not say with any certainty, even for those following Schlitt's
draft, that a specific mailbox-domain was checked.  Some chain of trust!

> > CSV is intended to protect the network using the same name
> > basis.
> 
> So far I haven't seen any spoofed HELO xyzzy.claranet.de in
> bogus "bounces to" me, so probably CSV would have done nothing
> at all for my problem.

With a field not often checked, there is little reason to lie.  Checking
the HELO is not disruptive, as it does not affect the content of the
envelope or the message.  Using HELO to establish accountability
protects the owners of the mailbox-domains from being falsely accused
for having sent unauthenticated messages.  Unlike other entities within
the message, without a signature, only the HELO can be authenticated.
SPF assertions assume a fallacious "chain of trust."  Schlitt's draft
makes a mockery of that concept.  

> > This domain assertion is not constrained to zone cuts.  After
> > lengthy consideration, it was determined this too would be
> > inappropriate.
> 
> Can it identify HELO xyzzy.claranet.de as rubbish ?  I've read
> the discussions about SPF's "zone cut" in CLEAR, and the Jan 4
> plus Jan 6 entries in <http://www.livejournal.com/users/fanf/>
> - but that doesn't explain the details.  And I haven't found a
> case where `nslookup -q=ns sub.dom.ain.example` fails to find
> the "zone cut".

There were several reasons for not using the zone-cut.  There is some
blood on the floor regarding this topic.  Complying with the way domains
and zones are used, how software is structured, and what is reasonable
to expect of the server in some odd cases, were the major reasons in my
view.  For the most part, this draft is in its final stages.  The design
team wishes to present their finished version rather than a series of
interim drafts.  CSV is very much a team effort.

> > A well understood problem is not FUD
> 
> Claiming that SPF causes mails to be lost is definitely FUD,

Odd, I would think that the mention of lost mail in Schlitt's SPF draft
would have removed the uncertainty and doubt.  I still think fear is
well advised. : )

> Like the collection of monstrous lies in the PDF:
> <https://antiphishing.kavi.com/events/Conference_Notes/sender_auth_myths_and_domainkeys.pdf>

I would be interested to know what you find in error, besides too many
slides. : )

> BTW, I found that link on the page:
> <http://podcast.resource.org/rf-rfc/index.html#item0003>

Marshall and Andy fail to consider anything other than path registration
in this bit of nostalgia.  They are consistent.

> > A script based language is inherently complex
> 
> Yes, the macro language has its drawbacks, and the TINW in "we"
> had some problems to get it right for some obscure cases like a
> slash in a FQDN used as HELO.   Actually that was in one of two
> relevant articles in MARID (with thanks to Julia and Margaret)
> 
> OTOH Wayne now defined exactly 10 macro letters, 8 are used in
> sender policies, anything else is an error.  That's not too
> complex.  Don't try to convince me that the SPF "explanations"
> are ridiculous, I wouldn't disagree (one of the minor issues I
> hoped to get rid of in the former MARID WG, but here we are).
> 
>   [overall timeout]
> >> From the Schlitt's draft:
> 
> ACK, error on my side, this "implementation detail" is still a
> part of the draft, and maybe I'll discuss it with Wayne on the
> SPF list, it's unnecessary.  Of course there must be a way to
> report and log "local error conditions".

Let me remind you.  A lookup may entail several queries to resolve the
resource.  The timeout for each query typically involves a timeout of 5
seconds with perhaps 2 or 3 retries which double the timeout after each.
Waiting for a timeout to expire is how UDP is prevented from flooding a
network during congestion.  With the 101 lookup limit (higher for
reverse DNS resolution), should an implementation attempt to resolve SPF
in parallel, there may be many outstanding DNS queries not completed
when everything starts anew for the next message.  A slow DNS server
could entail an average of 3 queries per lookup which totals 303 queries
or about 25 minutes per message, without invoking a single timeout or
lookup limit!  This time could be more than doubled by using reverse DNS
resolution. : ( 

> In the lentczner-drafts the optional overall timeout was a very
> important part of "processing limits" and indirectly "security
> considerations", now it's on the same level as say a "sigterm".

This view overlooks the need for UDP exponential fallback!  Think of the
network and not just the program.

> > The draft requires these 101 to 201 lookup limits depending
> > upon the mechanisms.
> 
> If you have some local reasons to ignore a sender policy, then
> just do it.  And if you want a documented reason to state that
> a sender policy is erroneous follow the spec.  The latter is a
> reason to reject the mail with a PermError, the sender policy
> has to be fixed.

Why write a spec and then imply that it should not be followed?  Don't
you see a problem?

> > By checking per HELO, the rate CSV does a lookup is some
> > multiple less than SPF, and constrained to a single lookup
> > and not hundreds of lookups.  This aspect is critical.
> 
> For me it's critical that a forged "mail from me" is rejected,
> and CSV won't do this trick.  If you think that the two lookups
> to check any "mail from me" are too much just don't do it.

SPF does NOT prevent mail from being mistaken as being from you.  SPF
normally checks the MAILFROM.  MAILFROM is rarely seen by the user.  CSV
can authenticate the HELO domain, SPF can only determine that the
message has been authorized by a domain.

When one considers the need to stop spam, reputation is an important
aspect of that effort.  No reputation service should hold a domain
culpable just because a message had been authorized.  With the HELO
authenticated, any abuse will be attributed to the correct entity.  With
SPF, you are using a weapon that does not always shoot straight.  Unlike
SPF, CSV is as accurate as the underlying infrastructure.

> SPF worked for me as soon as some "important players" (defined
> from the POV of "my" spammer) implemented it, it's not critical
> if you don't use it.

Take the longer view and considering what is needed to improve security
and to better protect networks.  There will be less mess than what may
be created by the many potential exploits enabled by SPF.  Although SPF
may temporarily achieve some reductions in bounce traffic, BATV is a
better immediate solution.  MASS and CLEAR still represent viable and
well considered solutions.  SPF still should be discouraged.

-Doug