Re: So here it is one year later...

On Mon, 2005-01-31 at 10:56 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:
>
> > Forwarding is a common practice within colleges, societies,
> > and many providers.
> 
> Sure, there's nothing wrong with forwarding, each and every
> mail I send is forwarded at least twice, from me to a smart
> host, from the smart host to an MX of the recipient, or to
> an MDA in the case of local users.
> 
> What happens beyond the MX of the recipient is none of my
> business, as you say there's no way for me to know what the
> recipient does.  My "sender policy" cannot cover forwarding
> by the recipient.  If he wants this for some reason, he has
> to use his own MAIL FROM and his own "sender policy".

SPF may say, "Here is a comprehensive address list", when in reality it
can never be comprehensive.  Either the sender policy allows open-ended
inclusion of other locations, or such sender policies may cause mail to
go missing or be refused.  This is a problem created by the sender, and
not the many unknown others using forwarded accounts.

Use of close-ended SPF records presumes excluding mail on the basis of
MAILFROM not matching a sending MTA has value exceeding a disruption it
may cause.  SPF does not prevent spoofing of the domain, as many share
common providers and there is no consensus which identities are checked
against such records (when these records are even examined).  The high
number of potential DNS lookups ~101-201 (requirements based upon
Schlitt's draft) ensures SPF can not be used to protect the network.
SPF represents an increased risk with respect to network security and
protection.

Protecting the domain from being spoofed is a feature safely offered by
digital signature techniques.  I see a real value in this, but yet no
signature scheme offers network protection.  CSV is intended to protect
the network using the same name basis.  

> > With the change being made, the HELO may now be checked
> > against a record found at the zone apex.
> 
> The "zone cut" isn't a new feature, it was in the last SPF
> draft published before MARID.  BTW, Tony's blog mentions that
> CSV now also solved this problem, but I don't find how, it's
> apparently not in the last (now expired) CSV draft.

There is not a major change being made to CSV.  The design team is
defining use of the Port field for Domain assertions.  Such assertions
could include use of signature algorithms, as example.  Use of a
wildcard label is unsuitable for publishing policy, so the prefix used
by the CSV-CSA record is not a deterrent toward establishing domain wide
assertions.  This domain assertion is not constrained to zone cuts.
After lengthy consideration, it was determined this too would be
inappropriate.  This updated draft should be ready very shortly.

> Here's a summary of my SPF experiences in the last 10 months:
> 
> Mar 04: some spammer(s) start to abuse "my" vanity domain, I
>         get about 1000 erroneous bounces / vacation mails /
>         challenges / etc. per day.
> Apr 04: My ISP publishes a sender policy.  Intuition or bug,
>         neither my ISP nor me saw the "minor" problem, that
>         this sender policy didn't cover "my" vanity domain.
> May 04: Wildcard added, now it works even without "zone cut".
>         "Zone cut" also added to the SPF draft replacing an
>         older "match_subdomains" construct.
> Sep 04: 180,000 bounces etc. so far in my mailbox (JFTR, I'm
>         a modem user)
> Oct 04: My spammer(s) got the SPF idea, again zero bounces
>         etc. per day
> Jan 05: So far not a single problem with SPF from my POV, and
>         4*30*1000 = 120,000 avoided bounces etc. since Oct 04.

There needs to be a means to discourage spoofing of domains and the
abuse of bounce traffic.  The MASS, CLEAR (BATV & CSV) efforts are aimed
at achieving those goals without onerous path registration discovery.
While a small domain may accommodate such related problems, security
considerations dismiss value in path registration and thereby the need
to also disrupt forwarding, various list servers, and user's access to
mail.  CSV looks to the administrators of the domain to control access
to their outbound servers, and uses both assertions and reputations to
protect the networks.  CSV does not require providers to make specific
DNS record changes to accommodate their customer's use of independent
mailbox domains.

> > I agree, the overhead for HELO checking would be unacceptably
> > high with SPF
> 
> Where that's the case (it depends on the sender policy) the
> sender could arrange his HELO domains to avoid it.  And if CSV
> has a better solution than SPF's "zone cut" for spoofed HELOs,
> I'd be interested what it is - I'm always curious.

Good to hear.

> > when publishing SPF, expect some of your mail to be lost or
> > rejected due to problems with either SPF et al or Sender-ID
> > algorithms.  This is shameful.
> 
> No legit mail is "lost" with SPF, that's FUD.  While MARID was
> a shameful disaster, one of the few results was, that Sender-ID
> evaluations of SPF sender policies generally do _not_ work.

A well understood problem is not FUD!    

> > those that have published what is now considered too many
> > records may find their mail being rejected.
> 
> If an erroneous policy "worked" with old SPF implementations,
> and new implementations report the error, then it's time to
> fix the old error.

With SPF, there is no means to prevent disruption and provide an orderly
introduction of newer revisions.  A script based language is inherently
complex and rarely stable. : (

> > The SPF timeout imposed still ignores UDP fallback.
> 
> The overall SPF timeout in old drafts was replaced by clear
> processing limits, which don't depend on a vague accumulated
> processing time.

>From the Schlitt's draft:
   MTAs or other processors MAY also impose a limit on the maximum
   amount of elapsed time to evaluate check_host().  Such a limit SHOULD
   allow at least 20 seconds.  If such a limit is exceeded, the result
   of authentication SHOULD be "TempError".

> > Is it reasonable to expect the receiving SMTP server to
> > perform a hundred lookups per message?
> 
> No, that's why it's impossible, you have constructed a worst
> case of 1+10+10*10 = 111 DNS lookups for a sender policy with
> 10 mx directives, each with 10 MX hosts.

The concern is not what most use, the concern is what limits must be
permitted.  The draft requires these 101 to 201 lookup limits depending
upon the mechanisms.

> The average case is probably more like 1 to 4 lookups, I've no
> data to support this guess.  And not necessarily per message,
> if you'd use CSV or SPF or an RBL to block an IP or to reject
> all mails after your HELO-test resulted in a FAIL, then you
> won't test the individual messages in this SMTP session.

By checking per HELO, the rate CSV does a lookup is some multiple less
than SPF, and constrained to a single lookup and not hundreds of
lookups.  This aspect is critical. 

> > This can be safely achieved via the efforts in MASS and
> > CLEAR, but not with SPF.
> 
> Please inform "us" when that's ready and deployed.  Bye, Frank

Gladly.

-Doug