Re: Spamming...

>On Mon, Apr 25, 2005 at 10:09:57PM -0000, James Craig Burley wrote:
>> Perhaps "injector" or "submitter" could be used to refer to whichever
>> (SMTP or whatever) client initially "injects" the message into the
>> Internal mail system as a whole.
>
>SMTP doesn't have such a concept, of course.

What?  Of *course* it does.  I'll try to clarify what I mean, since my
previous explanation must have been inadequate.

You're running an SMTP server.  It sees an incoming connection from IP
address 1.2.3.4, and the SMTP client on the other end sends the
following combination of commands and data (to which your server
presumably responds with appropriate 2xy/3xy codes):

  EHLO jcb-sc.com
  MAIL FROM:<[email protected]>
  RCPT TO:<[email protected]>
  DATA
  Received: from unknown (HELO not-a-spammer.example.com) (@218.0.0.0)
  	 by smtp.jcb-sc.com with Microsoft SMTPSVC;
	 Tue, 26 Apr 2005 04:55:26 -0400
  Date: [...]
  From: <[email protected]>
  Subject: En14rg3 y3r 1nb0x

  If u w4nt y3r 1nb0x en14rg3d, s33:

    http://192.168.0.253/en14rg3me.html

  Th1s 1s n0t sp4m!!
  .
  QUIT

Now, in *any* discussion concerning authentication of envelope
senders, it is ambiguous to use the term "sender" to describe *any* of
<[email protected]> and either 1.2.3.4 or 218.0.0.0 or
not-a-spammer.example.com.

The "alleged sender" in the above example is <[email protected]>.  The
*immediate* injector, or submitter, is *some* SMTP client at 1.2.3.4.
*That* client claims to have received the message, in turn, from some
SMTP client at 218.0.0.0, which, in turn, claims to be the original
injector or submitter.

Pretty much the only indisputable fact, from the point of view of
*your* SMTP server in the example above, is that the injector, or
submitter, is 1.2.3.4.  (Since SMTP is a two-way TCP communication,
forging an incoming IP address is very difficult, and the possibility
that it might happen is inherent to any claim made on behalf of your
system, so you don't have to hedge your bets on that in this context.)

The main issue with authentication, in an IM2000/SPF/SES context, is
whether [email protected] really sent *that* email.

The first order of business, so to speak, is to determine whether
jcb-sc.com ever delivers its email via 1.2.3.4 or, if that's a generic
(reasonably trusted, if perhaps naive) relay, 218.0.0.0.  SPF and SES
allow that sort of "umbrella" check to be made without necessarily
getting into the specifics of whether user "craig" at jcb-sc.com
delivers email via such a route.

So, when I read documents that purport to address authentication
issues, and they use the term "sender" to describe any of several
sources, I find myself trying to disambiguate it.  For example,
consider this bullet item in
http://ses.codeshare.ca/ses_vs_dk+iim.html:

  - sender has choice of validation servers: UDP, DNS, TCP or SMTP

What is meant by "sender" here?  A casual reading might suggest that
the *alleged* sender -- say, [email protected], an entity that is,
presumably, trusted -- is able to choose how a recipient validates an
email arriving with that alleged envelope sender.

But that appears to not be the case; the *injector*, or submitter --
the *untrusted* entity -- actually controls that choice, because that
is who encodes the envelope sender address in a fashion that tells the
recipient what validation method(s) to use.

(Otherwise, that "sender" could be allowed to provide, as a "choice of
validation servers", a "trust me, I'm the sender!" choice.  ;-)

So, change the bullet item accordingly:

  - untrusted injector gives recipient choice of validation servers:
    UDP, DNS, TCP, or SMTP

Now, it becomes a bit clearer, to someone reading this, that, hey,
maybe there's a problem with that, one that requires further study.
And indeed, there might be, and it does, though maybe the problem is
dealt with by sufficiently limiting this "choice".  (I'm not yet
convinced either way.)

In IM2000, the terminology problem, or confusion, is mitigated by the
fact that there is really no "injector" or "submitter" per se -- there
is a receipt (or message?) notification, which is sent by someone
purportedly on behalf of the actual sender.

But I'd still prefer a distinct term, such as "notifier" or "notifying
party", so readers are more easily able to keep in mind the
distinction between the entity that transmits a receipt/message
notification to a recipient and the entity that actually deposits the
message to which that notification applies in a message store (the
latter being the actual "sender").

>> >I also wanted to respond to the "spammers just buy lots of domains" 
>> >argument.  Whilst in a black and white argument, you are right, people can 
>> >just buy more, if you expand the argument to encompass the overwhelming 
>> >adoption of email as a communicative medium by the entire world, a "globally 
>> >deployed" SPF could have strong value. 
>> 
>> I'm not sure how.  As all the rants, pro and con, about SPF have
>> suggested, either we're going to have a world in which every "known"
>> email address is globally determinable to send email from only a
>> handful of (tightly controlled) origination points (IP addresses), in
>> which case SPF is helpful, or we're not, in which case deploying SPF
>> is a modestly expensive version of playing whack-a-mole.
>
>It's worse than that. SPF only ties *domains* to IP addresses, not E-mail
>addresses to IP addresses.
>
>So if I have account [email protected], I can send SMTP mail with a return
>address of [email protected] or [email protected]; I can send mail with any
>return address using a domain which happens to have a corresponding SPF
>policy (e.g. [email protected] where othercustomer.com sends their
>outgoing mail via the same ISP as me); and of course with no return address
>at all.

Right.

In my view, the set, or "space", of all existing email addresses plus
all routes legitimately taken by email, is best thought of as a
four-dimensional fractal, with one of the dimensions being time, and
with portions of the space being subject to laws not unlike those of
quantum mechanics, in that the space is not determinable until it is
explored, at which point email addresses might spontaneously come into
or go out of existence.  The remaining dimensions pertain to
interconnection topologies -- including not just IP-level routing, but
the topology of DNS and its various delegations as well.

So, this space is constantly changing, impossible to snapshot, and
never reliably coherently viewed by any two (or more) observers,
although it has arbitrarily-large portions that are not chaotic.

SPF and SES recognize the fact that vast swaths of this space are
fairly well-known, explorable, and thus knowable.  Accordingly, they
provide ways for "owners" of those areas (as defined by the DNS
ownership topology) to cordon them (the IP ownership topology) off
using the rough equivalent of fairly simple geometric shapes --
squares, polygons, etc.

But, unless all legitimate owners of the DNS space are willing and
able to tightly restrict, or cordon off, the IP space from which their
emails are legitimately able to originate, spammers will quickly and
easily exploit the unrestricted spaces.

SES offers a smoother, more elegant way for owners of the DNS space to
conform, or "shrink-wrap", their outgoing-IP space on a
per-email-address basis.  That improves the picture somewhat, but not
enough to solve the problem, as there will almost always remain
legitimately unrestricted IP spaces, and spammers will quickly and
easily discover and exploit those, too.

(Besides, we all know that spammers will probably continue to be able
to control substantial portions of *restricted*, yet legitimate,
outgoing-IP space, in the form of zombie machines.)

IM2000 could pretty much obviate the need for much of that, if your
proposal's section 3 was strongly implemented, because it could
require the actual sender to place the outgoing message on a message
store somewhere in IP space, from which the recipient obtains it
*only* by looking up the (alleged) sender's address (domain name at
least, possibly full email address) in DNS to locate, or at least
validate, the message store.

(This assumes message notifications are not themselves a problem.
Hence the need to disallow their carrying potential spam payload, such
as a "Subject:" header -- or, as JdBP puts it, "Senders must not be
given trivial covert channels".  I'm not convinced that this is
possible; isn't an alleged sender's address a trivial covert channel?
Your "sender fingerprint" idea kinda gets at this, but there seems, to
me, to be more of an issue here than meets the eye.)

(And, frankly, I keep forgetting whether it's an IM2000 message
*originator*, or a message *store*, that notifies a recipient
notification agent.  I tend to assume it's the former, and have
probably made related assumptions in previous posts as a result; but
JdBP's proposal specifies the latter.  Since a message store must be
always-online anyway, that makes much more sense; otherwise, mobile
senders would have to find a third party to repeatedly send
notifications!)

So, to keep things simple, we need to ask ourselves how will *any*
system -- SMTP, SPF, SES, DK, IM2000 -- handle a simple case of
likely, but not necessarily *actual*, forgery, as in:

  You have an email FROM <[email protected]>
    DATED 2005-04-26 17:30
    SIZE 79321
    SUBJECT "I have an offer for you!"
    STORED-ON someisp
      WHICH-GOT-IT-FROM someotherisp
      WHICH-GOT-IT-FROM some-laptop-with-a-dynamic-IP

That is, a pretty generic notification.  Feel free to cross out
whatever you think is unwise to include in a notification (e.g.
SUBJECT), but understand that doing so means a recipient of a
notification will have *less* information on which to base any
decision regarding whether to read the retrieve the full message.

Anyway, here's the lineup:

SMTP

  You can't know the date, size, or subject without also receiving the
  complete message.  Ditto for WHICH-GOT-IT-FROM stuff.

  The only thing you can trust is "STORED-ON someisp", since that's
  the injecting SMTP client.  You can leave the message there only by
  explicitly rejecting (temporarily or permanently) the transaction or
  by timing out or disconnecting the TCP connection, which is supposed
  to be interpreted as a temporary rejection.  But you can't reliably
  do any of that until after you receive the entire message.  (Except
  in cases where you can reliably do so based *solely* on FROM and
  STORED-ON.)  And subsequent "notifications" require the recipient to
  receive the entire message each time, in order to discover to which
  message each notification pertains.

SPF/SES

  SMTP, plus you can possibly find out that <someisp> is not
  authorized to deliver email for jcb-sc.com or [email protected],
  assuming you can trust DNS and it's sufficiently populated.

IM2000 [JdBP]

  You can't know the subject without requesting it.  (Maybe you can't
  know DATED without downloading the entire message either?)

  You can trust "STORED-ON someisp" as with SMTP, and you are
  encouraged to leave the message there until the end user reads it
  and decides to signal that he no longer needs it stored there.

  It's not clear any relaying will be allowed, or that it'll be
  supported in the sense of being encoded, so you either trust
  <someotherisp> to deliver email for [email protected] or you don't.

  Therefore, the message may be forged, just as with SMTP.

  Once you request anything further about the message, you "advertise"
  that there's some "interest" on behalf of the recipient.  You can do
  this for your spamtraps as well as your real users, of course.

IM2000 [your hash idea]

  You can't even know FROM, much less anything other than perhaps SIZE
  and of course STORED-ON, without requesting it of the message store.

  This "solves" the problem of a notification containing a trivial
  covert channel, but pretty much requires a recipient to retrieve all
  sorts of information that might as well be in the notification
  anyway.

IM2000 [JdBP, message stores subordinate to sender domain]

  Here, <someisp> is prepended to jcb-sc.com in some fashion and
  looked up via DNS, so a legit sender can tell a recipient which
  *legitimate* message store, acting on behalf of jcb-sc.com, holds
  the message.

  So <someisp> might be "im2k" instead of "im2k.jcb-sc.com", and, to
  designate an upstream ISP, my "im2k.jcb-sc.com" DNS SRV record might
  actually point the recipient to im2000.comcast.net, or to my own
  server, or maybe to both, in case one is unavailable.

  That "solves" the validation problem similarly to your item 3c,
  though it introduces other problems.


Whew.  This really has my head spinning.  But something just occurred
to me, while taking a much-needed break.

We have to avoid relying on sites publishing email-address info via
DNS, as some proposals seem to suggest, and which I'd tentatively
accepted as potentially reasonable.

Why?  Because once you expose email addresses to DNS, you expose *all*
DNS nameservers to brute-force pharming operations that spammers will
use to discover your email addresses.  (After all, it's one thing to
counter a pharming operation on your *own* server -- e.g. run
qmail-smtpd unpatched -- it's quite another thing to convince every
admin who runs a caching nameserver to do so for you!)

This is similar to the problem, which I posted about earlier,
surrounding attempts to blacklist email addresses.

Any email system *must* be designed to assume that *most* sites will
want to keep *most* of their email addresses private, even though the
corresponding mailboxes (outgoing and incoming) may be public.

The only way a site can do that is for its *own* servers to have sole
responsibility for validating its email addresses.

DNS exposure and blacklists work against this, because they tend to
move responsibility for maintaining a site's list of email addresses
onto other entities.  (In particular, a blacklist for forged email
addresses is easily destroyed by spammers, simply by feeding it an
infinite supply of such addresses.)

There is, however, utility in these techniques being available, which
any new system (IM2000, SES, whatever) might offer.

In particular, a site (domain-name owner) might *choose* to publish
certain email addresses, including information such as whether an
address belongs (or recently belonged) to a known spammer!

After all, we want ISPs to be able to afford to efficiently cater to
as many users as possible, meaning a few spammers *will* get through.
Why not give ISPs an easy way to notify the public about such cases,
if they so choose?  (This is even more useful in an IM2000-like
world.)

So, for example, my jcb-sc.com site certainly publishes the fact that
[email protected] is a valid email address.  And that he (I) is (am)
certainly *not* a spammer.

But the owner of example.com might choose to publish not only
[email protected] and [email protected] as valid public email
addresses, but [email protected] as a public email address of a
*known spammer*.

Addresses such as this, along with that sort of information, can, I
think, safely be put in DNS and in third-party blacklists.

(Generally, however, its best for blacklists to "expire" information
in a fashion similar to how DNS does it, so once a site decides to
stop advertising [email protected] as a known spammer, within a
few weeks or months, the address could be safely reassigned to a new
sales guy with a better attitude, or the same guy post-LART.)

-- 
James Craig Burley
Software Craftsperson
<http://www.jcb-sc.com>