Re: Comments on IM2000

>On Thu, May 05, 2005 at 06:46:50AM -0000, James Craig Burley wrote:
>> >Sure, any blacklist is subject to that kind of attack. Blacklists which run
>> >on top of the DNS benefit from distributed caching.
>> 
>> I've previously posted, at some length, about the dangers of relying
>> on distributed (DNS-style) caching as an effective solution to the
>> problem of giving attackers direct ability to trigger database lookups
>> (and especially inserts) based on arbitrary keys.
>> 
>> (IMO, distributed DNS-style caching is to this sort of attack as more
>> CPU and RAM are to a brute-force implementation of an NP-complete
>> problem such as TSP as the problem sizes increase.)
>
>My theoretical counter-argument to this is that if you can cripple the DNS
>using this approach, then you're saying that the DNS itself is open to a DoS
>attack. That may be true, but then that is an entirely separate problem and
>exists whether or not it's used for blacklist lookups.

That's not a counter-argument, IMO, since DNS *is* open to a DoS (or
DDoS) attack.  At least, that's my understanding.

Therefore, anyone who wants to bring down DNS can do so, but to what
end?  To keep people from reaching web sites (to which their local DNS
caches might successfully refer them anyway)?  To keep people from
receiving email sent via SMTP after MX lookups?

Once you "overload" the DNS system as a point of failure for a
callback mechanism (a la SPF, SES, IM2000, etc.), however, it becomes
a much more tempting target, and also easier for an attacker to focus
on a *specific* point on that target.

Why would an attacker do that?  To defeat the utility of callbacks
that validate incoming message notifications (and related
information), so email recipients can no longer rely on such
techniques to distinguish senders of desireable vs. undesireable
email.

>My pragmatic counter-argument is that DNS-based blacklists exist already,
>and work. Spammers *do* attempt DoS operations periodically against those
>blacklists - against their DNS infrastructure but also against their web and
>mail infrastructure and anything else they can attack. These attacks are no
>different to normal DoS attacks and are handled in the way such attacks are
>always handled.

And they sometimes succeed.  I haven't kept close watch, but my
impression is that several blacklists of various sorts have, in fact,
been shut down (after being partially or fully incapacitated) as a
result of such attacks.

Those who *relied* on such blacklists to decide whether to accept a
given incoming email were SOL as these attacks became more successful;
their false positive and/or false negative rates increased
substantially, or so I would assume.  (Else they didn't really *rely*
on those blacklists.)

If IM2000 relies, as a *technology*, on blacklists or their equivalent
(and third-party message stores are darn close to equivalent, to the
extent they're dragged into the mess needed to defend blacklists
against attack), then IM2000 can be shut down nearly across the board
by spammers attacking blacklists.

>Anyway, I'm not wedded to the idea of using DNS as a blacklist lookup
>mechanism, but I do think that it's a good idea to use a well tried and
>tested mechanism.

In *general* you're correct.  However, it's *not* a good idea, from an
engineering point of view, to *overload* a mechanism that was not
originally designed to bear such a load.

The best counter-argument to this I can think of is "well, the
Internet as a whole was not really engineered either, so let's just
keep adding things onto it and learn from what breaks and what
doesnt'".

IM2000 advocates, as well as advocates of other callback and/or
blacklisting schemes that rely on external third parties, should be up
front about the potential downsides along these lines.

Remember, whenever IM2000 is broken by spammers, people will just fall
back to SMTP, and IM2000's reputation will keep taking hits.  It'll be
very difficult to counter that by saying "well, the problems aren't
really with IM2000 itself, but with third-party blacklists, with DNS,
etc".

In the meantime, ordinary users will be increasingly upset that "the
Internet keeps slowing down" because the upstream DNS caches upon
which they rely (unawares) for much of their browsing are thrashing
about just dealing with callback and/or blacklist requests.

>> Whoops, good catch!  Lemme try that again:
>> 
>>    MAIL FROM:<[email protected]>
>>    RCPT TO:<[email protected]>
>>    DATA
>>    [...]
>>    .
>> 
>> Does that make more sense?
>
>Yep, although this spurious info will be hidden in a Return-Path: header and
>unlikely to trouble the recipient.

I think my original point was that, with IM2000, it *has* to be shown
in order for the end user to make a reasonable decision, and with any
similar system implemented via SMTP (a la greylisting, or
ecrulisting), it has to be shown as well.  (Except with SMTP, there's
nothing *but* the envelope sender and recipient, plus the injecting
SMTP client's TCP/IP info, to show, unless the server accepts the
entire message, which isn't necessarily how greylisting is
implemented.)

>> >Spurious notifications are unlikely to be sent often, since they don't
>> >achieve anything apart from a waste of resources, so I think it's unwise to
>> >build a separate infrastructure to filter them out, one which won't be
>> >tested very often.
>> 
>> In essence, a notification becomes a handshake between a message store
>> and a recipient (agent).  That could be handled solely via a TCP
>> connection a la SMTP, in which case the client's IP address can't be
>> (easily) forged.
>
>Yes. But in IM2000 the notification is "push", and it does not necessarily
>originate from the message store. The sender of the notification can't forge
>their source IP address, but that doesn't stop them sending a spurious
>notification, and the notification doesn't have to come from the message
>store itself. These two combined mean that you need to validate it.
>
>But that does make me think of another option. Suppose all notifications
>were forced to come from the originating message store. If you want to
>forward a notification to someone else, you do it by asking the originating
>message store to do it for you. Hmm... I'd have to think if there would be
>any benefits in that.

I think JdBP's IM2000 proposal, which is much more detailed than djb's
and which commits to certain design decisions, commits to this one as
well: *all* notifications come from the originating message store.

Whether that permits a recipient to simply forward a notification to
another party without notifying the store depends mainly whether the
request for the contents of a message include the recipient's IP
address (or similarly unique identification) in the key or
authorization for such a request.  If not, forwarding of notifications
is trivial; if so, then, yes, the *store* would have to handle
forwarding at the request of a recipient.

>The SMTP equivalent would be a kind of 'redirect' response to an incoming
>message. I vaguely remember the protocol actually has a response code for
>that, which nobody actually implements. Ah yes, RFC 821:
>
>      S: RCPT TO:<[email protected]>
>      R: 551 User not local; please try <[email protected]>
>
>RFC 2821 (3.4) advises caution again this, and SMTP clients are allowed to
>treat this as a bounce, which everyone does.

Heh.  I didn't recall seeing that.  But you're right, it's there!  I
gather the grammar of the message accompanying the response code is
inadequately defined for any clients to generally rely on such
messages, however.

>> >Except that at the moment, an SMTP callback only verifies that the address
>> >exists, not that the mail you're trying to receive was sent by that person.
>> 
>> Well, yes, but what I was referring to was that such callbacks were
>> considered highly annoying by (sysadmins for) victims of joe jobs,
>> weren't they?
>
>A callback to verify isn't annoying, since it doesn't deliver a mail - but
>an actual delivered bounce is, of course.

I suppose it can be annoying if spammers use it as a channel to do
dictionary attacks.  If callbacks necessarily validate *messages* as
well as *addresses*, that solves one set of problems, but creates
another, as we've gone 'round and 'round on that already.

>> IM2000 can (and IMO will) make them *cheaper*, but will they be cheap
>> *enough* to be sure *all* sysadmins will find such callbacks, which
>> appear to be intrinsic to a deployable IM2000 (*replacing* SMTP),
>> acceptable?
>
>Some people argue that callbacks are "too expensive". I don't think there
>are any hard economics to back that up; they just want to minimise their CPU
>and network overhead, which is fine, but then the cost of handling all that
>spam must be pretty high too. I don't think the overhead of a TCP socket
>setup is particularly high.

Again, it's not *scalar* overhead that's the real problem.  It's
blowback that's substantially out of proportion with regard to a given
host's *actual* use of email.

A given host *should* be "big" enough (CPU, RAM, storage, bandwidth)
to handle its *legitimate* incoming and outgoing email needs.

"Big" enough includes costs inherent to each legitimate incoming email
and each legitimate outgoing email.

Since, these days, each legitimate incoming email includes a cost for
dealing with incoming spam, hosts typically have to be "bigger" than
would otherwise be necessary.  (Obviously this is true for hosts that
send *outgoing* spam -- hence the phenomenon of 0wned machines.  ;-)

The problem with callbacks is potentially actually much larger than
that with bounces.

With bounces of joe jobs, a host must also cope with incoming email
that represents such bounces.  In a sense, this is part of the cost of
having *outgoing* email in the first place; if the host doesn't have
responsibility for any domain names from which email is ever claimed
to eminate, it presumably won't receive any joe-job bounces.

And since bounces are expensive for senders to process and send,
joe-job bounces have an *inherent* property such that sites that might
otherwise mindlessly send them are incentivized to not do so, because
each joe-job bounce it sends presumably represents some UBE *it*
accepted.

Now, with callbacks, you've not only de-incentivized those third
parties, you've actually *incentivized* them to rely *solely* on such
callbacks instead of other, more localized and/or dedicated, measures
to block incoming UBE.

So, now your typical host must be built out such that if it has
responsibility for *any* outgoing email (i.e. hosts a domain name), it
is going to be able to cope with a *deluge* of callback requests that
cannot feasibly be otherwise dealt with.  (The senders of those
requests have no incentive to stop sending them.  And they have no
*other* way to reliably disambiguate email legitimately
vs. illegitimately coming from that typical host.)

>> Instead, what if the MUA periodically contacted the recipient (or the
>> same store-and-forward, or upstream, relay) and asked it "how is
>> delivery of that message progressing?".
>
>I think that's almost the same, except you've turn an active notification
>into a passive polling.
>
>If the client polls, then it will need some sort of message key to query the
>state of a particular message. In that case, the server could have made an
>active notification using the same message key.

Right.  It's *almost* the same.  But it doesn't require a reverse
lookup a la a callback at all.  The *sender* remains responsible for
asking about messages.

That's kind of like how IM2000 makes the sender more responsible for
message *contents*, except the contents move (quickly) downstream
along with the notification, while the sender remains responsible for
tracking the package, as it were.

>> It would have to allow not only for answers equivalent to the
>> 2xy/4xy/5xy responses we presently have, but for "it appears to have
>> been lost, please resend", since the MUA would have responsibility for
>> that message.
>
>So, whilst the message itself may have been lost, the *state* of the message
>must be retained for the client to poll at some later stage - and
>potentially must be kept for a very long time.

Indeed.  In fact, this is exactly what the end-to-end principle
implies anyway.

>> >It can be done, but in general it isn't.
>> 
>> This is the line of reasoning you use to which I most object.
>
>I think it isn't done because the economic cost is extremely high, the
>immediate gains are virtually zero, and the gains are only realised when
>everybody *else* does it. In other words, there's no business case. Who
>wants to spend money on a project which will only deliver benefits if
>everybody else joins in as well - when we know perfectly well that there's a
>large subset of people on the Internet who won't?

Right.  That's how I view IM2000.

>OTOH, implementing something like SES/BATV on your own server does have a
>business case: you do some work, and you immediately stop joe-jobs coming
>into your mailboxes. That's much easier to take effect. You can then
>leverage other features on top of that (such as callbacks to validate
>envelope senders)

Exactly.

>> >(1) The ISP maintains a local list which ties foo123 <=> [email protected]
>> >Every time a new user comes along, they will have to state which E-mail
>> >addresses their account is allowed to use, *and* prove to the ISP's
>> >satisfaction that they do in fact own those accounts.
>> 
>> This isn't a problem if the envelope sender is required to be
>> [email protected].
>> 
>> If it can be anything arbitrary, then IM2000 will have similar
>> problems, correct?
>
>IM2000 (as in current document) requires that the 'envelope sender' be the
>actual mailstore hostname and account ID on that mailstore - otherwise it
>simply doesn't work.
>
>My own suggestion is that the sender ID be an E-mail address tied by design
>to the same mailstore account.
>
>I was trying to get at the issues which would occur if you go down the route
>of stay-with-SMTP-and-try-to-patch-it-up.

Okay.  I think I might be losing track of the conversation;
essentially, what I'm doing (or trying to do) is comparing the effort
needed to roll out IM2000 with that needed to incrementally improve,
site by site, SMTP to do *mostly* similar things.

>> IM2000 eliminates bounces, but only *after* SMTP is basically
>> eliminated from the Internet.  Until then, IM2000 will have the same
>> basic problem: a message store will have to validate whatever
>> *arbitrary* sender address is assigned to a user's outgoing message,
>> or worry that the user might send joe jobs.
>> 
>> Even once bounces are eliminated, how does an IM2000 *sender* actually
>> know whether an outgoing message has been received, if she's moved on
>> from her cybercafe?
>
>Well, the cybercafe probably wouldn't give you a mailstore account (unless
>you're a regular customer). You would use a mailstore at a third party (a la
>"hotmail") or your home ISP.
>
>Now, the same could apply today: the cybercafe could simply not install a
>mail relay, and block port 25. That would force you to connect over port 587
>to your home ISP, and use SMTP AUTH to submit a new mail.
>
>There's a chicken-and-egg though: most ISPs don't support this service; so
>cybercafes can't block port 25; and so ISPs have little incentive to add it.
>
>If we wanted to move to a new SMTP world order, it's hard to see how to
>force it to happen. You could take a radical approach: blacklist everyone
>who doesn't implement the new world order policies. In that case, you will
>find yourself unable to talk to almost anyone (in which case you might as
>well set up in the IM2000 world instead).
>
>But there's a more fundamental problem: how can you *tell* whether a
>particular SMTP relay server complies with the "new world order" rules or
>not?
>
>The new world order says things like:
>- you cannot submit mail on port 25 based on purely your IP address. You
>  must always use SMTP AUTH when submitting new mail.
>
>However, you can't actually *test* this without going to the site, setting
>up a machine on one of *their* IP addresses, and testing it for yourself.
>
>That's one reason why a completely new protocol actually makes sense, in my
>opinion. Mail received via the new protocol must be complying with the
>new rules, if the protocol itself requires those rules as part of its
>fundamental mode of operation.

I think you already answered your questions above, when you suggested,
in a previous email, that, in a mixed world (IM2000 and SMTP),
recipients would tend to give higher priority to messages arriving
(entirely?) via IM2000.

On the SMTP side, it's not all *that* hard to prioritize incoming
messages based on a recipient's perception of the trustability of
upstream relays, including whether SMTP AUTH was used.  (But in cases
where use of AUTH isn't reliably reported in "Received:" headers,
either the relay in question always requires AUTH and thus acquires,
over time, lots of trust, or will have to find other ways to be sure
it avoids the problem of being a mixed source of UBE and desireable
email and, thus, untrusted.)

>> >- There are many legitimate reasons for using variable envelope senders;
>> >  e.g. I may be sending using VERP, SES, SRS or BATV envelopes.
>> 
>> I don't know BATV offhand, but the rest strike me as kludges to work
>> around SMTP's built-in limitations.
>
>Absolutely. And even if we design an SMTP new world order, then either those
>kludges will still be needed, or a whole raft of extensions to SMTP will
>need to be added such that they are no longer needed.
>
>That would put another barrier to joining the NWO: you'd first have to
>upgrade your mailserver to SMTP++.

Yup.  I'm still not sure how best to proceed.

>> I don't know what BATV is offhand -- "Bounce
>> And Tell Vinnie"?.  ;-)
>
>"Batavia Access Television", according to Google. But the ninth hit is the
>one I meant: "Bounce Address Tag Validation (BATV)"

Ah, right, and, indeed, it shares with VERP, SPF, and SES, the
property of encoding the envelope sender to work around the lack of
another crucial field in an SMTP message notification (mainly, the
lack of a machine-encoded "in reference to" field).

>> >> I'm not sure why that's hard for SMTP -- aren't some ISPs doing that
>> >> already?
>> >
>> >Because
>> >- mail relays generally don't have an unambiguous indication of *who* the
>> >  person is sending through them (without SMTP AUTH, or some sort of
>> >  callback into a RADIUS accounting system)
>> 
>> Huh?  Source IP isn't enough to disambiguate a paying customer?  Or do
>> you want to include roaming users?
>
>With dynamic IP pools, source IP doesn't tell you *which* paying customer,
>only that they are *a* paying customer.

Really?  Why not?

>> What does IM2000, or any new system, offer that makes this a
>> non-issue?
>
>That you have to authenticate to the mailstore to submit a new message. It
>doesn't work without it.

Oh, okay -- so it is pretty much the same overall effort as moving to
SMTP AUTH, modulo the fact that SMTP AUTH got their first.  ;-)

>> My concern is that IM2000 isn't *enough* permanently harder, and has
>> an excessively high built-in failure rate for *legit* email due to its
>> additional points of failure, to justify the expense of rollout.
>
>You may well be right; I've not made my own mind up on that either, and
>certainly there may be ways to improve it.

I'd rather we get it right and deploy it before some proprietary
vendor does something similar *enough* to take over much of the useful
email-address space on the 'net.

>> After all, an outgoing SMTP queue *can* be structured so it is
>> per-user.
>> 
>> (I'm not sure an IM2000 mail store has to be so structured, offhand,
>> by the way.)
>
>Well, for example you don't want one user to be able to query the status of
>messages sent by another user, or to delete messages sent by another user.
>That means you authenticate to the mailstore, and have control only over
>your own messages.

This is an issue with my proposals (including ecrulisting) as well,
and in fact it might be an issue with greylisting too, though I
haven't studied it in detail.

(By "issue" I mean everything might seem to work as long as the
proposal is not widely-enough used to come under focused attack, but
could be a show-stopper otherwise.)

>SMTP *could* have per-user queues, but what I really meant was per-user
>*state* giving the history of messages sent. That's what I meant by a shared
>database, in the case of a cluster.

Yes, SMTP definitely *needs* that once ecrulisting and/or tracking are
added, and probably *wants* that even today as greylisting and other
anti-UBE measures increasingly tend to stop, or at least slow down,
deliveries of legitimate email.

>> On the other side of the equation, I personally hold out hope for a
>> more distributed, jungle-like world of clients injecting email
>> directly into recipient's systems (via their servers), since I believe
>> that, in the long run, that's a more robust model for a mode of
>> communication that *has* to be robust to be useful.
>
>I see where you're coming from - more peer-to-peer and less centralised. The
>instant-messaging model fits well here.

Indeed.

>The problem comes with spam and identities. Whatever identity I choose for
>myself (whether it be a domain name or a cryptographic key), if I can
>generate new ones at will, blacklisting becomes completely ineffective. You
>then have to consider options like limiting each person to one domain or
>having crytographic keys signed by the government, neither of which I like.

Agreed with the latter.  I'm not yet convinced that the at-will
generation of new identities is *solely* a problem; IMO it can be
leveraged as part of the solution.

>Any sort of closed-user-group model doesn't have this problem. But if you
>wish to be able to accept mail from strangers on the Internet, then it is a
>major problem. Some FUSSP suggestions are basically ways of making it
>difficult for strangers to introduce themselves to you - such as "hashcash".
>Challenge-response systems are in that class too (but they won't work once
>spammers build systems to response automatically to the challenges)

Right.  My ideal is that incoming email from total strangers would
*tend* to be immediately accepted, and my *proposals* revolve around
the notion that, prior to the human recipient actually reading a given
message, the sending stranger's MUA repeatedly requesting status (or,
in ecrulisting, resending the message) would, in the absence of other
information on the tendency of the sender to send Bulk email, tend to
*increase* the priority of that message as seen in the recipient's
list of pending messages.

That doesn't seem to *punish* anybody for sending legit email.  In
many cases, such emails are about as instantaneous as can be, since
there's no reverse lookup (not even rDNS), no callback, etc.  If the
recipient doesn't care for the message, they can either signal its
unacceptability to the sender, or instruct an upstream entity to
consider the message Unsolicited.

Since it would be mostly a matter of automation to infer whether it is
Bulk Email as well, that means that a sender of UBE becomes recognized
as such on a per-site basis.

And the sender of legit email naturally *wants* to know the progress
of the transmission, and, since her local DNS cache probably already
has the destination MX in it, repeated lookups to request status
updates are little different, in terms of the hit on the DNS system as
a whole, than users hitting "Refresh" or even just clicking on a link
that hasn't yet gone stale in their browsers.

That covers the *legit* cases of email, plus the general means by
which UBE is detected as such.

(Do I want a whole new protocol that makes all this even *better*?  Of
course!  ;-)

Now, *illegitimate* email can still be blocked by pretty much all the
"usual suspects" -- techniques like RBLs and SPF can be used -- but
they system as a whole is more flexible, so these techniques can
actually be used by the end user's *MUA* without the *sender*
necessarily being aware that the message progressed that much further,
beyond the SMTP server just upstream from the MUA, or the relay just
upstream from that, etc.

So, here we come to the nub of the "solution" I'm proposing, since it
is crucial to know whether it might actually work.

It depends on white hats having substantial resources and a
willingness to use them.

In essence, just as black hats can exploit the "fractal" nature of the
"timespace" of email as a whole -- as I pointed out earlier, there's
really no way to effectively snapshot the universe of legitimate email
addresses, legitimate domain names, etc., though at least the IPv4
space is theoretically manageable (but not the IPv6 space) -- white
hats can exploit that as well.

So, by sites individually choosing to deploy some spamtraps, and
perhaps using RBL and other technologies not to necessarily *block*
incoming messages but, rather, to simply *ignore* them and let the
senders repeatedly inquire as to their status...

...*receivers* of UBE turn their systems into "black holes" for mail
that actually *is* Unsolicited Bulk Email.

Again: if it's not Unsolicited, senders who realize their messages
aren't getting through will quickly find another communications
channel (which is something SMTP doesn't reliably provide information
on at all anymore), assuming they aren't told within, say, 24 hours
that their "To:" email address had a typo and was thus unrecognized.

If it's not Bulk, a receiving system can easily recognize that it
hasn't seen 1M messages coming in from that particular sender's IP
address in the past 48 hours, most or all of which have gone unread or
marked as Unsolicited by their readers.

Accordingly, the receiving system can increase the priority of pending
messages, take responsibility for them, or at least give senders more
useful information as to why delivery hasn't yet occurred.

This approach is a bit like IM2000 in that it gives the sender more
responsibility for "caring" for outgoing messages.

But it avoids the necessity of making the messages *themselves* sit on
sites remote from recipients.

So it "punishes" senders of UBE in that they either become (well,
remain) careless senders -- which recipients can more easily
distinguish from caring ones -- or they expend a great deal more
resources to send and care for each outgoing piece of their Bulk
email, while taking the very significant risk that Bulk email sent to
spamtraps and/or sites that otherwise autodetect Bulk sending will
result in such care accomplishing literally *nothing* in terms of
getting a message into a given end user's eyeballs.

If that isn't enough to stem the tide of UBE, I sincerely doubt IM2000
can do any better, because it can only promise increased costs for
sending UBE at the expense of increased costs to recipients to receive
*all* email (because of the pull model, at the very least).

>> Or, the mess of SMTP bodges might justify designing and rolling out a
>> system that is completely new, like IM2000, but is push-based, like
>> SMTP, yet bounce-free, like IM2000, because it employs tracking, like
>> FedEx, yet leaves responsibility delivery with the sender a la the
>> end-to-end principle, like TCP.
>
>The familiar push model has advantages too. It might be worth trying to
>design a new protocol under this model, and see if you can end up with the
>same advantages as IM2000 (or other ones).

Actually, I think that's what I've been doing, on this list mainly.
Just a question of whether and when I (or someone else) actually go
forward with it.

-- 
James Craig Burley
Software Craftsperson
<http://www.jcb-sc.com>