Re: Comments on IM2000

On Wed, Apr 20, 2005 at 06:01:19PM -0000, James Craig Burley wrote:
> I think this presents a *huge* problem for blacklists, because
> spammers can easily dDOS them by sending out bazillions of
> notifications containing arbitrary sender-account IDs "tupled" with
> otherwise-legit mail stores.

Sure, any blacklist is subject to that kind of attack. Blacklists which run
on top of the DNS benefit from distributed caching.

Dealing with junk notifications is one of the problems I have with the
IM2000 approach. Even without blacklists, you will need to validate them
somehow (e.g. by doing a callback to the originating message store, or by
some cryptographic method). You don't want a blast of fake notifications to
drown out legitimate ones.

With SMTP you can of course open large volumes of SMTP connections to hosts.
This may trigger some sort of validation process (RBL lookup, sender
verification callback etc). I think the trick is to keep to a minimum the
amount of work which a random attacker can cause the system he/she is
talking to (and any referenced third party) to perform.

> My SMTP server does nothing but log those, and does only *internal*
> lookups on domain names in envelope senders.  (I use no RBLs,
> amazingly enough, out of a sorta-principled desire to experiment with
> anti-UBE measures that don't let untrusted SMTP clients use my server
> as a proxy to beat up external shared DBs.  Pretty much the only
> useful "key" that a spammer can't forge is the incoming IP address of
> the SMTP client -- but I don't do any external lookups on that either,
> at the moment.)

That's a shame. You ought to try RBLs, because they are surprisingly
effective even in the current SMTP world, and to evaluate how blacklisting
through DNS works in practice.

I suggest you set up lookups against three RBLs:

(1) Known spammers and 0wned machines (e.g. sbl-xbl.spamhaus.org)
(2) Open relays (e.g. relays.ordb.org)
(3) Dynamic spam sources (e.g. bl.spamcop.net)

You can of course just configure your MTA to perform the lookups and add a
warning header, rather than rejecting the mail. You can then examine your
received spams and measure how effective they might have been.

> As they succeed in gumming up the works -- by discombobulating shared
> (community) DBs that blacklist all the arbitrary [mailstore, senderID]
> tuples spammers can come up with -- those "works" will either be
> disabled so people can receive email (opening the floodgates to UBE)
> or will keep mail from being successfully and quickly delivered.

I think notifications need to be validated, for example by
- a callback to the proported originating host; or
- cryptographic check using a key in the DNS

One notification generates one callback or lookup, so it's not a DoS
amplification service. The spammer won't actually achieve anything, since
the notification will be discarded having failed validation, so there isn't
anything to be gained.

Sure, they will have wasted some resources on some random machines on the
Internet - but there are plenty of ways to do that at the moment anyway.

> It's also not clear exactly how IM2000 will prevent spamming by
> sending notifications such as:
> 
>   [im2000.comcast.net, [email protected]]
> 
> Seems to me such spam might be fairly successful in an IM2000 world,
> unless sender addresses are normally hidden from users, which is not
> what you appear to be advocating.

I did suggest that (sender fingerprints), but I think it's better that the
sender address *is* a genuine E-mail address, and that the notification is
discarded unless it is proved that this notification did originate from that
E-mail address. That gives phishing protection too.

> With SMTP, it's easy for a spammer to have his SMTP client do:
> 
>   MAIL FROM:<[email protected]>
>   RCPT TO:<[email protected]>
>   DATA
>   [...]
>   .

Errm, but the recipient address won't exist, so there will be a 550 error
after the RCPT command, and the DATA command will be rejected.

> So, say spammers learn that my (legit) mailstore tuple is
> [im2000.comcast.net, [email protected]].
> 
> Now, they can't *send* IM2000 spam via that tuple, because they
> (presumably) can't convince the im2000.comcast.net mailstore to serve
> their spam under my senderID.
> 
> But that doesn't prevent them from sending out a bazillion spurious
> notifications to everyone, claiming that a bunch of emails are at that
> tuple, thus wasting everyone's time as they try to read those emails.
> 
> (That'll swamp im2000.comcast.net, but only with requests coming from
> *real* people -- not necessarily as a result of notifications being
> sent to nonexistent addresses, spamtraps, etc.)

Indeed.

> Some people will respond by "blacklisting" my tuple in their personal
> lists.

No, clearly you cannot blacklist a notification based only on the
(potentially forged) information it contains.

You could however blacklist notification sources based on their IP address.
There may be people sending notifications from dynamic IP addresses, but
that's unlikely; legitimate IM2000 message stores will be on fixed IPs, and
the notifications will *normally* come from the message store itself.
However, the IM2000 protocol explicitly allows notifications to be
forwarded, which is a useful capability.

That's why I think it's probably better to validate notifications to prove
they came from the claimed message store (and discard them if not), rather
than maintain a separate IP-based blacklist of spurious notification
senders.

Spurious notifications are unlikely to be sent often, since they don't
achieve anything apart from a waste of resources, so I think it's unwise to
build a separate infrastructure to filter them out, one which won't be
tested very often.

> In any case, that'll swamp im2000.comcast.net such that it can't serve
> *any* email, and there'll be nothing Comcast, nor recipients, can do
> about it.

For each one notification message sent (TCP connection), the recipient will
open one TCP connection to im2000.comcast.net

The attacker could just open those connections herself, and apply a similar
kind of DoS anyway. However, this approach *does* hide the source of the
attacker.

Perhaps, when contacting im2000.comcast.net, the validation request should
include the purported source IP address of the notification? Clearly this
could be forged too, but if you get a lot of notifications from people all
over the Internet and they all claim to have received notifications from the
same IP, you have good evidence of the source of the attack.

As I say, this attack won't achieve anything (such as the delivery of a
spam), only wasted resources - so eventually the attacker is likely to get
tired. But I do agree with you that this needs to be considered carefully,
and the best solution arrived at such that this approach is at least no more
efficient than DoS'ing the message store directly.

> So, the "win" for IM2000 here is that spam can't be forged as coming
> from someone else and still be delivered as spam.
> 
> The "loss" is that IM2000 pretty much requires the same sort of
> "doublecheck" that people used to do with SMTP, when a server would
> try to contact the envelope sender for untrusted email and see if
> *his* SMTP server would VRFY that the address existed or accept a
> bounce sent to that address (going as far as MAIL FROM:<> and RCPT
> TO:<address> and checking the response, before discontinuing the
> conversation).

Except that at the moment, an SMTP callback only verifies that the address
exists, not that the mail you're trying to receive was sent by that person.

Putting signed cookies in the envelope sender, such as SES/SRS/BATV are
proposing, does actually make that connection - although somewhat weakly,
because if a spammer captures one of your cookies, they could re-use it with
a different mail.

> Though IM2000 gives us the opportunity to improve the performance of
> that sort of doublecheck, it does so only on a scalar basis (let's say
> it makes it 10x more efficient), whereas spammers will have plenty of
> incentive to ramp up attacks on certain popular, trusted mailstores at
> a much higher rate than the performance is improved by IM2000.

Maybe. People do DDoS attacks on all kind of Internet infrastructure, for
all kinds of reasons. It's a pain, but you're not going to fix that without
fixing the Internet itself.

> Amusingly, I've recently started receiving spam that advertises its
> products via ASCII art, including the phrase "BEST PRICES"!  ;-)

Cool. You can't accuse them of lack of imagination. It's another reason why
(IMHO) content analysis will always be doomed, until we have genuinely
intelligent robots. And if they are that intelligent, they will wonder why
they are doing this job in the first place. "Filter my spam, Marvin" -
"Brain the size of a planet and they ask me to filter out spam..."

> I tend to think the biggest problem with SMTP is really the fact that
> it (normally) prevents a server from accepting a message without also
> accepting *responsibility* for that message -- which translates into
> the need for generating a bounce.

That's pretty much essential in a store-and-forward model though. In the
current world, all MUAs are set up to send outbound mail via a
store-and-forward server (and indeed, blacklists are set up to *refuse* mail
sent directly from end-user IP addresses, so replacing MUAs with something
cleverer which followed MX records in the DNS would not work)

Once you've handed the message over to the S&F server for delivery, if it
fails, it needs some way to tell you so. That need not be in another E-mail
message, but if not then there's an out-of-band notification required; and
to prevent this channel from being used for spamming, it needs to be
unforgeable.

> >But if I can only reliably blacklist on IP address, I am limited when it
> >comes to shared mail relays at ISPs. If 1 million users are all relaying
> >through one ISP's mail relay (a common case), plus 10 spammers, I either
> >have to blacklist the entire mail relay and suffer collateral damage, or I
> >have to accept the spam.
> 
> Right, I understand all that.  But that is simply a Very Hard Problem
> anyway; as I pointed out earlier, even the "hotpop"-like ads in the
> headers of otherwise-legit email could be defined as "spam".

Ah well, then you have a single E-mail with a split personality: part
legitimate, part spam. If you dislike it sufficiently, then you can refuse
to communicate with that person, or ask them to use a less obnoxious E-mail
service.

> And, as I point out above, if you resort to blacklisting [mailstore,
> senderID] tuples of "known spammers" at otherwise-legit mailstores,
> you either:
> 
>   - Overload the blacklist(s) you're using with all the spurious
>     senderID's that spammers can make up, or
> 
>   - Overload the mailstore(s) with requests to validate whether any
>     given senderID a spammer might invent actually has a legit account
>     there (equivalent to SMTP VRFY or RCPT TO)

I don't see that *necessarily* follows, nor that the overloading you're
proposing is any worse than a direct DoS attack from the spammer to the
message store or to the blacklist.

> This problem could be mitigated by insisting that outgoing message
> notifications go *through* (and thus be vetted by) mail stores, so
> only "legit" email addresses are exported.
> 
> But that can be done for SMTP anyway, and in fact is done, sorta
> kinda, in various ways.  E.g. some ISPs deny ordinary users outgoing
> TCP connections over port 25; filter incoming connections to their
> internal SMTP servers to ensure MAIL FROM:<whatever> identifies an
> addy the customer (identified by incoming internal IP address)
> actually owns; provide AUTH access for external use by legit
> customers; etc.

It can be done, but in general it isn't. Outgoing relays would have to
become more complex to perform AUTH database lookups, and huge customer
bases would have to be migrated over to using SMTP AUTH. Only when the last
customer has switched to using SMTP AUTH, and relaying by source IP address
only is disabled, can the source be considered 'trustworthy'.

And then there's the problem of which E-mail addresses a particular user is
allowed to use; either many local databases will have to be built, or one
distributed one.

As an example, I login to smtp.example.net as user "foo123", and then I want
to send outgoing mail with MAIL FROM:<[email protected]>.

Either:
(1) The ISP maintains a local list which ties foo123 <=> [email protected]
Every time a new user comes along, they will have to state which E-mail
addresses their account is allowed to use, *and* prove to the ISP's
satisfaction that they do in fact own those accounts.

(2) There is a global database. As owner of [email protected], I ask the
pobox.com domain owner to publish information in the DNS which ties
  [foo123, smtp.example.net] to my address, SPF-style.
There are problems with this if the ISP decides to change their
infrastructure, such that outgoing mail starts appearing from a different
machine or IP address. This would invalidate the policy.

(3) The ISP rejects all mail from me unless it has
MAIL FROM:<[email protected]>. This is unacceptable for lots of reasons:
- This may be a temporary account, e.g. at a cybercafe; I will need to
  receive bounces after I have left that location.
- I may be sending a bounce (MAIL FROM:<>)
- There are many legitimate reasons for using variable envelope senders;
  e.g. I may be sending using VERP, SES, SRS or BATV envelopes.

Anyone who did this would break all their users, and yet there would be
little benefit unless the *whole* Internet did it (which will never happen
even if this were decided to be 'best practice')

Seriously, changing to something different (maybe IM2000, maybe not) would
be easier. You can have a phased migration. Over time, you would give more
credence to your new mail and less to SMTP mail.

> E.g. I don't really care whether you actually are Brian Candler,
> because I'm not investing anything in the proposition that you are who
> you say you are.  So resources I might otherwise expend to validate
> your identity are spent on things I think are more worthwhile, such as
> considering your viewpoints, responding to them, etc.  You could be a
> very intelligent German Shepherd, for all I know (but, on the
> Internet, nobody knows etc. ;-).

Woof :-)

> >IM2000 also wins because it's easy to build message stores which rate-limit
> >the number of messages sent per day from a particular account; it could be
> >enforced as good practice (i.e. if you offer free signups for new accounts
> >over the web, then you get your message store blacklisted if you don't
> >follow this principle)
> 
> I'm not sure why that's hard for SMTP -- aren't some ISPs doing that
> already?

Because
- mail relays generally don't have an unambiguous indication of *who* the
  person is sending through them (without SMTP AUTH, or some sort of
  callback into a RADIUS accounting system)
- mail relays are typically built in clusters, so in order to count the
  number of messages sent by a particular user in a particular period of
  time, you would need to use a central database

Certainly you can rate-limit the number of messages sent within a single
SMTP session, but that's not very useful, because
- the sender can open multiple SMTP sessions
- a typical usage pattern for legitimate users is to compose a number of
  mails locally on their machine, dial up, and then send them out
  (which means the thresholds have to be set very high)

...[forcing the world to adopt SMTP AUTH]...
> >However, this ain't going to happen. The pain is huge, and the short-term
> >gain is negligible.
> 
> How will the equivalent *not* have to happen with IM2000?

You can start setting people up on IM2000, and include gateways to SMTP
(inbound and outbound).

People can immediately see which mail has come in through IM2000, and which
through SMTP. They can use this to make value judgements on their mail. It
would particularly benefit closed user groups (e.g. companies, groups of
friends) where they know the other party is on IM2000, as it means these
identities would be forged if they come in via SMTP.

You can apply more stringent filters on SMTP mail. As more and more of your
correspondents are on IM2000, the less worried you are about false positives
on SMTP. For granny who only wants to communicate with grandchildren, SMTP
could be disabled entirely for that account.

Over time, though, IM2000 becomes less and less of a "closed user group" of
course. Actually, I think a mail replacement architecture should explicitly
support the idea of closed user groups. And possibly instant messaging too.

> Okay, then this is not a case of *technical* advocacy, more a case of
> saying "it's a whole new system, we think it'll be fairly spam-free
> compared to SMTP" -- and in fact it probably will for some time -- and
> hoping to convince enough early adopters to jump on board, quickly
> enough, that everybody else, except spammers, switches soon after.

No, if I believed that then I wouldn't suggest it.

Any new solution has to be a *permanently* harder environment for spammers
and fraudsters to work in. I think there are some strong arguments why
IM2000 could have an anti-spam infrastructure which works much better and at
much lower cost (of complexity and false positives) than SMTP, which I
documented in my comments on the web.

> In any case, IM2000 does not, in any way, shape, or form I can see,
> dispense with the need for content analysis.  As an "illegit"
> mailstore first comes online, or a [legit-mailstore,
> evil-spammer-using-it] tuple springs into existence, *somebody* will
> have to take the first steps towards discovering that email springing
> from that source should have the Evil bit set.

Certainly, and that's why we need the third parties (blacklists) to help us
with that.

But IM2000 allows alarm bells to trip quickly based on message sources,
potentially to perform some first-line content filtering, then to highlight
the source to a human, who makes the final decision as to which button to
press (e.g. blacklist sender account, blacklist mailstore). Borderline
sources can be greylisted temporarily until this decision has been made.
Once the source has been blacklisted, anyone who logs in later will not even
see the notification.

If a spam broadside only gets in front of a few dozen eyeballs then the
spamming attempt won't have been worthwhile, and they will give up.
Furthermore, if there is better evidence of where the spam came from, then
anti-spamming laws might be more effective.

> IM2000 ultimately puts us in the position of hoping not just that this
> will be required *less* often, as a % of overall email sent, but that
> the economics will ultimate in spammers giving up and going away.
> 
> I'm not seeing a clear case being made for that scenario coming to
> pass, certainly not clearly enough such that it justifies the
> deployment of IM2000 as it now stands.

Well, maybe we have to build it and see. As well as writing IM2000, we write
an IM2000 spam-sending toolset and publish it too, to check that things work
as we expect.

> >> >They *will* happen, and soon (i.e. within months at most), as surely as the
> >> >widespread implementation of MAIL FROM domain checks made spammers change to
> >> >sending out mail with real (but forged) E-mail addresses as the envelope
> >> >sender. The cost is minimal.
> >> 
> >> Right, it's an arms race, we all know that already.  None of what you
> >> have said on that topic is news to me; I doubt it's news to anyone
> >> else here.
> >
> >All I'm saying is, multiplied across the Internet, it's (a) a huge waste of
> >time and effort, and (b) is contributing to the decreasing reliability of
> >E-mail, which I consider to be a very serious problem.
> >
> >People think they are doing good, when they are doing harm.
> 
> But IM2000 *requires* it, in essence: envelope senders specify domain
> names that *must* be looked up, and thus must have obtainable,
> verifiable DNS information, must not be in various blacklists, etc.
> 
> So IM2000 gives spammers a *guaranteed* weapon to use against DNS and
> blacklists that SMTP doesn't assure them

It's the same point - DoS is DoS. It doesn't really gain them anything more
than a direct DoS assault on whatever it is they're trying to attack.

> >But it's still not going to be as strong. For example, counting the number
> >of mails sent by a particular user is still quite hard. Consider an SMTP
> >relay cluster of 10 machines; they would need to communicate with a central
> >database to count the number of mails sent by one user over 24 hours.
> 
> How is that different from an IM2000 mailstore cluster of 10 machines?

Most likely each user would have an account on a single machine in the
cluster, and you'd spread your users across them. Then it's easy.

But even if you decided to have a cluster with a shared NFS backend, and
users could connect to any of those 10 machines to submit a new mail: all
you need to do is maintain a state file within the users' mailstore area
giving a history of sent mail. Each new submission adds to that history, and
may cause a threshold to be reached.

This is unlike SMTP, where there is no permanent record of mails passing
through the system, and all outbound mails are lumped into a single queue,
not a separate queue for each user.

> >And you'll still need a new blacklist infrastructure for looking up
> >[IP,auth-sender] instead of just IP address.
> 
> Sounds like item 2a on your web page!  Again, how is it different?

That's the point. Moving to a parallel SMTP world would be pretty much as
difficult as switching to IM2000.

> What it suggests is that SMTP could evolve to incorporate IM2000
> concepts in ways that *allow*, but don't *require*, clients and
> servers to cooperatively use them.

I'm happy to see specific proposals here, and as you're no doubt aware, a
lot of work is going on already.

The trouble with greylisting by giving a 4xx response after the DATA phase
(or unceremoniously dropping the connection) is that there are plenty of
broken MTAs out there which are likely to treat this is a permanent
rejection.

Like SPF, you may end up breaking more than you fix.

> But I prefer to *conceptualize* in terms of a brand-new, clean system,
> and then *actualize* in terms of incrementally improving SMTP in that
> direction, in order to speed uptake of whatever I might think is
> useful.

Well, that's reasonable enough. But the extended SMTP may look not very much
like SMTP by the time the job's done :-)

One example you didn't mention was the problem of making a canonical hash of
a message (for detecting tampering, for applying signatures and so on). Too
many SMTP agents mangle messages as they go through, and indeed the SMTP
specification *requires* them to do so: to add Received: headers, to add
other missing headers, to correct linebreaks into the canonical \r\n form,
to break lines if they are longer than a certain limit, to reformat 8-BIT
encodings as 7-BIT if the receiving SMTP server doesn't implement 8BITMIME,
and so on.

If we switched everyone to using SMTP CHUNKING (BDATA), that would be a
start. But we would then need an extension not to interfere with headers,
and perhaps a way to keep things in the envelope such as Received:
timestamps and signatures.

Ultimately, I believe a new messages store with an SMTP gateway would
probably end up being easier to build *and* easier to interwork with the
rest of the Internet... and it could deliver tangible benefits immediately.

Regards,

Brian.