Re: Comments on IM2000

>On Mon, May 09, 2005 at 08:56:04AM -0000, James Craig Burley wrote:
>> >There's no point having another button which says "send but don't
>> >try very hard". Why would I ever push that button when I could push "send"
>> >to have a better chance of my message being delivered?
>> 
>> If the default is like SMTP, the default "Send" button would probably
>> do little more than transmit the message and then send one tracking
>> request about a minute later, another maybe 10 minutes later, a third
>> maybe 24 hours later, and a fourth about a week or two later, as
>> necessary.
>
>And if, in practice, that is not good enough to ensure successful delivery
>in the vast majority of cases, then senders will choose more aggressive
>parameters.

Yes, assuming their messages are sufficiently important for them.  The
more outgoing messages they have, the more expensive those aggressive
parameters will be for them.

>> Spammers will naturally keep trying to gin up new identities, but the
>> new system is more easily able to "punish" that, without necessarily
>> triggering a lot of false positives, by expecting any *legitimate*
>> messages that come from unknown sources to be reasonably and
>> frequently tracked by their sender.
>
>And assuming that they do, since they desire to get their spam through, what
>have we gained? You assert that this tracking is likely to be too expensive
>for them. I assert that it is trivial, and demonstrated how to write a
>program to do it for many millions of recipients.

Why write a program, when simply using whatever vanilla software
implements the new protocol will work just fine?

Again, if you think that sending an average four tracking requests per
outgoing message for, say, over one million messages sent by a single
0wned box in a given day, and doing so over a period of as much as one
week, is something spammers can afford to do, great, that's your
opinion.

But that's *more* than they have to do *now* to send SMTP email (all
else being equal) and probably more than than they'd have to do under
IM2000 (as generally described), since the vast majority of their
message notifications would be ignored by recipients anyway.

Under my proposal, there's no question the "average" user would have
little trouble sending hundreds, even thousands, of pieces of outgoing
UBE without running into resource limits.  But spammers won't find
email to be such a worthwhile protocol to advertise their goods at
those rates of usage, especially given an increasingly sophisticated
end-user audience (and more-intelligent MUAs).

But as the number goes up, they run into all sorts of trouble with
their outgoing bandwidth, potential upstream ISP limits, downstream
(recipient) limits, and so on.

Since any of those limits can result in dropped email, the best system
is one that *gracefully* allows email to be dropped.  My proposal does
that, as does IM2000, without reducing the reliability of ordinary
exchange of email, which is not true for IM2000.

>> I get the impression you don't yet have a *coherent* view of what an
>> email *system* should do.
>
>My view has always been:
>- spam is anything which originates from spammers

And who are spammers?  People who send spam?  That ends up being a
recursive, therefore useless, definition.  It leads to problems
defining spam -- who is responsible for doing that, who has the right
to claim another person is a spammer, etc.  (These are *real* problems
we have right *now*.)

>- if we can quickly and unambiguously recognise new spam sources, before much
>  of it has been delivered, then we can block it before it causes much of a
>  nuisance, and also make it so ineffective as to be not worth sending in
>  the first place

If true, and if sufficient, that puts SMTP, IM2000, and my protocol on
an equal playing field, except that SMTP puts too much pressure on
relays and recipients to take responsibility for incoming email
*before* the source is recognized as sending too much spam -- though
ecrulisting addresses that (probably somewhat clumsily).

>- if we can identify the sender accurately and quickly, this would help
>  enforce anti-spamming laws

We can do that *now*: it's the "owner" of the immediately upstream IP
address or, if that's an innocent relay that is reasonably trusted to
insert proper "Received:" header, recurse to the next upstream IP
address.

If that final owner says "well, I can't be sure which of my users or
customers actually sent that email", that's *her* problem, for which
*she* should be held responsible, since *she* is responsible for
filtering, authenticating, and/or reliably logging all outgoing
communications through her IP address for just this case.

>What should the E-mail system do? It should have an *unforgeable* identity
>in each incoming message; and we should be able to tell where that identity
>originated from, so that a place where large numbers of identities are
>created for spammers can be treated as a single identity.

Again, identity is orthagonal to email exchange.  How do I know this?
Because sending anonymous emails makes perfect sense in all sorts of
situations, and because identity is important to *other* forms of
exchange.

If you don't want to accept anonymous email, your recipient MUA can
simply reject such email.  Ditto for email with an identity it is
unable to confirm.

The email *system* is *not* the right (and obviously not the only)
place to "enforce identity".  We've gone over this ground many ways
already.

>I'm afraid I don't see many other approaches. Trying to identify patterns of
>behaviour which characterise spammers from non-spammers is doomed, because
>spammers can *easily* mimic the behaviour of non-spammers. This is what they
>did when people started filtering on non-existent MAIL FROM:<...> domains,
>for example.

Trying to identify spammers by "unforgeable identities" is doomed as
well, becaues there will always be new/changed/reformed people coming
online with no absolute guarantee that they'll be linkable to any
former online identities.

>The one thing which spammers can't mimic is the fact that they send spam -
>vast quantities of untargetted mail. If you can detect this reliably,
>without harming those people who have legitimate reasons for sending large
>quantities of mail, then you have a good solution against spam. Sooner or
>later a human element and a trusted third party need to be involved, simply
>to share the work of classifying new sources. (You see such TTPs as
>potential points of attack - as indeed they are - but I believe such systems
>are capable of being built in such a way as to defend themselves).

They *are* built that way *now* -- they're the "smarthosts" that more
and more sites require incoming email (from some portions of the
Internet) to arrive through.

If that's enough, then we have no need for any *new* email technology,
since we already have SMTP AUTH and other means for those TTPs to
ensure that they remain Trusted even as they cater to more and more
users.

>Current solutions such as RBLs and DCC attempt to implement exactly this,
>with some degree of success, but with an unfortunate amount of collateral
>damage simply because SMTP doesn't give them a sufficiently fine-grained
>view of the sender identity, particularly when mail goes via shared relays.
>And because they are not sufficiently good at their job, people implement
>other types of filtering too, which often block legitimate mail.

Yup.

>Now, that's assuming you want to be able to receive mails from people you
>don't know. The only other solution I can see is to take the view that
>everyone you don't know is suspect.

Yes -- that's a solution that pretty much anyone can implement now,
via SMTP.

>Anyone who is not
>known, needs to be given a sufficiently difficult thing to do to get through
>to you - whether that be pay a 5c e-stamp, or spend 5 x 1GHz CPU-seconds
>calculating a cryptographic problem. This could be anything which a single
>person sending you mail could do at low expenses, but for which sending a
>million mails would become prohibitively expensive. (It may not be
>prohibitively expensive if spammers *targetted* their lists, but then it
>wouldn't be spam any more)

Right.  If I designed a new protocol along the lines I'm thinking, I'd
try to accommodate this sort of response by a recipient.  Given the
protocol I have in mind, it doesn't *seem* hard, but I don't know
enough about how hashcash or epostage actually would work to be sure.

I assume IM2000 would be extended, in the wild if not in its deployed
form, to support such options.

>But the extra work involved by the spammer has to be quantifiable, and
>controllable (so that over time, the "cost" can be increased proportionately
>as required by improvements in CPU speed, devaluation of the cent, or
>whatever). An arbitary constant constraint like "must poll for message a few
>times in a certain period" doesn't really cut it.

Yup.  The system -- whatever it is -- *has* to be flexible, adaptable,
and yet at least as reliable as we have today.

The problem is, the reliability of SMTP email appears to be rapidly
dropping, and I believe I've addressed several key reasons why
already.

-- 
James Craig Burley
Software Craftsperson
<http://www.jcb-sc.com>