Re: State of art

On Thu, Mar 24, 2005 at 04:36:30PM -0500, Erich Rickheit KSC wrote:
> > - you must get an account (username/password) on the mailing list message
> >   store before you're allowed to post, which forces you to go through some
> >   sort of registration procedure, and store the username/password on your
> >   client. That just makes life harder for people who want to join lists,
> >   although an easy hurdle for mailing list spammers to overcome.
> 
> That's called subscription; you have to do that for mailing lists
> now. And I don't see why telling your mail client
> 'username:password:[email protected]' is harder than telling it '[email protected]'

What I meant was, presumably you have to go to some sort of registration
page where you choose a username and password for the list (or are assigned
one), and then paste this into your MUA.

It's certainly not hard; it could even be automated by going to a webpage
which sends you something back that configures your MUA automatically - the
Windows world has this in the form of .INS files.

But the easier you make it, the easier it is for spammers to join these
groups, and therefore the list may as well have been publicly open in the
first place.

You're right that there's no official central directory of mailing lists as
there is for USENET; but that doesn't stop spammers collating their own
lists of them, as they do with E-mail addresses.

> > So: how, exactly, is life made harder for spammers, and easier for
> > legitimate users?
> 
> Essentially, in that a spammer has to have a message store and bandwidth.
> 
>    If he keeps it himself, he will quickly find it balcklisted
>    If he gets it through a responsible ISP, they will can his account when 
>      they discover him to be a spammer
>    If he gets it through an irresponsible ISP, the ISP will find itself
>      blacklisted, and feel pressure to can his account.

But in today's world it's the same:
- If a spammer sends using SMTP from his own box, he will quickly find it
  blacklisted
- If a spammer sends through a responsible ISP's smarthost, they will
  process the abuse complaints, tie his IP address to a sending account
  (maybe via RADIUS logs), and can his account
- If a spammer sends through an irresponsible ISP, the ISP will find itself
  blacklisted, and feel pressure to can his account

And you forgot to add (in both cases):
- the spammer will then sign up for a new account, unless the ISP prevents
  this somehow

However, there is one improvement in the IM2000 world: the
<ISP message store, account ID> tuple can be blacklisted. That reduces the
need for the ISP to behave responsibly or promptly to abuse complaints; the
blacklist manager does that for him. And that's a good thing: with
blacklists you have a choice of which blacklist to subscribe to. You can't
change the ISP that the spammer sent through, unfortunately.

(Also, ISPs are not inclined to cancel accounts, as it reduces revenue. They
are more likely to give a rap on the knuckles and warn them not to do it
again)

> Is this a hardship? It's based on the assumption that the cost of
> people retrieving spam is significant. If I send out ten million
> notifications, do I get ten million hits? That would make spam less
> cost-efficient. Will I get a million hits? A thousand? Ten? This
> statistic is what would make the difference.

I'm not sure I see the bandwidth difference compared to sending out ten
million spams via SMTP.

Remember, the spammer can write their own IM2000-compliant software. If it
were me, I would write software which keeps one copy of the spam, but sends
out notifications with ten million different message store account IDs (so
it looks like they're from different people). Only those recipients which
were active would come and download the mail from my server - saving me
potentially tons of bandwidth compared with sending out ten million copies
via SMTP which may fail to be ultimately delivered. I could install the same
software on 0wned machines, and use less bandwidth out of those too (not
that I care).

There are real advantages I can see though. Firstly, those "pull message"
hits are spread over time, as people check their mailboxes periodically.
That gives time for the message store holding the spam to be either
blacklisted or cleaned.

People who are on-line 24 hours per day may suffer the same with receipt
notifications appearing in their inbox, but those who connect only
intermittently will benefit.

DCC-type services, or automated spam detectors using spamtrap mailboxes,
could work more effectively.

There is the opportunity for "graylisting" to work more effectively too.
That is, if I get a receipt notification from a <message store, userID> that
I've never seen before, I hide it from the MUA for a few hours. In that
time, hopefully, if it's spam the server or account will have been
blacklisted. And unlike MAIL FROM, the <message store, userID> combination
cannot usefully be forged.

More interestingly, there is the opportunity for blacklists to get *real
proof* of spamming, directly from the horse's mouth so to speak. As long as
I've received what appears to be a spam, but have not yet downloaded the
body, I can pass on the notification message to a blacklist agency. They can
pull down the message body themselves; it can be examined for spamminess (by
a human being probably); and a decision made to blacklist either the whole
mail store or just the account. They can store the whole session as direct
proof of where the spam originated, not something which has been via a third
party and therefore could be forged.

However, note that you *cannot* download the body first before reporting it
as spam. If I were a spammer, then I would write my sending software so that
if the body were downloaded, the message would vanish immediately from my
message store (*without* an un-pin request being received first). Then the
spam checker would go to the message store but the evidence would be gone.

It does mean you are only guaranteed to be able to submit your spam proof to
one blacklist, but that's probably sufficient; the whole point of spam is
that there's lots of it to go round :-)

But it could work the other way round. That is, for *every* mail I receive,
I can pass on the notification to a blacklist agency (that I trust!) which
allows them to download a copy, *without* unpinning it. An automated system
can then decide if it may be spam; a human can check; and if it is, they
perform the blacklisting. Non-spam remains pinned so I can read it.

It only needs a small percentage of netizens to cooperate in this way -
perhaps only has to be the people who work in the blacklist organsations.
The benefit to me as an individual is I get my mailbox swept, but the
benefit to the net as a whole is that the blacklists get updated much more
quickly.

The advantage here over things like SpamAssassin is that each blacklist
agency can make their own rulesets for detecting spam, *and keep them
private* (whereas spammers can look at the source code for SpamAssassin and
work out how to tailor their spams to pass all the checks).

> OK, you can send notifications from your zombies to your heart's
> content. But your message only gets to people if that particular
> zombie is up and able to serve it if, as, and when they request it.
> (again, I need some statistics here)

Yep, so I just came to the same conclusion as you. The zombies may remain
on-line for a long time, but they might get blacklisted quickly.

> The real mechanism here is that notifications distinguish between
> mail stores identified with IP addresses, and stores distinguished
> with domain names. My software has an easy policy:

You have some IM2000 software? Where can I find it to play with? :-)

>   whitelist for IP addresses. Only go to mail stores identified by
>   IP addresses if I have a previous arrangement with them (id est,
>   they're a mailing list or something I care about)
> 
>   blacklist for domain names. Only someone who can publish a SRV
>   record for a domain can set up a message store for that domain;
>   now I can distinguish good domain from bad domains, as we've
>   discussed.

...although spammers could still get throwaway domains.

> The spammer bears more of the cost of sending mail. An ISP cna
> charge a spammer for bandwidth to their message store.

Can't an ISP charge for IP bandwidth used now, and bandwidth through their
SMTP server now? (If you were billing by SMTP usage then you would need to
tie the sender identity to a billing entity and sending time; probably SMTP
AUTH to be most reliable). But ISPs dislike charge-by-usage in general, (a)
because it's hard to bill reliably, and (b) because customers don't like it
and will go elsewhere.

> This, it is
> hoped moves us towards a postal-mail model, where senders of junk
> mail actually pay their costs and help support the email systems,
> rather than just leeching off it.
> 
> All this is the theory; I haven't yet decided whether I believe in it.

:-)

As I say, I can see how the new model would *reduce* the bandwidth that
spammers use (and pay for, if they're not stealing machines); I don't see
how it increases it. What I do see is ways for controlling spam that have
already been invented, to work *much* more effectively.

The main costs incurred in handling the spam problem actually end up in the
hands of the blacklists, and how they recover their costs (and deal with the
bullying and thuggery of spammers) is perhaps the biggest non-technical
problem.

Cheers,

Brian.