Re: Strawman abuse report format proposal

This seems like a great first cut to me.

Quick summary:

1a) Need to specify subject line standards including important keywords
1b) Should have multiple messages, one per abused resource
2) Should suggest who receives the report, paying particular attention
to IP address of the sender.
3) Should consider not reporting abused resources other than the sender,
or limiting such reports to cases when we can verify that the resource
was actually abused

Details

1a) I don't see anything in here about the subject line.  There should
be some specifications for subject lines in particular, both because
humans can read them more easily and there are many rules systems and
scripts that can react in a particular way to subject lines.  I'd
suggest something like

SUBJECT: Abuse Report Standard ????? <KEYWORDS>

With the list of KEYWORDS to be decided.  There are some cases in which
we can be sure we are sending to the correct recipient of a complaint,
and others in which we may be sending to an unintended victim of a
spammer.  For example, if the From: line is [email protected] we may
send a complaint to spamhaus.com even if we suspect the mail is not
really from them.  Spamhaus might be a frequent victim of such attacks,
and may want a rule that lets them bitbucket all unverified reports of
spam.  On the other hand, if the spam comes from 1.2.3.4 and the HELO is
evilspammer.com and a forward IP lookup verifies that 1.2.3.4 is
evilspammer.com, we can be sure that [email protected] really is a
good place to send the report.  Thus, I suggest in particular keywords
VERIFIED and ALLEGED.  I'm happy to see other keywords as needed.

1b) If we decide to go with the multiple resource approach, I'd suggest
multiple messages per abuse report, one for each abused resource.  The
subject line should then specify the abused resource.  People with open
redirectors are mostly well aware of the problem and mostly have plans
to fix it, and would probably like to be able to easily toss any repeat
messages notifying them.  So, the keywords should include the resource
type, and then I can create a rule to trash any messages about WEB.
Also, many spammers include links to legitimate sites, e.g. msn.com
along with links to their spam, and the fact that a link to my site is
in the message is useless to me.  So, I'd like to be able to write a
rule to immediately trash any such abuse reports.  Alternately, maybe we
should dispense with abuse reporting standards for anything except
sending mail, since all other resources can only be verified by hand
inspection (typically.)

The abuse-cookie report idea, which is a good one, also requires sending
multiple messages for multiple resources, since the sender of the mail
could include an abuse-cookie header, but the other resource, e.g. web
page hosting, may want the full message.

2) There is not much discussion of who should in general receive
reports.  The standard need not require any process here, but at least
should have some best practice suggestions.  Here are some notes:

2a) If we can positively identify the IP address of the sender, we
should send the report there, using either a reverse-IP lookup followed
by forward IP lookup for verification (unforgeable) or a forward lookup
of the HELO matched to the IP address if the first method fails.  This
is the single most important place to send complaints.  This case
deserves special keywords.

2aI) Note that email clients like Pine and Outlook cannot reliably
determine the IP address of the sender in general.  This is highly
unfortunate.  Email servers typically can.

3) I'm nervous about reporting abuse for any resource other than sending
spam.  A great deal of spam includes, e.g., extraneous links to victim
sites like Yahoo.com and MSN.com, in addition to their own sites.  I'd
like to see lots of reporting, but primarily when we can be certain who
the sender is.  I'm happy to see reports of other abused resources, but
these should not be automated in the same way.  In particular, we should
only make a "This is junk button" or a trap-account reporting system
report other resources when the other resource is in a domain that
verifiably matches the sender's IP address.

P.S.

For those who don't know me, I'm a Researcher at Microsoft Research.
I'm not part of MSN or Exchange or Outlook or the spam team, and speak
only for myself.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Steve Atkins
Sent: Monday, September 27, 2004 8:18 PM
To: [email protected]
Subject: Strawman abuse report format proposal

This is a strawman proposal for an abuse reporting format. Comments and
discussion actively encouraged:

Context
~~~~~~~

Primarily the reporting of spam-related issues, though if it can be
tweaked
to be appropriate for the reporting of other issues, that's great.

The goal is primarily reporting from individuals (or a representative
of those individuals - such as AOL scomp infrastructure) to ISP abuse
desks rather than sharing of information between abuse desks,
postmasters, filter operators etc. There is space for bulk exchange
of abuse information, primarily machine-to-machine, but this is outside
the scope of this proposal (and likely to be covered by the INCH working
group).

Goals
~~~~~

It must be possible to receive and handle a report with a standard email
client, with no loss of information. That doesn't mean that there can't
be additional machine-readable information that could be used by a smart
ticketing system, but a Mom-n-Pop ISP with a copy of pegasus must be
able
to use it.

It should be possible to send a report with a standard email client. If
there
is some subset of the functionality that goes beyond a standard email
client
then the additional effort to implement that functionality should be
minimal.

Requirements
~~~~~~~~~~~~

To handle a report efficiently and reliably, with or without
automation, an abuse desk needs the following features:

 o A complete, uncorrupted copy of the appropriate offending email, or
   logfile data.

 o The reason the email was sent to them - particularly in the case
   where the same report was sent to multiple ISPs for different
reasons.
   Some way to do automated routing on that information would be nice.

 o Easy triage. A human must be able to glance at the email and
   understand the context in a few seconds. (If there is addidional
   metadata to allow automated routing that's great, but the data must
   still be available in a human readable form).

   The rule of thumb I've used for a long time is that the first 15
lines
   of the report must be adequate to triage the report.

 o It should be possible to reply to any report via email and reach
   someone who can provide further information.


Minimalist Proposal
~~~~~~~~~~~~~~~~~~~
For any report of unwanted email (whether that be spam, viruses, or
simply
an unsubscription request) the full email should be attached to the
report
as a MIME attachment (message/rfc822, ideally, but text/plain might be
supported too).

An exception to this might be this: In many cases the sender of the
email is only interested in the body of the message so they can pull
out a unique cookie (such as the message-id, the VERP envelope or
similar) that gives them all the information needed to categorise the
complaint. Most ESPs, for instance, categorise a mail by the triplet
of customer, mailing number and recipient number; that's enough
information for them to categorise a complaint and completely generate
the original mail. Perhaps if there's a header in the original email
of the form 'Abuse-Cookie: <email address> <opaque token>' and a report
is sent to the included email address then only the Abuse-Cookie line
need be sent.

For any report including system logs they must be attached in a
plain-text, human readable format as a MIME text/plain attachment.

(There are ISPs that decline to accept attachments because they use
 Outlook to handle their email, so requiring MIME attachments has a
 few issues. However, ISPs can fairly trivially filter their incoming
 abuse mail to either strip or reject attachments other than of the
 type above. Of course, if the message/rfc822 attachment contains an
 email virus there's still a problem - do we care, or is current virus
 filtering up to the problem?

 But a message with corrupted formating (line wrap, character encoding)
 can be impossible to parse automatically, and difficult to process
 manually, and MIME attachements are the only widely supported way to
 avoid that).

Any time a single MIME attachement is needed the same data should be
supported inline in the message, if possible.

A single line in the body of the message for each recipient, giving
enough information to triage the report. Each line should have an
(optional) tag to define which recipient the information is intended
for, a keyword chosen from a short set of keywords (TBD) to define
different complaint types, e.g SPAM - generic unwanted email
complaint, and an associated (optional) IP address, email address,
hostname or URL.

Following those lines, some human readable text with additional
information if needed. Providing additional (phone) contact
information here is strongly encouraged, especially for security
issues. I'd actually encourage ISPs to prioritise and place more
weight on reports with additional contact information if that
were easy to do.

So, to give an example or three:

For spam received from [email protected]:

    To: [email protected]

    topica SPAM [email protected]

    I didn't ask for this email, blah, blah, blah

    [MIME message/rfc822]

or

    To: [email protected]

    SPAM [email protected]

    [MIME message/rfc822]

or
    To: [email protected]

    SPAM

    [MIME message/rfc822]

or

    To: [email protected]

    SPAM

    Abuse-Cookie: [email protected] MSG-12345-1234567-12345

For a set of proxy attempts from 10.11.12.13, an example.com address:

    To: [email protected]

    PROXY 10.11.12.13

    [MIME text/plain]



For spam coming from a compromised roadrunner customer, 10.3.2.1,
advertising a website hosted on webstuff.net, bounced through a Yahoo
redirector:


    To: [email protected], [email protected], [email protected]

    roadrunner SPAM 10.3.2.1

    yahoo WEB http://rd.yahoo.com?http://spammerdomain.com/foo

    webstuff WEB http://spammerdomain.com/foo


    Blah, blah, spam, blah, blah.

    [MIME message/rfc822]

or

    To: [email protected], [email protected], [email protected]

    roadrunner.com SPAM 10.3.2.1

    yahoo.com WEB http://rd.yahoo.com?http://spammerdomain.com/foo

    webstuff.net WEB http://spammerdomain.com/foo


    Blah, blah, spam, blah, blah.

    [MIME message/rfc822]


This syntax is easy to generate, manually or automatically, and provides
enough data to easily triage the issue, again manually or automatically.

EOF