Re: Moving forward ...

> Here are some thoughts on some of these issues.
>
> > * spam filtering isn't the only potential filter running.  A filter
> > header specification should be extensible to other filtering types.
>
> Filtering headers add to the information processed downstream.
> To prevent unwanted feedback (ie a procmail filter catching
> on to a SpamAssassin recommendation), the spec needs to be so
> that added headers can be recognized by all and skipped. This
> suggests a common prefix such as "X-SPAM-", so that the
> programs downstream don't need extensive logic to identify
> and skip headers.

Agreed.  These headers have to be simple enough that a dumber MUA with a
simple rules system can find the header information and sort the
messages accordingly.  We've seen that customers want a label and a
confidence to allow the recipients to set their own spam thresholds.
(this particularly applies to non-commercial deployments)

>
> Similarly, adding spam scores or "convenience" functions
> (e.g. URLs) inside the body of an email should be prohibited.
> Anything that makes it so the downstream filter cannot see
> exactly the same message as
> the upstream filter is unstable (ie if you remove a filter in
> the chain, you've sabotaged training for all filters downstream).

Agreed.

> > * The definition of spam isn't universal and filtering technologies
> > may have subtle differences in their implementation of spam
> detection.
> > To this end, labelling a message as spam may not be consistent.
> > (unwanted vs. UCE or some combination thereof)
>
> Moreover, the labelling may not even be consistent over
> several versions of the same filter...
>
> >
> > * filtering decisions are dependant on the state of the filtering
> > (virus definitions, software version) at the time of
> message scanning
> > resulting in different results for the same message at different
> > times.  It is important for filters to be able to add meta
> data to the
> > label.
>
> This is bad. I'd much prefer a system where my filter can
> easily ignore anything done by other filters. Let the user
> decide how to
> integrate all the various recommendations.

Yes, but the headers have to express this information in a way that
lotsa different rule systems can understand them.  (from procmail to
simple header pattern matching)

>
> >
> > * some filters develop a confidence as well as a label,
> allowing later
> > processing to be tuned to reduce errors.  Some filters are
> unable to
> > parse these confidence levels.  A common format for
> confidence is also
> > required.
>
> Is the confidence intended for the originating filter only,
> or for every filter downstream (including the MUA).
> Confidence on its own doesn't mean much without reference to
> the filter algorithm. Other filters which use different
> algorithms would have to be intimately familiar with the
> workings of the originating filter to make use of
> the stated confidence.

See above thoughts about confidence.

>
> >
> > * Labels may be added to the message prior to gateway filtering (on
> > the outgoing MTA?) and may not be trusted.  How to tell if
> a label was
> > added by the trusted MTA?  What should an MTA do if it receives an
> > incoming labelled message?
> > 	- this message was virus scanned by ...
>
> I'm not sure trusting is the way to go. I'd rather leave this
> to the user. In other words, when a message arrives with a
> spoofed label from filter X, then:
>
> If independent filter Y sees the label, it should ignore it.
> If filter X sees it, it should recompute the label anyway and
> overwrite. If MUA sees it, it's up to the user to decide.
>
> Again, I really don't like the idea that filter should
> somehow cooperate with an upstream filtering system.

I just threw this in, as untrusted, incoming assertions should be
considered in, and potentially explicitly excluded from, any
internet-drafts developed by this group.

>
> >
> > * Recipients may believe that a message was labelled in
> error and want
> > to provide as a defect to the administrator.  This
> administrator (or
> > the filter vendor) will need more meta data about the
> filtering than a
> > single judgement label.  More state data will be required
> to diagnose
> > and adjust/reject the feedback.
> >
>
> This suggests a different type of labelling is already used
> by some filters: instead of advisory labels only, the message
> also carries state information. I think that means we need to
> discuss two different types of headers separately:
>
> 1) headers which advise the downstream programs.
> 2) headers which are meant to be private to the filter.
>
>
> --
> Laird Breyer.
>