Re: Requirements for a filtering header

Philip Miller wrote:
>> There has been discussion back and forth on different aspects of this 
>> problem. At this time, I want to get a feel from the members as to:
>> 1. If there is a rough consensus that a standard filtering header would
>> be useful.
> 
> I think we have pretty clearly agreed that such a header would be useful.

Here are the benefits, which can eventually end up in our final spec 
abstract and/or introduction:
1. Save filter authors the trouble of reinventing the wheel when designing 
the mechanism to report results to the user
   a) No need to invent new syntax, unused header name, etc.
   b) Pre-existing library to generate/parse the specified syntax [1]
2. Give MUA authors a well-defined, unambiguous standard header to interpret
3. Allow better interaction among filters
4. Facilitate future efforts at MUA -> filter communication

All of the above eventually resulting in improved end-user experience with 
filters that can now more easily 'just work'.

>> 2. If there is a rough consensus as to what issues need to be solved for
>> such header to be useful.
> 
> Major existing issue:
> Should the header appear only once, representing the results of the
> last-acting (and therefore most trusted?) filter, or should it appear once
> for each analysis action?

I believe that such a header should appear in a message as many times as 
that message is processed by any relevant piece of software that passes the 
message on rather than rejecting it outright. This allows the user (and the 
software acting on the user's behalf) access to as much data about a message 
as possible.

As I write this, I've thought of another use-case in which allowing multiple 
copies is good. What if we want multiple filters to process a given message 
on a given host at the same time? The pass-through model, currently used by 
Postfix, only allows a single piece of software to handle the mail at a 
time. Instead of this, how about running each of these tasks in parallel, 
with the message on the scanner's standard input, each of them writing their 
own tag to standard output, all of which the controlling MTA process adds to 
the message.

>> If these are satisfied, than I would like to ask for some folks to
>> volunteer to write up a requirements draft from which we can continue the
>> discussion.
> 
> Please try not to be so document focused, at least until the point where 
> we're actually writing a specification. If it's that important, I can 
> throw this stuff in the wiki.
> 
> At any rate, once we have the above major issue resolved, I see the need 
> for the following things to be determined:
> 1. Syntax of this header

I would prefer strict key-value pairs, but if we go with some fixed fields, 
I'm fine with that too, just so long as there is room for specified 
extension with a low barrier to entry.

> 2. Fields
>    a) mandatory

I would support making a number of fields mandatory in all instances of the 
header, including:
1. Host on which processing occurs
2. Timestamp of processing
3. Name of processing software
4. Version of processing software

>    b) optional, but specified in the base spec

My list of these would be:
1. Virus - yes/no
2. Virus name
3. Virus forges headers (more specific? necessary at all?)
4. General Classification - i.e. what POPFile is designed to provide
5. Confidence in General Classification - statistical certainty of result

Note that a set of spam-specific fields is quite superfluous. The value of 
the 'General Classification' field can be 'spam' or 'junk' or whatever, and 
the user can set all such things to go in one folder, be deleted, get a 
challenge, etc.

>    c) IANA registration of new fields

I think this is absolutely crucial to success, and I see no reason not to 
require full syntax specification of the values such fields will contain.

> 3. Name of this header

Right now, I'm leaning towards the most-generic 'Processed', as a more 
active sounding parallel to 'Received'.

Philip Miller

[1] When we get a spec ready for publication, we should probably have a few 
sample implementations in different languages. Both generators and parsers.