mail header rfc validation

Hi all,

I'm new to this subgroup, and what you could call a bayesian filter writer.
I'm interested in starting a discussion on RFC compliant header formatting,
which seems to be under the purview of this group.

I'm the author of a statistical mail classifier called dbacl (can be
found on sourceforge http://dbacl.sourceforge.net). Right now, I'm
toying with the problem of validating message headers according to the
relevant RFCs. I've started writing some GPL code which will end up as
part of the previously mentioned dbacl project, but it's early days.

My general (long term) aim is to extract as much information as
possible from the headers, and then perform consistency checks, and
possibly obtain probable sender info. The first step for this requires
parsing the headers correctly. I'm hoping there are some experts
reading this subgroup, who can offer advice to a newby.

An advantage of discussing this here is that the subgroup can build up
an archive of somewhat relevant practical information, which should
come in handy for its other aims.

Enough introductory material, on to my first problem:

I'm currently trying to parse the Received: lines and I'm finding that
the grammars in rfc2822 and rfc2821 are slightly contradictory, at
least the way I interpret them. Possibly, this is due to typos (or my
failure to read correctly), but which standard is authoritative?

For example, in rfc2821, on p.36, the "Domain" must have at least one
period.  This "Domain" is used in the Stamp (p.51) after the keywords
"from" and "by".  But in rfc2822, the "domain" is defined on p.16 as
either a "dot-atom" or "obs-domain", which both need not have a
period, or a "domain-literal" which is within square brackets.

So a line which looks like

Received: from localhost by ...

is not rfc2821 compliant, but is rfc2822 compliant. 

However, now consider the definition of "ID". In rfc2821, the token
after an "id" is an atom or a quoted string (p.51). But in rfc2822, 
an "item-value" (p.27) is either an atom, or contains an "@", "<" or ".", but 
cannot be a quoted string, as far as I can make out. So if we have
a line such as

Received: from localhost.localdomain ... id "hello" ...

then this line is rfc2821 compliant but not rfc2822 compliant.

So I now know (I think) that these two rfcs generate different sets of messages. Thinking ahead to spam filtering and header validation, is there a consensus on which rfc is more useful, or what the domain of applicability is for each?
For example, would legitimate Received: lines (as opposed to fake ones added by spammers) follow rfc2821 rather than rfc2822?

There are similar issues with the older rfc821/rfc822, but I'll leave that
for now.

All types of comments are welcome.

Regards,
Laird Breyer. 


 







-- 
http://www.lbreyer.com
The mind boggles, but does the Boggle mind? The mind boggles...