Re: Realisticness of header rearrangement

david nicol wrote:
> On Tue, 2004-03-30 at 19:24, Philip Miller wrote:
>> However, ordering is not guaranteed in the transport system, 
>>which is relevant in at least one not-uncommon case:
>>Bob has an address, [email protected]. The MX for ieee.org scans and tags 
>>messages and then forwards them on to [email protected]. In transit between 
>>ieee.org and other-isp.net, the headers get rearranged.
> 
> Rearranged by what?
> 
> By a router? no. Routers operate at the IP layer and are oblivious
> to the nature of the traffic passing through them.
> 
> By a relay? relaying went out of style a decade ago.

The forwarding I used as an example, and the relaying of my mail that you
used as an example below, could potentially present problems if some
provider not directly controlled by the final recipient uses MTA software
that rearranges headers.

> ieee, doing the tagging, will have a non-rearranging outbound MTA that
                                        ^^^^^^^^^^^^^^^
That's the statement I don't agree with. Because the standards don't require
non-rearrangement, we should not depend on that behavior.
> connects directly (no intervening SMTP relay) to other-isp.net's
> inbound. I will allow that other-isp.net's inbound MTA might rearrange
> headers between receipt and delivery, but the concept of a
> interconnected web of SMTP relays is completely obsolete in my opinion.

OK, perhaps I didn't state my example too clearly above. I meant that
other-isp.net, or some smarthost that ieee.org goes through, rearranges the
header. Not that the mail was simply being sent along a bang path one closer
to the final destination.

> I am not opposed to including an optional checksumming and signing
> protocol but I do not believe it to be a core requirement, especially
> since it assumes distribution of public keys of filtering agents.

I am not opposed to signing or checksumming mechanisms as an optional
extension. However, as the sole method of authenticating a tag header, I am
opposed to them. I am opposed to them as the sole authentication method
because they are much more expensive to compute than copying the date stamp, 
and that could needlessly hinder adoption by large service providers.

> I think a key included in both Received and Tagged is superior
> to reusing the timestamp.  Tagged would have two timestamps, if
> there was a delay, and that would be confusing.  Received usually names
> the receiving agent and domain, so Tagged could use the same identifying
> information.

As far as I can tell, we are trying to design this header so that the vast
majority of its use will be in a parsed form by a GUI MUA that presents the
contained information to the user in a useful format or processes the mail
as a result of the tags attached. MUA programmers are quite capable of using
two separate variables independently.
Anyone reading headers, who we can assume is a fairly advanced user, will
learn pretty quickly that this one has 2 time stamps, and they are used for
different things.

> About PK-signing individual e-mail header lines? Cold we just punt that
> concept to a working group on PK-signing individual e-mail header lines
> instead of tackling it here? Reccomend that if a standard emerges, the
> filtered/tagged line should follow it?

I don't think we need any particular way to sign headers. It's an expensive,
silly proposition with limited use.

> Here are a couple header lines for reference:
> 
> Received:  from fda.zemos.net ([127.0.0.1]) by localhost (fda
> [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 20168-03; Tue, 30
> Mar 2004 20:24:38 -0500 (EST)
> 
> Received:  from zemos.net (phil [10.0.0.2]) (using TLSv1 with cipher
> DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by
> fda.zemos.net (Postfix) with ESMTP id 5BDD41867F; Tue, 30 Mar 2004
> 20:24:38 -0500 (EST)
> 
> Received lines generally identify themselves with hostname and
> software used.  Tagged lines could do the same.

And I think they should, but for debugging and trust reasons. The tuple
(hostname, received-timestamp) in a tagged header authenticates that said 
tag was added with or after the Received header it matches. The user can 
choose to trust or not trust the judgement made at that point in the system.
Note: when I say 'user' here, I mean 'the user or the software under the
user's direct control'. That would be the MUA, and in my case, my private
email server, fda.zemos.net.

> Yes, I trust that Philip Miller uses a secure connection to upload
> e-mail to zemos.net; that zemos.net uses amavis anti-virus software;
> that zemos.net then passes its e-mail to comcast.net, which forwarded
> the message to the asrg mailing list on iecc.com, where qmail copied it
> a couple times before sending me a copy from a VERP return address.

The first half of that trust is misplaced. For all you know, I connected
directly to the iecc.com MX and forged all the prior headers. As long as I
had the right IP address, it would not be too hard. Perhaps viruses will
start forging such received headers, to make them look like they've been
scanned.
Basically, it's impossible to trust headers that were added on 'my side' of
the mailing list server. Making sure that tag headers are uniquely
associated with specific Received headers means that you can choose exactly
which step to start trusting.

> Received:  from xuxa.iecc.com (xuxa.iecc.com [208.31.42.42]) by
> tipjar.com id i2V1Onnt012224; Tue, 30 Mar 2004 18:24:50 -0700 (MST)
> X-Received-From:
> [email protected]
> X-Received-For:  <[email protected]>
> Received:  (qmail 729 invoked by uid 85); 31 Mar 2004 01:24:47 -0000
> Received:  (qmail 719 invoked from network); 31 Mar 2004 01:24:44 -0000
> Received:  from rwcrmhc13.comcast.net (204.127.198.39) by mail2.iecc.com
> with SMTP; 31 Mar 2004 01:24:43 -0000
> Received:  from fda.zemos.net ([68.38.12.170]) by comcast.net
> (rwcrmhc13) with ESMTP id <20040331012440015007pgdte>; Wed, 31 Mar 2004
> 01:24:40 +0000
> Received:  from localhost (fda [127.0.0.1]) by fda.zemos.net (Postfix)
> with ESMTP id EE91518683; Tue, 30 Mar 2004 20:24:38 -0500 (EST)
> Received:  from fda.zemos.net ([127.0.0.1]) by localhost (fda
> [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 20168-03; Tue, 30
> Mar 2004 20:24:38 -0500 (EST)
> Received:  from zemos.net (phil [10.0.0.2]) (using TLSv1 with cipher
> DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by
> fda.zemos.net (Postfix) with ESMTP id 5BDD41867F; Tue, 30 Mar 2004
> 20:24:38 -0500 (EST)
> Message-ID:  <[email protected]>
> 
> The fact of the comcast step between zemos and iecc leads to the
> conclusion that zemos is on comcast's network, since in general
> postfix is perfectly capable of delivering without using a smarthost.

Close enough. I use the comcast smarthost because too many sites block
connections from end-user cable systems. Pretty much all of comcast's public
IP blocks are blacklisted all over the place. On the other hand, so is one
of their outgoing SMTP hosts at times. Less of my mail bounces this way than
if I did direct-to-MX from my server (fda.zemos.net).

> Were a Tagged: header to be a standard, amavisd, which is a virus and
> spam filter, is the sort of program that would add one. On what I guess
> is Phil's firewall, the Received: header currently added by amavisd
> when postfix hands incoming to amavisd and then gets it back for
> upstream delivery might be a Filtered: header _instead_of_ a Received:
> header
> 
> Tagged:  Host=(fda[127.0.0.1]); Tagger=amavisd-new; Tue, 30 Mar 2004
> 20:24:38 -0500 (EST); Result=CLEAN

I would write it more as follows:

Tagged: Received-from=zemos.net (phil [10.0.0.2]); Received-time=Tue, 30 Mar
    2004 20:24:38 -0500 (EST); Host=fda.zemos.net; Tagger-name=amavisd-new;
    Tagger-version=xxxxxx; Virus=no; Tagger-time=Tue, 30 Mar 2004 20:25:42
    -0500;

Note the following:
1. Everything is a key-value pair, timestamps and all.
2. This tag was provably added after a particular receipt in the mail
transport system. It's ambiguous

The timestamp in yours matches the timestamp in the Received lines, but I
think it would be useful, for debugging, if nothing else, for tags to
include the time they were added, as it may differ by quite a while from the
time of receipt. In the case of resource-intensive scans, such as virus and
statistical or DNS-lookup-based scanners, being able to attribute a delay to
a specific scan is a good thing.

Philip Miller