Re: if you really want utf-8 headers...

> Even potentially without an SMTP extension, I think that defining
> a new header format and a way to down- and upgrade is important,
> because only this will allow things such as simplified clients
> based on upgraded delivery agents,...

I think we should think in terms of a new message format, not merely a 
new header format, because MIME is so complex and so irregular (some 
would say baroque) that the amount of simplification you get from 
having utf-8 in message headers is only a small part of that which 
could be gained.  even if you have utf-8 transparency, mail readers 
still need to know how to parse address fields, normalize/canonicalize 
addresses, look up IDNs, etc.  you still need to deal with RFC 2047 in 
received messages and old messages.  you still have to support a 
different syntax for each different header or bodypart field.  compared 
to all of this cruft, the extra overhead required to translate 
addresses between ACE and raw UTF-8 is minimal.

> And once we are there,
> defining an SMTP extension isn't really a big deal, although
> adoption may take quite a while.

the big deal is the leakage and damage to messages that we can expect 
from the extension.

>> b) we could use the expensive upgrade to drastically improve email
>> service in many other ways than just to provide IMAs.
>
> If it were up to you alone to choose, what would you do in that 
> upgrade?

off the top of my head?

for mail transport:
- clean separation between relaying and submission - MUAs talk only to 
submission servers
- mutual authentication (say, based on TLS) would be required
- all messages reliably traceable to origin submission server
- sending MTAs would require external registration (say, in DNS)
- binary transparency assured
- true pipelining
- checkpoint/restart
- engineered so that most failures and configuration errors are 
detected and reported at the point where corrective action can be taken
- well-defined behavior for multiple servers
- discourage store-and-forward processing - most messages should make 
one hop from the submission server to the recipient's message store  
(pass-through processing would be available for routing through local 
proxies/firewalls)
- option to request immediate (pass-through) delivery
- ability for submission server to poll for delivery completion 
(submission server is responsible for ensuring that the message is 
delivered rather than expecting a message to be mailed back to the 
sender)
- explicit support for proxies/firewalls/filters  (i.e. the behavior is 
defined as part of the protocol)
- fast, easy to parse PDU format
- built-in ability to query recipient capabilities, including recipient 
filtering preferences
- built-in content negotiation ability
- [maybe] ability to use e.164 addresses as recipient addresses (for 
voice mail, fax, sms, etc.)

for message format:
- text header fields in utf-8
- binary transparency for all body parts and extension fields
- extremely regular, easy to parse, probably binary, format
- directories for multipart messages to allow quick access to 
individual message components
- alignment with [2]822/MIME data model, field names, content-type 
names, charset names, etc.
- s/mime compatibility (yes, I think I know how to make this work)
- clean separation between: envelope/trace information, information for 
the recipient UA, information supplied and/or used by recipient message 
store
- ability to associate additional information with header addresses - 
multiple names (in different languages), spoken name, photo, alternate 
addresses (including E.164 & IM), web page URL.
- ability to specify which recipients the message author(s) think 
should be included, by default, in a reply-to-all
- all messages traceable to origin domain, with opaque "nonce" 
sender-ID traceable to message author (with provision for author to be 
anonymous, but recipients don't have to accept anonymous mail)
- well-defined spec for downgrading to 2822/MIME such that there is a 
repeatable, "canonical" translation

of course, the devil is in the details, and I'm sure I wouldn't get 
agreement on all of these features.  but I'm convinced most of this is 
technically feasible.