Re: Fwd: I-D ACTION:draft-hoffman-utf8headers-00.txt

Adam,

While this idea is an interesting one in principle, the
particular proposal you make would break a very large fraction
of the RFC822/2822 parsers in the world, which assume 
   Header = *C ":"
where "C" is an instance of a permitted character.

   They may then treat the character after the colon, if it is
not a space, as an error or as the first character in the field
that follows the header.  They will break either way.

While Paul and I continue to disagree about the level of badness
associated with transport bouncing of an extension, I think
that "deliver and then fail badly" is the worst of all possible
cases, since it may not even permit delivering a competent error
message.

      john


--On Saturday, 27 December, 2003 08:02 +0000 "Adam M. Costello"
<[email protected]> wrote:

> 
> Charles Lindsey <[email protected]> wrote:
> 
>> Far better to have some indication in the message that it is
>> contains 8bit stuff (most likely an extra header to say so).
>> 
>> So let me suggest a header so that UTF-8 users can mark their
>> messages as "unclean".
>> 
>> Header-Transfer-Encoding : "Header-Transfer-Encoding:" (
>> "8bit" / "7bit" ) *( ";" parameter )
>> 
>> It might be argued that this header SHOULD precede any use of
>> Non-ASCII in the headers (but given the propensity for
>> transports to reorder headers, I doubt that would survive).
> 
> Consider this:
> 
>     UTF-8-header-field = "8:" field-name ":" utf-8-field-body
> 
> where field-name is the same as always, and utf-8-field-body
> is like the normal field body for that field-name except that
> certain Unicode characters are allowed in certain places
> (encoded as UTF-8) (details to be worked out).
> 
> For example:
> 
>     8:From: blah blah <blah@blah>
>     Date: Fri, 26 Dec 2003 12:00:00 -0000
>     8:Subject: blah blah blah
>     8:Reply-To: blah blah <blah@blah>
>     In-Reply-To: <foo@bar>
> 
> (Pretend "blah" is non-ASCII text.  At the moment I'm using a
> crippled terminal and cannot generate such examples.)
> 
> This would automatically satisfy the goals you describe above.
> Every message that contained any non-ASCII header text would
> contain a particular field whose presence could be easily
> checked for ("8:"), and this special field would automatically
> appear before the first occurrence of non-ASCII text, even if
> the fields were reordered.
> 
> User agents might want to elide the "8:" for display purposes.
> (That probably won't be the only alteration made for display
> purposes.  For example, I imagine that a regular Date: field
> would get displayed with the word "Date" and the date itself
> translated into the local language.)
> 
> There is room for future expansion simply by creating a new
> special field (like "8a:").  Or we could insert an extra colon
> in the syntax now:
> 
>     8::From: blah blah <blah@blah>
> 
> and allow parameters between the first two colons.
> 
> There would be two methods for downgrading.  For fields whose
> syntax is known, you can remove the "8:" and use
> encoded-words, IDNA, IMAA, Address-Map, and/or whatever.  For
> fields whose syntax is unknown, you can use another special
> field:
> 
>     downgraded-header-field = "7:" FWS field-name ":"
> downgraded-field-body
> 
> for example, given an unrecognized UTF-8 field:
> 
>     8:Prior-Subject: blah blah blah
> 
> it could be downgraded to:
> 
>     7:Prior-Subject: ASCII-ENCODED-GARBAGE
> 
> The conversion from UTF-8 to 7bit would need to be worked out,
> but it would be an opaque reversible string conversion.  Any
> user agent that understands 8: would easily understand 7: as
> well.
> 
> AMC
>