Re: Fwd: I-D ACTION:draft-hoffman-utf8headers-00.txt

Charles Lindsey <[email protected]> wrote:

> Far better to have some indication in the message that it is contains
> 8bit stuff (most likely an extra header to say so).
>
> So let me suggest a header so that UTF-8 users can mark their messages
> as "unclean".
>
> Header-Transfer-Encoding : "Header-Transfer-Encoding:" ( "8bit" /
>   "7bit" ) *( ";" parameter )
>
> It might be argued that this header SHOULD precede any use of
> Non-ASCII in the headers (but given the propensity for transports to
> reorder headers, I doubt that would survive).

Consider this:

    UTF-8-header-field = "8:" field-name ":" utf-8-field-body

where field-name is the same as always, and utf-8-field-body is like
the normal field body for that field-name except that certain Unicode
characters are allowed in certain places (encoded as UTF-8) (details to
be worked out).

For example:

    8:From: blah blah <blah@blah>
    Date: Fri, 26 Dec 2003 12:00:00 -0000
    8:Subject: blah blah blah
    8:Reply-To: blah blah <blah@blah>
    In-Reply-To: <foo@bar>

(Pretend "blah" is non-ASCII text.  At the moment I'm using a crippled
terminal and cannot generate such examples.)

This would automatically satisfy the goals you describe above.  Every
message that contained any non-ASCII header text would contain a
particular field whose presence could be easily checked for ("8:"),
and this special field would automatically appear before the first
occurrence of non-ASCII text, even if the fields were reordered.

User agents might want to elide the "8:" for display purposes.  (That
probably won't be the only alteration made for display purposes.  For
example, I imagine that a regular Date: field would get displayed with
the word "Date" and the date itself translated into the local language.)

There is room for future expansion simply by creating a new special
field (like "8a:").  Or we could insert an extra colon in the syntax
now:

    8::From: blah blah <blah@blah>

and allow parameters between the first two colons.

There would be two methods for downgrading.  For fields whose syntax
is known, you can remove the "8:" and use encoded-words, IDNA, IMAA,
Address-Map, and/or whatever.  For fields whose syntax is unknown, you
can use another special field:

    downgraded-header-field = "7:" FWS field-name ":" downgraded-field-body

for example, given an unrecognized UTF-8 field:

    8:Prior-Subject: blah blah blah

it could be downgraded to:

    7:Prior-Subject: ASCII-ENCODED-GARBAGE

The conversion from UTF-8 to 7bit would need to be worked out, but it
would be an opaque reversible string conversion.  Any user agent that
understands 8: would easily understand 7: as well.

AMC