Re: Fwd: I-D ACTION:draft-hoffman-utf8headers-00.txt

Martin Duerst <[email protected]> wrote:

> I definitely like a header much more than the 8: header prefix
> proposal, because ... 8-bit-clean software can just work on headers
> without having to care about 7-bit/8-bit issues except at very
> specific points (downgrading/upgrading).

Charles Lindsey <[email protected]> wrote:

> ...I don't like the "8:" header prefix.  In some environments (notably
> Netnews) it would be much simpler to leave the headers in their
> present form (otherwise, all agents will have to learn to recognise a
> new set of headers which are really just synonyms for existing ones
> - that could be true of mail user agents too).  The advantage of the
> special header is that agents that don't need to be aware of the
> distinction can just ignore it.

There seems to be assumption here that existing "8-bit clean" software
will automagically understand "UTF-8 header fields" that use the same
field-names as existing ASCII header fields.  But "UTF-8 header fields"
have not even been defined yet, and there are plenty of important
details to work out.  All standard header fields (like To:) are defined
by grammars that currenly allow only ASCII characters.  UTF-8 header
fields would have different grammars.  Exactly which Unicode characters
would be allowed, and where?  The Unicode standard recommends that
equivalent strings be treated the same.  Will that be true for UTF-8
header fields?  If so, it means normalization needs to be done at some
point.  At what point?  When the field is created, or when it is parsed?
Which normalization, canonical or compatible?  Or some profile of
Stringprep?  What profile?

Given that none of these questions have been answered yet, how can we
expect existing "8-bit clean" software agents to interoperate if we
throw "UTF-8 header fields" at them?  I don't think we can.  I think
if we define UTF-8 header fields, we'll need to acknowledge that they
are new header fields with a new syntax and new requirements, and that
they can be properly handled only by new software that is aware of the
new rules.  If we need new software anyway, to accomodate the various
changes mentioned above, then it's no big deal to accomodate one more
slight change, like an "8:" prefix.  Encapsulating each UTF-8 header
field inside a new 8: header field would ensure that existing "8-bit
clean" software does not attempt to digest things that it has no proper
understanding of.

Keith Moore <[email protected]> wrote:

> the possibility exists that the gateway isn't aware of the utf-8
> extension, so it injects messages with utf-8 headers and addresses
> into the legacy mail system without doing a conversion.

Yes, that's a good argument that UTF-8 header fields are likely to
fall unexpectedly into the hands of software that doesn't know how
to handle them.  If UTF-8 header fields use the same field names as
the corresponding ASCII header fields, there's no telling what will
happen.  By using new never-before-used field names for the UTF-8 header
fields (or even an entirely different header format) we could avoid that
pitfall.  The 8: idea is one way to do it; we could certainly imagine
others.

By the way, although I'm trying to help sort out what would be a good
way to define non-ASCII headers, I have no strong position on the
question of whether they should be defined at all.  I see good arguments
on both sides of the issue, and I'm biased toward sticking with the
status quo unless the alternative clearly has more benefit than cost.

AMC