Re: UUCP, etc., and SMTP/822/MIME mail (was: Re: I-D ACTION:draft-hoffman-utf8headers-00.txt)

> >> Keith, while I agreed with (and strongly supported) that
> >> reasoning at the time, in the ensuing eleven or so years
> >> Unicode  (and maybe UTF-8) have achieved sufficient adoption
> >> that it  might now be reasonable to say "systems injecting
> >> non-ASCII  characters into header fields or equivalent
> >> contexts are  required to take responsibility for converting
> >> to UTF-8"...
> >
> > that's onerous but more-or-less doable.  what doesn't seem
> > doable is to prevent subsequent systems that handle the
> > message (or reply to it) from adding their own, non-utf-8,
> > header contents.
> 
> I'm not sure I see the issue.  At one level, nothing can prevent 
> anything or anyone from adding trash, anywhere they like.

no, but the "reply" operation is fairly normal - in particular,
taking existing to/cc/reply-to and subject header fields, adding 
new things to them, removing other things, and generally rearranging
them are all common operations - and these "new things" can be
from machine or human sources.

> At another level, if there is a 
> specification that says "if you add 8bit header content, it must 
> be UTF-8; anything else must either be converted into RFC 2047 
> form or must be converted to UTF-8", then we are probably ok. 

well, we already have widespread practice of taking rfc 2047 and
decoding it into whatever charset the MUA happens to want to use -
mixing utf-8 with that probably produces unpredictable results,
and insisting that all non-tagged non-ASCII text is utf-8 is probably
just naive.

> We _could_ have adopted a rigid, Unicode-only, rule a decade ago 
> rather than doing charset-specific tagging in text content types 
> and 2047 encodings. 

IIRC, unicode wasn't known to be stable at that time.  it certainly
wasn't widely adopted, and it's hard to imagine that we could have
gotten consensus on such a rule.  it's only after 10 years' experience
with unicode that we have some confidence in its character repertoire,
and we have even less experience with other aspects of it.

> It would have made many things more simple. 
> But it would not have been practical, since, despite the claims 
> and optimism of their advocates, Unicode (in UTF-8 form or 
> otherwise) wasn't nearly widely enough deployed.   But, in the 
> ensuing years, it has gotten more widely deployed and, at least 
> as important, we have made a number of other decisions and 
> standards, including IDNA, that assume it.   So a "UTF-8 only" 
> decision just takes a step further down a path to which we are 
> clearly already committed.

arguably utf-8 only isn't even workable today, because of Chinese 
government regulations.  but I'm not arguing that we should further
encourage diversity in character encodings; rather I'm arguing that
we should avoid disrupting the installed base - and that we to the
extent that we are going to disrupt it, IMAA is very dubious as a
sole justification for doing so.