Re: Problems of Internationalized Mail Address eXtensions (IMAX)

Paul Hoffman / IMC <[email protected]> writes:

> At 10:13 PM +0100 2/25/03, Simon Josefsson wrote:
>>Isn't (one
>>of) the goal of the IMAA protocol to make it possible for MTA
>>implementations to support non-ASCII?
>
> Asking ludicrous questions is not a good form in technical
> discussions. Of course that is a goal. And IMAA does that already.

My question was sincere.  IMAX appears to be a solution for
internationalization of MTAs, at the SMTP layer.  It does not propose
solving the internationalization problem for MUAs.  SMTP is an
interactive protocol between two end-entities, and can therefor
negotiate non-ASCII support, which is different from RFC (2)822 where
all entities that will handle the stored data is not able to interact
with the creator of that data to negotiate non-ASCII.  IMAX takes
advantage of this difference.  I believe it would be possible to
design a internationalization solution for RFC (2)822 that would be
distinct from a SMTP internationalization solution.  Those two
distinctions could be investigated in parallel and evaluated on their
own merits.  If you think this is ludicrous and want this to be a
productive discussion, please take the question seriously and explain
in technical terms why your proposal is better.

>>The end result is that punycode decoding is required
>>in the implementation, which is what I consider the problem.  If a
>>solution that didn't involve encoding techniques such as punycode
>>could be developed, I think that should be preferred.
>
> Then you are not talking about IMAX. If you have some other protocol
> in mind that doesn't require punycode but still guarantees mail
> delivery, please write an Internet Draft for it. (If you are thinking
> of a protocol that doesn't require punycode but would instead simply
> bounce or lose mail that was sent to MTAs that didn't understand the
> new protocol, please don't bother writing an Internet draft...)

Why not?  That seems to be one serious alternative solution to IMAA.
I can only interprete your dismissal of alternative solutions without
a serious analysis that you either have done this analysis already and
know the answers or that you don't want to see alternative ideas
discussed.  In the former case, I think it would be useful to read
your analysis.

>>Now that you write this I would agree that IMAX is unclear on
>>one thing:
>
> One of many things....

Perhaps your experience in this area could be applied at improving the
specification?  I'm sure having two serious alternative to look at
would help the discussion.

>>  > You are free to say that. Others would disagree. In the case of IMAX,
>>>  what would you want in your log file. All UTF-8? That means you need
>>>  converters from every accepted charset to UTF-8. Careful sysadmins
>>>  would probably want to know *exactly* what came in, not some converted
>>>  form, but that means that their log file would have multiple charsets
>>>  in it, which would make display a mess. A reasonable option is to
>>>  store the addresses as ACE and to have a log-file viewer that converts
>>>  on display (and has an option for not converting).
>>>
>>>  Again, this is an implementation issue, not a protocol issue.
>>
>>Yes.  But it is an important point.  A internationalization solution
>>that doesn't consider these practical issues is of only theoretical
>>value.
>
> So the current SMTP, POP, IMAP, and HTTP protocols is only of
> theoretical value. Oh, well.

Those protocols do consider practical issues.  A simple proof that
they aren't of theoretical value is that they are used in practice.

>>I would want the log file to contain...
>
> Fine. Ask your vendor to include that feature. This is not part of a
> protocol specification.

I'm the vendor, and I'm here to understand how to implement it.  If
the protocol specification doesn't give guidance or have considered
how it will be implemented, I fear it will not work.  If you take this
fear seriously and want to prove me wrong, please explain how it can
be done in the real world.  This exercise would be useful when
comparing IMAA with IMAX since it would give a complete picture.

>>It seems we disagree that it is reasonable to require users to use
>>special applications to view log files, or edit configuration files,
>>etc.
>
> No. We disagree as to whether this is part of the protocol. Few (if
> any) IETF protocols cover this.

Few (if any) IETF protocols have designs that makes this a problem.  A
ESMTP extension with tagged charsets might not make this a problem,
but IMAA does.  If the IMAA design makes you propose that a reasonable
approach is to implement special applications for viewing log files or
edit configurations, a valid critique of IMAA would be that
alternative solutions would not generate these problems.  Dismissing
this critique because the IMAA protocol doesn't clearly state the
consequences of its design or declare it out of scope isn't
productive.

>>  > IMAA describes in detail when and how to display the Unicode form to
>>>  the user; IMAX mostly glosses over this.
>>
>>Yes, IMAX is not a final document so this isn't surprising.
>
> Neither is IMAA. At least the IMAA authors admit where the open issues are.

Are you accusing the IMAX authors are holding out on what the open
issues are?  I can't tell, but of course as an evaluator of both
specifications I'd appreciate if you could disclose those problems.

>>I'm sorry, I'll try to make it more clear when I talk about the
>>implementation or the specification.  If you are saying that we should
>>simply ignore all implementation related aspects in a proposed
>>solution, then I guess I simply don't agree with that.  I'll continue
>>to relate a proposal to the real world.
>
> This is a discussion of a potential IETF protocol. Please hold your
> discussion to things that could be included in an IETF protocol. If
> you don't like the way the IETF makes protocols, there are other
> standards organizations in which you might want to be active instead
> of the IETF. Or, you can take your concerns about the way we create
> protocols to the main IETF mailing list and see if there is enough
> support for your views to change the way the IETF works.

I'm sorry, I was under the impression that the IETF worried about how
protocols are implemented in practice too.

>>  > Correct, and IMAA describes when and how to convert for display.
>>
>>Right.  This is what cause the dependence on punycode decoding.  Since
>>administrators not only view non-ASCII but input non-ASCII too,
>>punycode encoding is required too.
>
> Wrong, yet again. There is nothing in the IMAA document about how the
> administrator views documents. IMAA is about mail transport, not
> system administration.

You said that a reasonable approach to implement a non-ASCII solution
based on IMAA was to implement special applications for viewing log
files and editing configuration files.  I don't consider this a
reasonable solution, and thus object to IMAA based on this.  In a
discussion, the productive response to this criticism would be to
explain how IMAA can accommodate other views as well.  If IMAA cannot
accommodate this view, all it would take to say that and I'll be
enlightened.  I agree the IMAA document doesn't answer my question,
that's why I brought it up.

>>  > If you want to propose a ESMTP extension with no fallback, either
>>>  change IMAX or create your own Internet Draft. In either case, you
>>>  will have to say explicitly how this will interact with SMTP servers
>>>  that do not support the new protocol, how bounces would be handled,
>>  > how users would know if they could send a message, and so on. I
>>>  think when you write that, if you do so honestly, you will see that
>>>  it would be silly to propose such a solution.
>>
>>Discarding it as silly seems a bit premature to me.  Having such a
>>proposal, that discusses all the consequences you mention seems like a
>>valuable contribution to this discussion.  But I guess it is easier to
>>advocate one solution if the competition are discarded early on...
>
> We disagree here. This discussion has been happening for over 10 years
> with respect to ESMTP extensions. I don't consider that to be "early
> on".

If an idea can't be dismissed after 10 years of discussion, perhaps
there is some merit with that idea.  Since technical solutions are
continuously proposed based on the idea, perhaps it would be useful to
document why that idea is a bad one, if you believe that.

>>  > I think I hear you saying that you think that the protocols should
>>>  allow any repertoire and any encoding of those repertoires. If so, we
>>>  certainly disagree. The IETF is not very keen on creating protocols
>>>  for which there would be limited and unpredictable
>>>  interoperability. Other standards group might not be so picky.
>>
>>That is stretching it a bit, I think.  I believe that a solution worth
>>its salt should consider existing habits, and whether we like it or
>>not there is more than charset used on the Internet.  MIME appears to
>>acknowledge this and is rather successful.  HTML acknowledge this and
>>is rather successful.  Same for HTTP.  Come to think of it, I can't
>>recall any successful internationalization product the IETF has
>>produced to counter my examples, can you help me?
>
> You just helped yourself.
>
> -MIME headers (RFC 2047) often display unreadable gibberish for
>  charsets that the recipient can't decode, even when using
>  quoted-printable (commonly called "quoted-unreadble" by people in the
>  mail world).
>
> -HTML shows unintelligible gibberish if the charset used and stated in
>  the document cannot be displayed by the user. People see this every
>  day.
>
> -HTTP fails compeletly if the client lists charsets that it can read
>  and none of those are charsets that the server can write. This is
>  uncommon because this feature is rarely used because of the high
>  failure rate.
>
> The third case is analogous to what you are proposing with "fail to
> deliver if the charset is not supported".

So you are saying that the IETF is "not very keen on creating
protocols" like MIME, HTML and HTTP?  That's an interesting
proposition.

>>If you are speaking for IETF,
>
> I'm not, and you should assume that anyone other than Harald
> Alvestrand or Leslie Daigle who speaks for the IETF is bluffing or
> lying.

Right, I didn't want to make that assumption in this case as it would
be offensive.

>>  I find it interesting that RFC 2277
>>"IETF Policy on Character Sets and Languages" says that protocols MAY
>>allow use of any repertoire.
>
> Really? It says that? Could you quote the sentence for us here? I
> couldn't find the word "repertoire" anywhere in RFC 2277.

The word "repertoire" is indeed not used.  The following section
(§3.1, page 3 of the document, if you want to look it up) says that
other charsets or other encoding schemes may be used.

,----
|    Protocols MAY specify, in addition, how to use other charsets or
|    other character encoding schemes for ISO 10646, such as UTF-16, but
|    lack of an ability to use UTF-8 is a violation of this policy; such a
|    violation would need a variance procedure ([BCP9] section 9) with
|    clear and solid justification in the protocol specification document
|    before being entered into or advanced upon the standards track.
`----

I note that punycode is a encoding scheme, and thus IDNA and IMAA
violates this by lacking an ability to use UTF-8.

>>   It doesn't say that it is a bad idea to
>>allow more than one charset.  I agree with that document, let's
>>require the use of UTF-8 in protocols, but allow negotiation of other
>>charsets to smooth transition and deployment.
>
> You should take this up with Harald Alvestrand, the author of RFC
> 2277. Note that IDN chose not to use UTF-8, and Harald (as chair of
> the IESG) approved it to be on standards track.

Perhaps he is busy with other things, but I will ask if the policy in
RFC 2277 doesn't apply any more, or where the variance procedure steps
for the IDN working group are documented.  Thanks for the suggestion.