Splitting and encoding of local-parts: some thoughts

Some thoughts on where to split, and how to encode.

(1) We could encode the whole dequoted localpart in one go.  But then
we can't use punycode as the ACE, because it could produce multiple
consecutive dots, and I really don't think we want to go there.
(That's not to say we shouldn't consider this option with a different
ACE, which could still be a bootstring profile.)

(2) We could simply split at dots, and encode using punycode as the
ACE.  Each token (between dots) would of course have to be marked in
some way (eg with an ACE prefix) if it's ACE-encodeed.  This means
that if dots are commonplace in internationalized localparts, the
extra ACE prefixes would eat into our length limit.

Splitting just at dots however doesn't allow much of the maniplulation
that is commonly performed on localparts, so the base document
proposes an alternative for consideration, namely

(3) split at all non-alphanumeric ASCII characters.  So we need to
pose the question, are non-alphanumeric ASCII characters (or
characters that normalize to them) likely to be commonplace in
internationalized localparts?  If so, this will eat into our lenght
limit further.

If we want to go down the road of accepting some added complexity in
the algorithms in order to maximimize the length of string we can
encode, then perhaps neither (2) nor (3) are ideal.  Then again, maybe
even (3) is good enough if such characters will in practice be
uncommon in internationalized local parts.  (I note that the IDN WG
essentially chose to trade simplicity for coding efficiency in their
selection of punycode against the previous WG favourites of RACE and
DUDE.)

However, if we opt for (1) then we pretty much rule out punycode
(though we could use a bootstring profile in which dot is not a basic
code point).  If we have to go for something different from the IDN
encoding, there's not real extra cost in going for something more
radically different.

While I'm there, though, I'd suggest that if we end up defining a new
bootstring profile here, we should consider going for underscore
rather than hyphen as the delimited, and use underscores rather than
hyphens in the ACE prefix.  The reason is that hyphen is often used as
a delimited in structured local parts, whereas underscore is a safe
character to use in a localpart that is almost never used in this way.

I'd also like to throw a slightly wacky idea out there: if we accept
that any prefixes and suffixes applied to (or stripped off from)
e-mail addresses are restricted to ASCII, then it is adequate to
identify the substring of the localpart from the first non-ASCII
character to the last non-ASCII character, and encode and mark that in
some way.  It then becomes safe to append and strip arbirary ASCII
suffixes from IMAs.  If we avoid common separators such as plus and
minus in the output from the ACE, and in whatever markup we use to tag
the string as ACE-encoded, it also becomes safe to split on common
separators.

My concern here is that maybe even this doesn't go far enough.  Is it
reasonable to restrict users of localpart suffixes (delimited by plus
or minus) to restrict their suffixes to ASCII characters?

Just some more thoughts, and apologies for the fact that this has
turned in to a bit of a stream of consciousness... :)

       -roy