Re: interpretation of whitespace inside obs-phrase

Timo,

First off, thank you so much for taking the time to reply to my question; when I ran across this question I was not certain I would be able to find anyone with much domain experience who would take the time to look at this question. Again: thanks!

----- "Timo Sirainen" <[email protected]> wrote:
> On 25.8.2012, at 8.25, Jay Freeman (saurik) wrote:

> >   display-name    =   phrase
> >   phrase          =   1*word / obs-phrase
> >   obs-phrase      =   word *(word / "." / CFWS)
> >   word            =   atom / quoted-string
> >   atom            =   [CFWS] 1*atext [CFWS]
> >   quoted-string   =   [CFWS]
> >                       DQUOTE *([FWS] qcontent) [FWS] DQUOTE
> >                       [CFWS]
> 
> There is no dot-atom above.

Correct. I purposefully only provided a reference to the rules relevant for the expansion of these display names, and a dot-atom is not a construct you can produce from a display name; those are only relevant when producing addr-spec and msg-id.

> >   Both atom and dot-atom are interpreted as a single unit,
...
> So the "dot-atom" mentioned here doesn't apply.

Also correct. However, that is the wording of the paragraph from the specification, which I copy and pasted verbatim: the same section that discusses the semantics of atoms uses the same rules for dot-atoms, which in this situation should be ignored.

> > In these cases, I would then presume, I would end up with the
> display names |JayFreeman| and |JayR.Freeman|.
> 
> But they don't say that whitespace doesn't matter for the entire
> phrase, only its individual parts. I'm not sure if anything
> specifically requires you to show the whitespace in any specific way
> though, so my parser attempts to use a single space character where
> there originally was whitespace.

This text also, however, doesn't say that the quotation marks "[don't] matter for the entire phrase, only [the] individual [quoted strings]". In fact, it is the same sentence with the same clause: should they not have the same semantics?

   Semantically, neither the optional CFWS outside of the quote
   characters nor the quote characters themselves are part of the
   quoted-string; the quoted-string is what is contained between the two
   quote characters.

If we are then deciding that the content outside of the tokens should be considered part of the phrase, I will argue that we cannot differentiate between removing quotation marks and removing space characters (which was the behavior of Java's JavaMail).

However, in the normal/common case of |"Jay Freeman (saurik)" <[email protected]>|, it is both common practice and quite clear that we are supposed to remove the surrounding quotation marks. :( I thereby am hoping to get a "we intended you to do X".

> So the end result is that you get: 
> 
> 1:|Jay Freeman|
> 2:|Jay R. Freeman|
> 3:|Jay R Freeman|
> 
> And also |Jay.R.Freeman| also produces |Jay.R.Freeman| (I think many
> parsers will add whitespace after dots there).

I went back and re-tested with 4:|Jay"."Free.man| (which now seemed interesting) to see what would happen, including against the parsers in Dovecot, Cyrus, and Thunderbird. (I also verified that PHP's imap really does just call out to c-client.)

4:|Jay . Free.man|:
C Dovecot
Python email.utils
PHP mailparse
Ruby TMail

4:|Jay.Free.man|:
C c-client (PHP IMAP)
C Cyrus
C++ Thunderbird
Java MIME4J

4:|Jay"."Free.man|:
Java JavaMail
Java MIME4J (Lenient)

4:|Jay "." Free man|:
C++ mimelib

I must say that I'm still not certain what the correct way to parse these primitives is; as far as I currently understand, in other contexts (inside of e-mail addresses, for example) the whitespace around atoms needs to be totally ignored; why not here?

(While I was at it, I also did 5:|"Jay""Freeman"|; I went ahead and built a framework to easily test all of these different parsers at once. I could also see it interesting to do clients like Outlook and Gmail, but that would require more automation work. ;P)

5:|Jay Freeman|:
C Dovecot
Python email.utils
PHP mailparse
Ruby TMail

5:|JayFreeman|:
C c-client (PHP IMAP)
C Cyrus
C++ Thunderbird
Java MIME4J
Java MIME4J (Lenient)

5:|Jay""Freeman|:
Java JavaMail

5:|"Jay" "Freeman"|:
C++ mimelib

Sincerely,
Jay Freeman (saurik)
[email protected]
_______________________________________________
ietf-822 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ietf-822