Re: draft-resnick-2822upd-02 and Netnews

In <p06250101c2d30b5454a1@[172.28.170.243]> Pete Resnick <[email protected]> writes:

>I want to start out by saying that in answering this, I'm trying to 
>take RFC 2119, section 6 *very* seriously:

>6. Guidance in the use of these Imperatives

>    Imperatives of the type defined in this memo must be used with care
>    and sparingly.  In particular, they MUST only be used where it is
>    actually required for interoperation or to limit behavior which has
>    potential for causing harm (e.g., limiting retransmisssions)  For
>    example, they must not be used to try to impose a particular method
>    on implementors where the method is not required for
>    interoperability.

>Insofar as I interpret this:

>MUST means, "You must do this or seriously bad things will happen, 
>either with regard to interoperation with other implementers of this 
>specification or known harm will come to this or other parts of the 
>Internet."...

Yes, and I shall try to justify them on that basis.


>>However, I cannot conceive that any future IP-address format or 
>>other domain literal would ever contain a <quoted-pair> or any 
>>whitespace (that, together with '>', is the other thing Netnews 
>>cannot tolerate at this point) - there are just too many existing 
>>protocols that would promptly
>>fall over. So why not fix the problem once and for all now? And fix 
>>it for <domain-literal> at the same time - that and 
>><no-fold-literal> and the only places where <dtext> and <dcontent> 
>>are used, so it is easily done, and the obs-syntax would remain, of 
>>course.

>My main issue here is that we already have examples where limiting 
>the syntax in "obvious" ways in 2822 screwed up 2821 (e.g., 
>"Received:" syntax) and I'd prefer not to shoot myself in the foot 
>again. Do we know of implementations (especially non-netnews 
>implementations) which can't deal with quoted-pair in domain literals 
>or message ids, or won't strip whitespace out of domain literals? 
>Should this really come out?

It would probably be helpful for a start to state clearly that what is
actually written in a domain literal (whether in an address or a mesg-id)
MUST/SHOULD agree with the specification of some widely recognized
numbering/routeing scheme (which, at the moment means IP4 or IP6 of
course).

However, suppose some future addressing scheme _does_ admit quoted-pairs,
then accoring to the rest of RFC 2822 [xyz\qabc] would be semantically
equivalent to [xyzqabc]. In email that might cause threading algorithms to
break (e.g. if the Reference header contained a different form to the
original msg-id). But for Netnews it would be an utter disaster. So there
is an interoperability problem (mild with email, severe with news), but
the problem is simply avoided by forbidding quoted-pair in the first place
(or, alternatively, allowing '\' as a permitted character in dcontent, but
without the quoted-pair semantics). Removing SP and '>' from dcontent
would also simplify implemtations and avoid problems with naive
implementations.

>>2. SP after the ':' in headers
>>------------------------------
>>
>>That is REQUIRED in draft-ietf-usefor-usefor-12, and it is REQUIRED 
>>in the new NNTP standard RFC 3977. Since every known MUA already 
>>generates headers that way

>Please some support for this, including a review of the DRUMS 
>archive. If we're going to make this change, you do the research. I 
>seem to remember a discussion of this during DRUMS, and having the 
>space optional was where we ended up. Can we check?

There undoubtedly exists software within Netnews that will break if this
SP is absent, hence the reason we felt it necessary to REQUIRE that SP in
both USEFOR and NNTP.

So if you want email to interoperate with Netnews (and gateways from
mailing lists to News are quite common), then you indeed have an
interoperability problem sufficient to justify a "MUST". As for DRUMS, I
doubt this particular consideration was ever taken into account.

OTOH, since it is the invariable practice within email MUAs to include
this SP, putting such a MUST in email would incur no problems with the
installed base (of course, it would still be acceptable in the
obs-syntax). Whilst we may not be able to remove all discrepancies between
email and news, it would be desirable not to have any differences where
they can so easily be avoided.

>>There is one slight consequence that you should then forbid header 
>>lines with only whitespace after the header-name (because trailing 
>>whitespace has a habit of getting lost).

>And this gets us into a "changes with potential side-effects" problem 
>that worries me.

Yes, that is a bit messy, and so I would not push the point too hard.
Problems only arise with rogue agents that insist on discarding trailing
whitespace (which I regard as broken anyway).

>>3. CFWS between msg-ids in References
>>-------------------------------------
>>
>>It is currently [CFWS], but usage in Netnews has always placed 
>>whitespace there (and References has always been more of a News 
>>featrure than an Email feature). Would it hurt to REQUIRE it?

>See top of this message. Would it hurt to *not* have it as a MUST?

There undoubtedly exists Netnews software which will break without that
whitespace, hence the reason USEFOR RECOMMENDS it. So there is indeed an
interoperability problem, but to follow USEFOR a "SHOULD" would be
enough.  But the problem is nowhere as severe as the SP after ':' one.

Question. Does current typical email software routinely include that
whitespace? I had a quick look at a mailing list I subscribe to, and I did
not find any example where that whitespace was absent.

>>And also, please can we promote its use in Replies from SHOULD to 
>>MUST? It was SHOULD in RFC2822, but many MUAs have not taken the 
>>"hint" and, as a result, threading in mailing lists often gets 
>>broken, which is a confounded nuisance. It is invariably done 
>>properly within News.

There is also an interopability problem here, but is is not so clearcut as
the other cases. If this header is omitted, then threading algorithms just
do not work. Now you might argue that threading (an operation performed by
MUAs) is not a REQUIRED feature of the email protocol - just a smart
feature that some MUAs provide. However, even the smart MUAs can't do it
if the header is absent, and you will find it hard to convince the average
user that threading is not a 'normal' part of his mailing-list experience.

We agonized over this in USEFOR (and even had some wording at one stage
declaring threading to be an 'optional' feature of the protocol).
Eventually, what we wrote was:

   A "followup" is an article containing a response to the contents of
   an earlier article, its "precursor".  Every followup includes a
   "References" header field identifying that precursor (but note that
   non-followup articles may also use a References header field).

That wording is normative, so if the References is absent it ain't a
'followup'. So any user agent with a "followup" button which failed to
include a References could hardly claim compliance with the standard.

>>4. Allowed positions for <comment>s
>>-----------------------------------
>>
>>Allowing <comment>s in Message-ID header fields will break Netnews, 
>>and therefore the Usefor draft disallows them. Would it hurt to 
>>bring Email into line? For sure nobody ever uses them there.

>Again, potentially breaking some netnews implementations is not by 
>itself a reason to remove this IMO. Are there known non-netnews 
>implementations that break because of this? (And why are netnews 
>implementations so non-robust in the first place, unable to parse 
>comments and quoted-pairs?)

Yes there are undoubtedly news implementation that would break in that
situation, and the lack of that robustness is simply because of the severe
performance penalty in what is the critical inner-loop of relaying agents.

But I don't think the case for the same restriction in email is as strong
as some of the others I have raised, even though interopability problems
would arise.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email protected]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5