RFC 2449: clarifications needed

Hi,

As this list is once again active, I've decided to post my thoughts
and a few questions on the changes to POP3 defined by RFC 2449.

1. "This specification increases the length restrictions on commands
and parameters imposed by RFC 1939.

   The maximum length of a command is increased from 47 characters (4
   character command, single space, 40 character argument, CRLF) to 255
   octets, including the terminating CRLF."

This looks like a misinterpretation of RFC 1939 to me.  Here's what
it used to say:

"  Commands in the POP3 consist of a case-insensitive keyword, possibly
   followed by one or more arguments.  All commands are terminated by a
   CRLF pair.  Keywords and arguments consist of printable ASCII
   characters.  Keywords and arguments are each separated by a single
   SPACE character.  Keywords are three or four characters long. Each
   argument may be up to 40 characters long."

Even if POP3 commands defined by RFC 1939 accepted at most one
argument (which isn't the case: TOP has two arguments), this doesn't
imply that POP3 servers were allowed not to parse longer commands
correctly.  In particular, I consider a server that processes one
long command as two or more short ones not fully RFC 1939 compliant.
I was specifically looking for a command length limit in RFC 1939
when developing my POP3 server, and came to the conclusion that
there's none, even though this may require extra code.

Basically, while I like the fact that a length limit is now defined,
I believe that our new RFC is wrong in saying that it "increases" the
length restrictions.  In reality, RFC 2449 compliant servers are not
necessarily fully compliant to RFC 1939: they are now allowed to not
process commands longer than 255 octets correctly (in the 1939 sense).

2. RFC 1939 didn't specify that POP3 clients are required to wait for
a response before issuing another command.  It only said that the
client and server exchange commands and responses.  The "exchange"
doesn't sound like a permission to not support pipelined commands to
me, even though there're many (broken) servers that don't and this
can be tricky to implement if AUTHORIZATION and TRANSACTION states
are implemented with different processes or even physical servers.

The addition of the PIPELINING capability effectively makes these old
POP3 servers (which didn't support pipelining) RFC compliant.

I admit that this one issue isn't very clear.  Some might say that
"exchange" implies waiting for a response.  I would prefer a MUST,
though.

3. This requirement with PIPELINING seems to be too strict: "If
either the client or server uses blocking writes, it MUST not exceed
the window size of the underlying transport layer."

Here's my understanding of the related issues:

3.1. Both the server and the client process commands and responses in
order, so I don't see any harm in a write blocking on just one end.
Implementations may want not to block for some reason, but this is an
implementation issue and has nothing to do with the protocol.

3.2. Simply not exceeding the window size doesn't guarantee that a
blocking write won't block.  There're often more requirements, and
these depend on both the underlying protocols (currently, TCP with
some security layer on top of it), their implementations, and the OS.

3.3. It is often not possible for an application to know if a write
would exceed the window size.  POP3 clients can assume that a write
would likely not block if it doesn't exceed a certain size after all
responses to a previous group of commands have been received.  I
don't know of a way to use such an approach for POP3 servers, as
command responses don't get acknowledged on the application layer.

RFC 2197 ("SMTP Service Extension for Command Pipelining") explains
the real reason behind a similar requirement: avoiding a deadlock
condition.  If the client blocks on a command write, it won't be able
to read responses off the socket, so, once the kernel buffer fills
up, it may not acknowledge new responses coming from the server.  The
write on the server may then block as well.

To solve this problem, it seems to be enough to ensure that writes
don't block on the client end only.  This is what RFC 2197 requires.

Besides, is there any useful work for a POP3 server to be done if it
would otherwise block on a write of a response?  Reading all commands
off the socket immediately (to be reasonably sure the client won't
have a need to block) would potentially require an unlimited storage.
This is not good anyway.

Finally, it is not blocking writes that cause the problem -- it is
blocking _anywhere_ and not reading responses off the socket in time
that does.  The fact that a client doesn't use blocking writes or
makes sure they don't block doesn't yet imply that it doesn't have
the deadlock problem.

I suggest that the requirement be relaxed to apply to POP3 clients
only, and the reasoning behind it explained in the RFC.

I'll appreciate any comments on the thoughts above.  I am going to
add CAPA support into my POP3 server, but would like to make these
issues clear first.  Current version of the server can be found at:

	http://www.openwall.com/popa3d/

Signed,
Solar Designer