legality of interleaving untagged message segments

The current RDDP draft requires that messages be submitted
to the LLP in order.

It also requires that when a message is segmented, that the
final segment MUST be the last one submitted to the LLP.

It does not, however, define what is meant by submitting
"a message" when in fact the DDP layer submits *segments*
to the LLP.

I believe this could allow a very strained interpretation
that would allow interleaving of untagged segments to the
LLP, as long as the *final* segments were submitted in order.
For example, it might allow the following submission order:

     Untagged MSN 7 segment 0 (not final)
     Untagged MSN 8 segment 0 (not final)
     Untagged MSN 7 segment 1 (final)
     Untagged MSN 8 segment 1 (final)

A typical out-of-order receiver would not even notice
anything unusual about this sequence of untagged segments.
However, it has a major impact on in-order receivers. It
requires that multiple RQ (or S-RQ) entries be 'current'
for the packet processor. If the in-order packet-processor
is on the RNIC and the RQ/S-RQ is in host memory this is
a totally unnecessary complication.

I still believe that the sender should be free to re-order
segments within a message (but SHOULD NOT unless it gains
some advantage in doing so, such as sending data that is
already cached), but I cannot come up with any benefit
for the sender to interleave message segments. The complexity
of doing so is simply too great for any potential additional
caching gains.

The fact that such sequences are so unlikely means that it
is highly likely that there will be implementations that
will not properly handle this case, whether or not the
specification explicitly forbids it. This makes any theoretical
advantages even more remote. Customers seldom care that
your esoteric optimization is "legal", they just know that
*your* RNIC is the one that isn't working with everyone else.

So I believe the draft should be clarified to require that
all segments for untagged message N MUST be submitted to
the LLP before any segments for untagged message N+1 are
submitted to the LLP.

Doing so will have no impact on unordered receivers. I doubt
it will have any impact on any existing senders, and it
eliminates one corner condition that would require extra
resources and testing for in-order receivers.


>>
>> Several months ago I raised a query as to whether DDP could/should
>> mandate that the process of segmenting DDP Messages MUST NOT
>> result in interleaving of messages. That is *all* segments for DDP
>> Message N must be submitted to the transmitting LLP: before any
>> segment of DDP Message N+1 is submitted.
>>
>> I believe the current draft could, in a very strained malicious way,
>> be interpreted to only require:
>>
>>     that the segment with the L bit for each message be the last
>>     segment for that message.
>>
>>     that the segments with L bits are themselves submitted in order.
>>
>> The only real response on the list, as I recall, was from Mike Krause
>> who pointed out that such interleaving would not even be noticed
>> by an out-of-order receiver. And that is completely correct.
>>
>> It does have an impact on in-order receivers, and particular on
>> how many untagged buffers an in-order RNIC has to have looked
>> up from the host-side receive queues (or worse SRQs).
>>
>> I doubt very much whether anyone will ever produce a sender
>> with a segmentation algorithm that will violate this rule, and I
>> am one of the strong advocates of allowing senders to use
>> creative ordering *within* a message to optimize for sending
>> data that is already in the RNICs cache.
>>
>> But it would be handy to have something firm to justify coding
>> an RNIC based on this assumption.
>>
>> Do you feel that the current language already forbids interleaving
>> of message segments (in terms of the order they are submitted
>> to the LLP, and hence their LLP sequence numbers)?
>>
>> If so, it is probably enough to just have that stated on the
>> reflector.
>>
>> Otherwise, as one of the principal advocates of having the protocols
>> be friendly to *both* ordered and unordered implementations, would
>> you support such a change?
>>
>> In my evaluation, the drafters are firmly in the "out of order" camp
>> and hence do not see any need for this clarification.
>>
>


--
Caitlin Bestler
http://asomi.com/