RE: RE: [Ips] DDP message interleaving

Pat Thaler wrote:
> [This discussion seems more appropriate to the rddp reflector
> so I've copied that and suggest we move over there if more is needed]
> 
> Caitlin,
> 
> In the general case, the RDMAP protocol and the ULPs built on
> it are designed to rely on in order delivery of messages. A
> lower layer protocol that sends out of order but reorders
> before the packets leave the llp at the receiver (e.g. TCP
> retransmission of dropped packets) is okay. A lower layer
> level that allowed interleaving but didn't have a mechanism
> to undo it at the receiver, would have to be protocol aware
> to an extent that would be a layering violation.
> 
> The particular example show doesn't cause a problem at the
> receiver because the last segment of each message occurs in
> the right order so the message completions in DDP and RDMAP
> will occur in the correct order. However, one could have
> something else like:
> 
> Message 1 - tagged (e.g. the last RDMA data transfer of an
> SCSI Read over iSCSI/iSER) FPDU frame TO = 0, L = 0, T = 1
> FPDU frame TO = 1000, L = 0, T = 1
> 
> Message 2 - untagged (e.g. the status message indicating that
> the SCSI Read has completed) FPDU frame MSN = 0, M0 = 0, L = 1
> 
> If those frames were interleaved and sent:
> 
> FPDU frame TO = 0, L = 0, T = 1
> FPDU frame MSN = 0, M0 = 0, L = 1
> FPDU frame TO = 1000, L = 0, T = 1
> 
> the untagged message indicating that the SCSI Read has
> completed would be sent to RDMAP before the transfer of read
> data had finished. Data corruption could occur. To avoid such
> problems, the last segment of each message must be sent in order.
> 
> Even if last segments are kept in order, there can be
> undesireable consequences at the receiver of interleaving. It
> shouldn't be done. 
> 
Agreed, it SHOULD NOT be done. But the justification for having
predictable segment ordering, beyound the ordering requirements
associated with the L-segments, is optimization not interoperability.
That is why there is no MUST language associated with this. Note
that the ascending order of for Message Offset in a message
is similarly a SHOULD, not a MUST.

My advice to implementers is 1) transmit in order, 2) optimize
your caching and related resource usage assuming that you will
receive in order, but 3) be ready to accept in any order and
4) do not waste receiver code/firmware trying to detect
non-standard segmentation strategies.