RE: [Ips] DDP message interleaving

[This discussion seems more appropriate to the rddp reflector so I've
copied that and suggest we move over there if more is needed]

Caitlin,

In the general case, the RDMAP protocol and the ULPs built on it are
designed to rely on in order delivery of messages. A lower layer
protocol that sends out of order but reorders before the packets leave
the llp at the receiver (e.g. TCP retransmission of dropped packets) is
okay. A lower layer level that allowed interleaving but didn't have a
mechanism to undo it at the receiver, would have to be protocol aware to
an extent that would be a layering violation. 

The particular example show doesn't cause a problem at the receiver
because the last segment of each message occurs in the right order so
the message completions in DDP and RDMAP will occur in the correct
order. However, one could have something else like:

Message 1 - tagged (e.g. the last RDMA data transfer of an SCSI Read
over iSCSI/iSER) FPDU frame TO = 0, L = 0, T = 1 FPDU frame TO = 1000, L
= 0, T = 1

Message 2 - untagged (e.g. the status message indicating that the SCSI
Read has completed) FPDU frame MSN = 0, M0 = 0, L = 1

If those frames were interleaved and sent:

FPDU frame TO = 0, L = 0, T = 1
FPDU frame MSN = 0, M0 = 0, L = 1
FPDU frame TO = 1000, L = 0, T = 1

the untagged message indicating that the SCSI Read has completed would
be sent to RDMAP before the transfer of read data had finished. Data
corruption could occur. To avoid such problems, the last segment of each
message must be sent in order. 

Even if last segments are kept in order, there can be undesireable
consequences at the receiver of interleaving. It shouldn't be done.

Unfortunately, we didn't put a very clear statement of this in rddp's
list of reliable delivery requirements. The closest we came was:
4.  The LLP MUST preserve DDP Segment and Message boundaries at the 
       Data Sink. 

   5.  The LLP MAY provide the incoming segments out of order for 
       Placement, but if it does, it MUST also provide information that 
       specifies what the sender specified order was. 

With 5. we were thinking more of out of order arrival of segments due to
reordering in the path or packet drops - not at the transmitter.

Regards,
Pat 

-----Original Message-----
From: Caitlin Bestler [mailto:[email protected]] 
Sent: Thursday, April 20, 2006 8:25 AM
To: Ben Sum; [email protected]
Subject: Re: [Ips] DDP message interleaving


> -----Original Message-----
> From: Ben Sum [mailto:[email protected]]
> Sent: Wednesday, April 19, 2006 06:09 PM
> To: [email protected]
> Subject: [Ips] DDP message interleaving
> 
> Is sender ever multiplexing messages?  Considering following message:
> 
> Message1:     
> 
> FPDU frame MSN = 0, MO = 0, L = 0, T= 0
> 
> FPDU frame MSN = 0, MO = 1000, L = 1, T= 0 
> Message2:      
> 
> FPDU frame T = 1, TO = 0, L = 0
> FPDU frame T = 1, TO = 1000, L = 1
>  
> Can the sender transmit as following: 
> 1)       FPDU frame MSN = 0, MO = 0, L = 0, T = 0 
> 2)       FPDU frame T = 1, TO = 0, L = 0 
> 3)       FPDU frame MSN = 0, MO = 1000, L = 1, T = 0 
> 4)       FPDU frame T = 1, TO = 1000, L = 1
> 
>  
> 
> Regards,
> 
> Ben
> 

There are two issues here: what you should accept, and what you should
send.

On the receive side, if you are processing the incoming packets
optimally you will not even notice this type of anomolous ordering. A
receiver definitely SHOULD NOT add extra steps to attempt to detect
something of this nature.

On the other hand this is definitely something that a sender SHOULD NOT
send, possibly MUST NOT send.

The DDP spec clarifies that the DDP layer segments messages and submits
them to the LLP. It is clear that the DDP layer MUST submit all of the
segments for a message in sequence and non-interleaved.

What is not clear is if the LLP layer MUST assign these segments
successive LLP sequences, or if it can interleave in ways that do change
the sequence of L flags or the fact that the L flag is the last segment
of the message.

Nobody has identified a reason why a MPA layer would do such a thing,
and assigning the expected TCP sequencing is at the minimum an implied
SHOULD.
But this is not explicitly a constraint. The SCTP Adaptation, which is
under the exact same constraints as the MPA/TCP adaptation explicitly
recognizes that SCTP stacks are allowed to re-order Data Chunks in order
to optimize transmission. IF DDP does not impose a MUST behavior on
SCTP, then it does not impose it on MPA/TCP. But SCTP has a valid
"SHOULD trumping" 
reason for reordering packets, MPA/TCP does not.




_______________________________________________
Ips mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/ips