Re: comments on draft-kashyap-ipoib-connected-mode-02.txt

Hi All,

I also have some comments on draft-kashyap-ipoib-connected-mode-02.txt.

I haven't thoroughly reviewed the document, so I may have some 
further comments later, but in my initial read, one major concern 
jumped out at me.

If I understand correctly, the reliable connected mode of IPOIB is 
TCP/IPv6.  Is that correct?

If so, I think that you need to carefully consider the issues 
associated with tunneling TCP inside of TCP.  In particular, I am 
concerned about this section:

        The default MTU of the IPoIB-CM interface is 2044 octets i.e.
         2048 octet IPoIB-link MTU minus the 4 octet encapsulation
         header.

         The connected modes of InfiniBand allow message sizes up to 2^31
         octets.  Therefore, IPoIB-CM can use a much larger MTU for
         unicast communication between any two endpoints. At the same
         time the maximum and/or optimal payload that can be received or
         sent over an InfiniBand connection is dependent on the
         implementation, HCA and the resources configured.

Having a larger MTU at the higher-level TCP layer than at the 
lower-level TCP layer may lead to situations where the upper and 
lower layers' retransmit timers will both fire at the same time, 
causing real problems in the case of congestion.

This may happen because bothTCP layers will use the same algorithm 
(and maybe the same code) for calculating the retransmission time. 
When a packet is lost, the lower layer's TCP retransmission timer 
will fire, causing a retransmission of one lower-layer MTU of data 
(2048 bytes), AND the upper layer's TCP retransmission timer will 
also fire, causing a retransmission of one upper-layer MTU of data 
(up to 2^31 bytes).  The lower layer will not recognize that the 
upper layer's retransmission is duplicate data, so this will result 
in the transmission of many 2K packets when a single 2K packet is 
lost.

We may want to talk to the Transport area to determine if there are 
other issues associated with tunneling TCP inside of TCP/IP.

Thoughts?

Margaret