Re: small change in the connected mode draft

Michael Krause wrote:
> At 02:51 PM 3/23/2006, Dror Goldenberg wrote:
>>
>> The case that you mentioned, RNR driven retransmission, is another 
>> case of retransmission. It has nothing to do with congestion or 
>> packet drop in the fabric. It is just being impacted by the ability 
>> of the receiver to post receive buffers on the RQ/SRQ. Here the 
>> timeouts are application based and as you wrote can be configured to 
>> infinity. I agree that infinit number of retries would be a bad 
>> choice of RNR retry count. So, maybe we should recommend on selecting 
>> also the RNR timeout and retry count to be low too. I am also 
>> wondering what happens when there is a slow receiver, e.g. posting 
>> too slow on the RQ/SRQ, in which case the RNR Nak will happen very 
>> frequently and might cause the QP to get into error state because of 
>> RNR timeout exahusted.
>
> The question raised was what impact will retransmission have on a ULP 
> such as TCP which would treat timeouts as congestion events.  I raised 
> two cases which are not easy to control at the IB protocol level since 
> they are admin managed:
>         - RNR which can lead to very long delays in the connection 
> transmission completion
>         - Arbitration which can lead to very slow forward progress on 
> a given QoS channel
>
> Both of these should be comprehended minimally as informative text to 
> guide developers and management solutions to do the right thing.  They 
> also need to be aware that in the case of RNR, there is no single 
> right answer and its usage may entail long delays while the OS does 
> whatever it needs that triggered the RNR operation.
My naive understanding of how IB stacks operate would be to give STRONG 
advice to TCP/IP/IB-CM implementors that the IB interface should be fed 
enough buffers to fill all the open TCP connection windows. If that is 
done, the only time RNR could occur is when the TCP window is closed - 
and in that situation, the application is in trouble anyway.

Of course, this approach has its own set of cross-layer issues (and 
doesn't work for UDP-based applications). But they are all about buffer 
management within a single host, which makes layering violations 
slightly less egregrious.

                            Harald