RE: Connected mode draft and retransmit implosion

On Mon, 23 Aug 2004, Michael Krause wrote:

> At 06:25 PM 8/13/2004, Fab Tillier wrote:
> > > From: bill [mailto:[email protected]]
> > > Sent: Friday, August 13, 2004 5:59 PM
> > >
> > > Yes, that is exactly the problem... Both layers retransmitting at the
> > > same time
> >
> >While both layers issue retransmits at the same time, the packets on the
> >wire maintain their order.  The first packet sent will be delivered first,
> >with the second queued up behind.  The data will not get interleaved.  There
> >is no way for the retransmitted packet to get delivered first.
> >
> > >
> > > And possibly not recovering for quite a while
> >
> >IB recovery is pretty much binary - if an RC QP exceeds its retry limit, it
> >will transition to the error state.  At that point, all processing of the
> >send queue stops and the consumer gets an error completion and an async
> >error notification for that QP.
> >
> >I still don't see the harm in TCP retransmitting while IB is also
> >retransmitting aside from inefficiencies.
> 
> Given this is a RC QP, the WR are processed in order and must complete in 
> order to make forward progress.  As such, the consumer which is posting WR 
> can post new or duplicate packets and the underlying QP will not see these 
> until it completes the current WR.  As such, IB retransmission and consumer 
> retransmission algorithms are orthogonal to one another in all respects 
> sans perhaps doing a poor job configuring the associated IB retransmission 
> timers as well as the arbitration policies within the fabric such that the 
> consumer falls into its retransmission algorithm too often.
> 
> Mike 

Yes, TCP will not see out of order packets since the underlying layer delivers
in order, reliable data. Also, TCP timers are likely to be much longer than 
the RC timers. Therefore, it is quite likely that RC will retransmit multiple 
times and recover by the time TCP timers expire.

As an aside, IPoIB over connected mode is not limited to RC, it includes UC 
too. Also, it is IP (which implies UDP, SCTP etc.) over RC or UC. 

Vivek