RE: ipoib-cm multiple connection model
Dror Goldenberg <[email protected]> Wed, 26 Oct 2005 09:20:55 +0200
| Newsgroups | gmane.ietf.ipoib |
|---|---|
| Message-ID | <[email protected]> |
This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --===============1745897787== Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C5D9FD.CE2A2930" This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C5D9FD.CE2A2930 Content-Type: text/plain As Jerry and Vandana mentioned I was referring to the IPoIB-CM connection establishment and not to how IP datagrams are striped amongst them. As of the point you're raising, maybe it worth noting it in the RFC draft. E.g. when multiple connections are being used between two IPoIB-CM peers, for optimal performance it is desirable to send IP datagrams of the same TCP/UDP session on the same IPoIB-CM connection. A common implementation method is to choose the connection according to a hash value calculated on the TCP' 5-tuple. -Dror > -----Original Message----- > From: Bill Strahm [mailto:[email protected]] > Sent: Saturday, October 22, 2005 5:33 AM > To: Dror Goldenberg > Cc: [email protected] > Subject: Re: [Ipoverib] ipoib-cm multiple connection model > > > Let me try to understand what you are worried about: > I am assuming that each TCP connection (5-tuple) can be on > its own CM connection, and not multiplex a single TCP > connection over multiple CM connections. > > Is this your understanding ? > > Now if this is what we are talking about - we don't have a > problem. TCP only guarantees in order delivery to a single > connection - if I have multiple TCP connections to the same > hosts, IP packets can be completely reordered/jumbled by the > intermediate network - and all I am guaranteed is that each > TCP stream will have in order delivery relative to itself. > > I am not aware of any applications where in order IP packet > delivery is required between multiple streams. If this is > the case - it has to be handled above TCP anyway, as TCP does > not make this guarantee. > > Comments ? > Bill > > Dror Goldenberg wrote: > > > I think that the current model for multiple connections > establishment > > between > > two peers works well for 1 connection but is broken for > more than one > > connection. > > Because of cases where CM packets are dropped or travel on > different > > VLs, they > > might get reordered on the way and observed in a different > order than > > were originally > > sent. This breaks the current model which assumes that the REQs are > > sent and > > processed in order. > > > > I think that we need to think of a model which is robust to > CM packets > > reordering > > which can be one of: > > - Adding a "channel ID" in the private data. The field goes > 0,1,2 and > > on. It helps > > synchronizing both peers on which is the channel at > question now. > > The rest > > of the decision (per channel) of whether to accept or > reject can be > > the same > > as defined in the draft. > > - Not supporting multiple connections between the same peers. > > > > -Dror > > > >------------------------------------------------------------- > ---------- > >- > > > >_______________________________________________ > >IPoverIB mailing list > >[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib > > > > > ------_=_NextPart_001_01C5D9FD.CE2A2930 Content-Type: text/html Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> <HTML> <HEAD> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = charset=3Dus-ascii"> <META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version = 5.5.2654.45"> <TITLE>RE: [Ipoverib] ipoib-cm multiple connection model</TITLE> </HEAD> <BODY> <P><FONT SIZE=3D2>As Jerry and Vandana mentioned I was referring = to the IPoIB-CM connection establishment and not to how IP datagrams = are striped amongst them.</FONT></P> <P><FONT SIZE=3D2>As of the point you're raising, maybe it worth noting = it in the RFC draft. E.g. when multiple connections are being used = between two IPoIB-CM peers, for optimal performance it is desirable to = send IP datagrams of the same TCP/UDP session on the same IPoIB-CM = connection. A common implementation method is to choose the connection = according to a hash value calculated on the TCP' 5-tuple.</FONT></P> <P><FONT SIZE=3D2>-Dror</FONT> </P> <P><FONT SIZE=3D2>> -----Original Message-----</FONT> <BR><FONT SIZE=3D2>> From: Bill Strahm [<A = HREF=3D"mailto:[email protected]">mailto:[email protected]</A>] </FONT> <BR><FONT SIZE=3D2>> Sent: Saturday, October 22, 2005 5:33 AM</FONT> <BR><FONT SIZE=3D2>> To: Dror Goldenberg</FONT> <BR><FONT SIZE=3D2>> Cc: [email protected]</FONT> <BR><FONT SIZE=3D2>> Subject: Re: [Ipoverib] ipoib-cm multiple = connection model</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> Let me try to understand what you are worried = about:</FONT> <BR><FONT SIZE=3D2>> I am assuming that each TCP connection = (5-tuple) can be on </FONT> <BR><FONT SIZE=3D2>> its own CM connection, and not multiplex a = single TCP </FONT> <BR><FONT SIZE=3D2>> connection over multiple CM connections.</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> Is this your understanding ?</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> Now if this is what we are talking about - we = don't have a </FONT> <BR><FONT SIZE=3D2>> problem. TCP only guarantees in order = delivery to a single </FONT> <BR><FONT SIZE=3D2>> connection - if I have multiple TCP connections = to the same </FONT> <BR><FONT SIZE=3D2>> hosts, IP packets can be completely = reordered/jumbled by the </FONT> <BR><FONT SIZE=3D2>> intermediate network - and all I am guaranteed = is that each </FONT> <BR><FONT SIZE=3D2>> TCP stream will have in order delivery relative = to itself.</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> I am not aware of any applications where in = order IP packet </FONT> <BR><FONT SIZE=3D2>> delivery is required between multiple = streams. If this is </FONT> <BR><FONT SIZE=3D2>> the case - it has to be handled above TCP = anyway, as TCP does </FONT> <BR><FONT SIZE=3D2>> not make this guarantee.</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> Comments ?</FONT> <BR><FONT SIZE=3D2>> Bill</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> Dror Goldenberg wrote:</FONT> <BR><FONT SIZE=3D2>> </FONT> <BR><FONT SIZE=3D2>> > I think that the current model for = multiple connections </FONT> <BR><FONT SIZE=3D2>> establishment</FONT> <BR><FONT SIZE=3D2>> > between</FONT> <BR><FONT SIZE=3D2>> > two peers works well for 1 connection but = is broken for </FONT> <BR><FONT SIZE=3D2>> more than one </FONT> <BR><FONT SIZE=3D2>> > connection.</FONT> <BR><FONT SIZE=3D2>> > Because of cases where CM packets are = dropped or travel on </FONT> <BR><FONT SIZE=3D2>> different </FONT> <BR><FONT SIZE=3D2>> > VLs, they</FONT> <BR><FONT SIZE=3D2>> > might get reordered on the way and = observed in a different </FONT> <BR><FONT SIZE=3D2>> order than </FONT> <BR><FONT SIZE=3D2>> > were originally</FONT> <BR><FONT SIZE=3D2>> > sent. This breaks the current model which = assumes that the REQs are </FONT> <BR><FONT SIZE=3D2>> > sent and</FONT> <BR><FONT SIZE=3D2>> > processed in order.</FONT> <BR><FONT SIZE=3D2>> > </FONT> <BR><FONT SIZE=3D2>> > I think that we need to think of a model = which is robust to </FONT> <BR><FONT SIZE=3D2>> CM packets</FONT> <BR><FONT SIZE=3D2>> > reordering</FONT> <BR><FONT SIZE=3D2>> > which can be one of:</FONT> <BR><FONT SIZE=3D2>> > - Adding a "channel ID" in the = private data. The field goes </FONT> <BR><FONT SIZE=3D2>> 0,1,2 and </FONT> <BR><FONT SIZE=3D2>> > on. It helps</FONT> <BR><FONT SIZE=3D2>> > synchronizing both peers = on which is the channel at </FONT> <BR><FONT SIZE=3D2>> question now. </FONT> <BR><FONT SIZE=3D2>> > The rest</FONT> <BR><FONT SIZE=3D2>> > of the decision (per = channel) of whether to accept or </FONT> <BR><FONT SIZE=3D2>> reject can be </FONT> <BR><FONT SIZE=3D2>> > the same</FONT> <BR><FONT SIZE=3D2>> > as defined in the = draft.</FONT> <BR><FONT SIZE=3D2>> > - Not supporting multiple connections = between the same peers.</FONT> <BR><FONT SIZE=3D2>> > </FONT> <BR><FONT SIZE=3D2>> > -Dror</FONT> <BR><FONT SIZE=3D2>> ></FONT> <BR><FONT SIZE=3D2>> = >-------------------------------------------------------------</FONT>= <BR><FONT SIZE=3D2>> ----------</FONT> <BR><FONT SIZE=3D2>> >-</FONT> <BR><FONT SIZE=3D2>> ></FONT> <BR><FONT SIZE=3D2>> = >_______________________________________________</FONT> <BR><FONT SIZE=3D2>> >IPoverIB mailing list</FONT> <BR><FONT SIZE=3D2>> >[email protected] <A = HREF=3D"https://www1.ietf.org/mailman/listinfo/ipoverib" = TARGET=3D"_blank">https://www1.ietf.org/mailman/listinfo/ipoverib</A></F= ONT> <BR><FONT SIZE=3D2>> > </FONT> <BR><FONT SIZE=3D2>> ></FONT> <BR><FONT SIZE=3D2>> </FONT> </P> </BODY> </HTML> ------_=_NextPart_001_01C5D9FD.CE2A2930-- --===============1745897787== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPoverIB mailing list [email protected] https://www1.ietf.org/mailman/listinfo/ipoverib --===============1745897787==--