Re: A Couple of IPoIB Questions

At 11:38 PM 11/16/2004, Vivek Kashyap wrote:



>Hi,  I have a couple of questions relative to IPoIB:  1. 
>draft-ietf-ipoib-ip-over-infiniband-07.txt states: "Every IPoIB interface 
>MUST "FullMember" join the IB multicast group defined by the 
>broadcast-GID."  Isn't the broadcast group for IPv4 ? When the IPoIB 
>interface is IPv6 only, does this group still need be joined ?  If not, 
>where do the parameters for any IPv6 groups come from ? I am presuming 
>that this group needs to be joined in  the IPv6 only case. I just want to 
>be sure.
><VK> Yes, the broadcast-GID is at the InfiniBand layer and MUST be joined 
>whether you are running at v4 or v6 layer. <VK>  2. ALso, what is the 
>latest status of the Vivek's connected mode draft ? Will it be moving 
>forward ?  <VK> I'll be submitting it as 
>draft-ietf-ipoib-connected-mode-00.txt by the end of the month. There were 
>some interesting suggestions that were made during the IETF WG meeting. 
>Two of the suggestions of consequence are given below. The others we can 
>discuss when the minutes are published (they include some additional 
>requests on clarification on the transmission draft too).  a. The current 
>draft makes the various modes mutually exclusive i.e. RC, UC and UD are 
>not allowed simultaneously in the same IP subnet. The thought is that it 
>is a link characteristic and hence different per connection mode. It was 
>suggested that one be allowed to mix up RC/UC. This goes back to the 
>original suggestion in the first draft which was:  IPoIB-UD must always be 
>supported. Additionally, the interface can also support either both of RC 
>and UC, or one of them. Or neither of them.
>
>UD MUST always be supported.
>
><VK> That is and has always been the requirement right from the first 
>draft. <VK>
>
>I personally don't care whether one does RC or UC but I don't think both 
>are required as a MAY option. The advantage of RC is the send credit 
>algorithm. The advantage of UC is the lack of ACK packets. ACK is noise in 
>the fabric while send credits provide a simple method to maintain 
>bandwidth / injection control on a per flow basis.
>
>I see no problems with supporting both UD and *C on the same subnet; it is 
>rather foolish to attempt to mandate these be on separate subnets.b
><VK> As per the connected-mode draft the UD mechanism is *always* 
>required; address resolutoin depends on it.
>
>The only point of discussion is whether all nodes must support the same 
>link characteristics in the subnet i.e. all are RC (and UD), or all or UC 
>(and UD), or all are UD only.

Obviously I would oppose such a solution as it creates artificial 
constraints with little benefit.

>The alternative is to allow all the nodes to be mixed up with some nodes 
>being RC/UD, others UC/UD and a third set UD only and yet others probably 
>supporting all. within the same IP subnet. [Can the same serviceID be used 
>by both RC and UC ?]
>
>The third alternative is to associating UD only or UD + one of RC or UC on 
>the same interface. In such a case if mismatched/unsupported connected 
>modes are supported by two nodes then the fall back to UD. This option is 
>not too different from UD QP + RC or UC mechanism.

KISS:

- UD universal
- *C opportunistic
         - Local management issue to control what is sent on the *C 
interface.  No need to specify
         - Advertise whether one or more ports are supported by UD or *C
         - Advertise whether one or more QP are supported by UD or *C
         - Let local management determine policy for what services are 
mapped where - no need to specify

This is both an interoperable approach and simple to implement.  There may 
be some desire to add a policy interface to state preference for specific 
types of traffic over a given QP.  I would not oppose this but would view 
this as a separate draft once the basics are worked out.



><VK>
>b. Another suggestion was to allow multiple connected mode links (i.e. at 
>IB UC/RC level) between peers.  One thought can be 'yes, but user beware': 
>The IB connections are made using the service ID that is derived from the 
>QPN as described in the draft. If a second attempt succeeds then there are 
>two links. It is up to the implementation to either allow or disallow 
>multiple links.
>
>Again, this has been suggested in the past (though most who were involved 
>in the original discussions years gone by are likely gone since much of 
>this discussion occurred before the IETF workgroup was established).
>
><VK> I'm one of the vestiges of those early times along with you and a few 
>others...so we have hope :). <VK>
>
>There is obvious benefit to supporting multiple RC per endnode pair. I do 
>not see any technical reason to oppose nor any issue from an 
>interoperability perspective. There is no reason for a "user beware".
>
><VK> It is not opposed. The 'user beware' is only underscoring that the 
>the peer interface might not support multiple links- it might enforce a 
>limited number of connections (maybe only one) between a pair of GIDs. 
>Similarly, an implementation not wanting to support multiple links MUST 
>take steps to deny multiple requests.

*C requires CM to operate thus it is a local issue whether additional CM 
operations are accepted or not.  A given requester node may issue N and a 
given responder may state 0-N as an implementation may limit the number of 
*C available for IP traffic.


><VK>
>
>The work is rather straight to do and implement and the benefit to 
>customers, is again, rather obvious when one considers what the IB fabric 
>offers and how connections can be enable flows through multipath as well 
>as transparent fail-over, flow scheduling, mapping of DiffServ to 
>different arbitration / paths, etc.
>
><VK> In addition Large MTU and APM are two of the main reasons why I've 
>been proposing IPoIB-connected mode for so long. In terms of IPoIB itself, 
>except for the Large MTU, the parameters are hidden from it.<VK>

Mike

_______________________________________________
IPoverIB mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/ipoverib