Re: comments on draft-kashyap-ipoib-connected-mode-02.txt

On Thu, 18 Nov 2004, H.K. Jerry Chu wrote:

> In the last IETF61 IPoIB meeting I made several comments on the
> connected mode draft. I'm sending them to the list for a general
> discussion. (Yes I saw some disucssion on the connected mode
> draft already. I'll try to catch up with the thread after this
> mail.)
> 
> 1. The draft makes a distinction between IPoIB-CM interfaces
> and IPoIB-UD interfaces, and portrays IPoIB-UC or IPoIB-RC as
> separate subnets superimposed on top of an IPoIB-UD subnet.
> 
> For the above to work, due to a lack of multicast support, a fully
> connected network by itself can't meet the requirement of an IP
> link unless multicast is fully emulated through the use of
> multiple unicasts. The latter is complex and cumbersome.

Exactly. The current draft also continues to use UD for 
multicast.

> 
> A much simpler model, which I think was presented in earlier
> drafts, is to fold the use of IB connections fully into a
> regular IPoIB-UD subnet, allowing any two IPoIB nodes to
> optionally negotiate the use of IB connection between themselves.

The difference in the earlier draft and this one is that
I modified the requirement on the UD QP. That is, it need not be that
IPoIB-CM and IPoIB-UD share a QP but that any UD QP will do for IPoIB-CM. 
In effect an implementation can still share the UD QP.

The only issue is whether the same IP subnet can contain pure 
IPoIB-UD mixed in with IPoIB-CM nodes or, all nodes must be of the same type.
	- all IPoIB-UD
or
	- all IPoIB-RC

or 	-- all IPoIB-UC

I beleive all of the same type is a good option to choose.

> 
> This much simplified model is not without its drawback. Some
> nice IP link attributes are no longer unique within a link.
> E.g., the link MTU now becomes per-node-pair MTU. Moreover,
> the MTU size for multicast will be different from the MTU size
> for unicast if IB connections are used. IB UC/RC may exhibit
> different RAS, flow control, QoS or other link characteristics
> than UD. But I consider these problems a reasonable price to
> pay for a seamless support of UC/RC mode in an IPoIB link
> defined by UD.
> 
> 2. The negotiation of the per-connection MTU seems more
> complicated than necessary. I think all is needed is for a
> node to advertise its own "receive MTU". That is, the MTU
> size its peer should never go over when sending packets
> to the local interface. Yes this may break the traditional
> concept of "symmetric" MTUs. But we're already breaking the
> notion of per-link MTU, requring a lot of changes in the host
> stack anyway. This additonal breakage doesn't seem much.
> 
> I haven't verified if this asymmetric MTU matches well with
> IBA connections though.

How about:

The MTU I would think is exchanged at the IB level during the
IPoIB-CM connection setup. The IP layer at both ends keeps a per connection
MTU if the implementation permits it. At the link layer the connection will
not send messages larger than that requested by the peer.


> 
> 3. Regarding allowing multiple IB connections between a node
> pair, since given an IP address there is only one link-address
> for it implying one QPN, hence one service-ID, if a single
> service-ID can be used to create multiple IB connections
> then this can happen transparently. Otherwise we've got a
> problem.
> 
> Jerry
> 
> 
> _______________________________________________
> IPoverIB mailing list
> [email protected]
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
>