Re: comments on draft-kashyap-ipoib-connected-mode-02.txt

<snip>

>> A much simpler model, which I think was presented in earlier
>> drafts, is to fold the use of IB connections fully into a
>> regular IPoIB-UD subnet, allowing any two IPoIB nodes to
>> optionally negotiate the use of IB connection between themselves.
>
>The difference in the earlier draft and this one is that
>I modified the requirement on the UD QP. That is, it need not be that
>IPoIB-CM and IPoIB-UD share a QP but that any UD QP will do for IPoIB-CM. 
>In effect an implementation can still share the UD QP.
>
>The only issue is whether the same IP subnet can contain pure 
>IPoIB-UD mixed in with IPoIB-CM nodes or, all nodes must be of the same type.
>	- all IPoIB-UD
>or
>	- all IPoIB-RC
>
>or 	-- all IPoIB-UC
>
>I beleive all of the same type is a good option to choose.

I don't see a clear benefit for this restriction. E.g., even in all IPoIB-RC or
IPoIB-UC, the nice per-link MTU property is no longer there due to multicast
supported through UD. Also this restriction will require those implementations
that don't support IPoIB over UC or RC to form a different subnet in order to
talk IPoIB, hence forcing the adminstrator to maintain at least two IP subnets
with one fully contained within another. I don't see why this is needed.

>
>> 
>> This much simplified model is not without its drawback. Some
>> nice IP link attributes are no longer unique within a link.
>> E.g., the link MTU now becomes per-node-pair MTU. Moreover,
>> the MTU size for multicast will be different from the MTU size
>> for unicast if IB connections are used. IB UC/RC may exhibit
>> different RAS, flow control, QoS or other link characteristics
>> than UD. But I consider these problems a reasonable price to
>> pay for a seamless support of UC/RC mode in an IPoIB link
>> defined by UD.
>> 
>> 2. The negotiation of the per-connection MTU seems more
>> complicated than necessary. I think all is needed is for a
>> node to advertise its own "receive MTU". That is, the MTU
>> size its peer should never go over when sending packets
>> to the local interface. Yes this may break the traditional
>> concept of "symmetric" MTUs. But we're already breaking the
>> notion of per-link MTU, requring a lot of changes in the host
>> stack anyway. This additonal breakage doesn't seem much.
>> 
>> I haven't verified if this asymmetric MTU matches well with
>> IBA connections though.
>
>How about:
>
>The MTU I would think is exchanged at the IB level during the
>IPoIB-CM connection setup. The IP layer at both ends keeps a per connection
>MTU if the implementation permits it. At the link layer the connection will
>not send messages larger than that requested by the peer.

Not quite understand the above. I'm suggesting to simplify the MTU negotiation
at the IPoIB-CM connection setup time by each side advertising the "receive
MTU" it can take. The peer must not send more than that size in each post_send().
E.g., if node A advertizes 32KB as its receive MTU and node B advertizes 64KB
as its receive MTU, node B must not send any IP pkt through IPoIB-CM to node
A that is larger than 32KB. Node A is free to send IP pkts of up to 64KB in
size to node B. (But if node A decides to restrict its outbound MTU to 32KB,
that's fine too. Node B doesn't need to know about it.)

I'm not sure what you mean by the last two sentences above. MTU value must
be made known to the IP layer so that latter won't send anything larger
than that. Otherwise the pkt will get dropped by the IB layer (unless the
latter performs SAR, which is a bad idea).

Jerry

>
>
>> 
>> 3. Regarding allowing multiple IB connections between a node
>> pair, since given an IP address there is only one link-address
>> for it implying one QPN, hence one service-ID, if a single
>> service-ID can be used to create multiple IB connections
>> then this can happen transparently. Otherwise we've got a
>> problem.
>> 
>> Jerry
>> 
>> 
>> _______________________________________________
>> IPoverIB mailing list
>> [email protected]
>> https://www1.ietf.org/mailman/listinfo/ipoverib
>> 
>> 
>
>
>