Re: IPoIB link address

Hi Vivek,

<snip>

> >
> >         1. Define a broadcast-GID defined as follows (one of - please
> >         comment):
> >
> >         a) FF1x:401B::255.255.255.255
> >         b) FF1x:001B::255.255.255.255
> >         c) FF1x:0000::255.255.255.255
> >         d) FF1x:401B::1
> >
> >     My preference is b) or c) since it removes the IP version
> >     completely from the mapping. a) is what we have today and requires
> >     no change and so could be used. d) is derived from the multicast
> >     mapping and so I'd like to leave it separate.
>
>I would go for a) since it is already there; if we do change, I would
>go for d, but can live with b or c.
>
><VK> OK. I find (a), (b) and (c) acceptable. My order of preference (c),
>then (b) then (a). No (d). <VK>


My order of preference is (a) then (b). No (c) or (d). The reason I do not 
think (c) is a good idea
is because having a signature defined is useful to mark these MGIDs as 
IPoIB MGIDs. There is less possibility of MGID collision with non-IPoIB 
applications
Also, (a) makes the most sense as a broadcast group applies only to v4, so 
why not leave things as is.

> >
> >         2. The multicast mappings stay as currently specified.
> >
> >         3. The broadcast-GID (one of)
> >
> >         a) MUST be FullMember IB_joined
> >
> >         b) MUST be used to derive the attributes such as the MTU, Q_Key.
> >         The attributes MAY be obtained by FullMember IB_Join of the
> >         broadcast-GID or by some other operation supported by IBA. The
> >         node MUST fail all IP operations on the interface if the
> >         Broadcast-GID ceases to exist since that will correspond to a
> >         downed link. It is upto the implementation to track the existence
> >         fo the Broadcast-GID in a suitable way. Note: If the
>broadcast-GID
> >         is FullMember IB_joined then its existence/deletion is
> >         automatically tracked.
>
>I would go with option b, but without the requirement "MUST fail all IP
>operations ..." for multiple reasons; firstly, it requires at least a
>memory check in the send/receive path, and secondly, if other L2's
>(ethernet)
>aren't requring this, why should IB? (Seems like mingling an L2 issue with
>L3,
>the same logic that was used against seperate groups).
>
><VK> On further thinking tracking with a requirement to re-acquire modified
>attributes is needed. 'must fail' is not needed though for reasons below
>(analogous to why we want to track other multicast GIDs):
>
>Assume that the broadcast-GID was not FullMember joined and so ends up
>getting deleted. There are 3 possibilities (not counting a recreate with
>the same attributes and MLID):
>a) Broadcast-GID is not recreated.
>b) Broadcast-GID is recreated with different attributes but same MLID
>c) Broadcast-GID is recreated with different/same attributes but different
>MLID
>
>
>Now, if (a) occurs - the nodes still have agreed upon MTU, Q_Key etc. and
>can continue to communicate with peers. No new nodes can join the link
>though. No broadcasts can occur if the MLID was removed from the switch
>tables => no ARP. One  could let it all go on till nodes drop off(ARP
>timeouts for example) or declare interface is down.
>
>Now, if (b) occurs the link's attributes have changed and communication
>with nodes that were on the link before, and are still using the old
>information, can occur however communication with other nodes using new
>attributes on the link may fail. Broadcast however will probably succeed
>with some set of nodes (dependent on how the fabric is setup for this
>MLID).
>
>if (c) occurs then broadcast with nodes that were part of the link before,
>and are still using the old information, cannot occur (don't know the new
>MLID) though the node can still communicate with the nodes using older
>attributes.
>
>Therefore, the requriement really needs to be:
>       - track broadcast-GID
>       - reacquire attributes if broadcast-GID is recreated
>       - (possibly) if no broadcast-GID down the interface
>
>Thoughts?, especially for the case where the broadcast-GID is deleted and
>not brought back.
><VK>

I agree. If end-nodes do not want to join the broadcast GID, they need to 
deal with all the requirements that
you have detailed above.

-Vandana