Re: IPoIB link address

On Wed, 24 Sep 2003, H.K. Jerry Chu wrote:

> Following are my comments (as a co-author not a co-chair).
> 
> From: Michael Krause <[email protected]>
> 
> >At 01:25 AM 9/24/2003 -0700, Vivek Kashyap wrote:
> >>On Tue, 23 Sep 2003, Kanoj Sarcar wrote:
> >>
> >> > Vivek Kashyap wrote:
> >> > >
> >> > > Bill,
> >> > >
> >> > > [Vandana had asked me this question and so I brought it up with you and
> >> > > Jerry. I then requested Vandana to send it to the list. So not a
> >> > > coincidence :)].
> >> > >
> >> > > I discussed it with Jerry and it is clear that the drafts are not 
> >> explicit
> >> > > in making it a requirement that the attributes have to be the same. There
> >> > > are implementations that can work with different Q_Keys (since they 
> >> choose
> >> > > to support different QPs), however an implementation that cannot/does not
> >> > > choose different QPs for the two protocols will have to 'work harder'
> >> > > within the driver to make it work. Therefore I agree that a common set of
> >> > > Q_Keys (and other attributes) needs to be an explicit requirement.
> >> > > Architecturally too, a link shouldn't depend on the upper layer 
> >> protocol --
> >> > > whether it is v4 or v6 -- it is just a link.
> >> > >
> >> > > How we do it has two options:
> >> >
> >> > Hi,
> >>
> >>Hello,
> >>
> >> >
> >> > >
> >> > > 1) Two Broadcast-GIDs
> >> > >
> >> > >    a) Explicitly require that the two broadcast-GIDs have same attribute
> >> > >    values
> >> > >    b) Explicitly require that both the broadcast-GIDs are always created
> >> > >
> >> > >    This is closer to the current specification except we are adding some
> >> > >    requirements.
> >> >
> >> > Note that this scheme allows the fabric administrator (via the creation of
> >> > v4 and/or v6 groups) to indicate to the nodes whether v4 and/or v6 
> >> operation
> >> > is intended on the fabric. Smart nodes can then decide to fail or succeed
> >> > setting up IPv4 or IPv6 protocol on the interface (during the DLPI bind
> >> > operation or similar). Whether such a way of controlling operation of the
> >> > nodes by the fabric administrator is desirable or not is another issue.
> >>
> >>Agreed that it allows such a function. However, I also believe that such a
> >>notion must be rare.  When a user sets up an ip interface he/she expects to
> >>just assign IP addresses and send the data out. It would be odd if the user
> >>is only able to assign one set of addresses but not the other set or transmit
> >>v4 packets but not v6 packets.
> >
> >Agreed.  Layer 2 should stay as ignorant about Layer 3 and above thus 
> >applying KISS, a single QP should support both v4 / v6.
> 
> Actually the notion of v4-only interface or v6-only interface is not
> rare. Whether or not it's useful or even appropriate to allow a L2
> configuration process to influence a L3 decision is a different matter.

Interface at the IP layer is fine..however, in this case the lower
layer will dictate/deny the IP  layer from even defining an IP
interface supporting the other version. 

> 
> Wrt a single QP supporting both v4/v6, two broadcast groups with the same
> attribute set allow that too.
> 
> >
> >> > There is also the issue of broadcast traffic isolation that has been
> >> > pointed out; any port which has a only a single qpn accepting v6 traffic
> >> > currently does not need to receive broadcast traffic, saving both fabric
> >> > resources and host interruption.
> >>
> >>The v6 qpn need not join the broadcast-GID if option 2) below is chosen. The
> >>driver can, on startup, join the broadcast-GID on the qpn associated with v4
> >>broadcasts. The attributes can then be borrowed for the v6 qpn without it
> >>having to actually join the broadcast-GID. The isolation can still be
> >>maintained in those implementations that prefer it.
> >
> >The expectation was that this broadcast GID was primarily "management" 
> >focused (e.g. ARP / ND) thus the overhead / interrupt rate / etc. is 
> >expected to be relatively low.  If we all agree that the separation does 
> >not impact interoperability, then it is a local issue.
> 
> Actually the broadcast traffic is not rare either, not until we move
> all the apps off broadcast to SLP/DHCP or well-known multicast.
> IPv6 has got it right by basing ND on multicast addresses instead of
> the broadcast address.

Yes, however, as I noted earlier an implementation can always join the
broadcast-GID on one QP and then borrow its attributes for v6
multicast related QPs; it does not have to join the broadcast-GID
itself. Therefore an implementation can choose to keep the traffic
separate.

> 
> >
> >>I'm unclear on the port supporting only one of the ip protocols and not 
> >>the other.
> >
> >Don't do this with other layer 2 so why do this now?
> 
> Ok.
> 
> From the responses so far it looks like the majority prefer a single
> broadcast group. Therefore I'd like to propose the following amendment
> to the draft (similar to what Kanoj has proposed):
> 
> 1. a single broadcast group will be defined and shared by v4 and v6.
> Both v4 and v6 will use the single broadcast group to retrieve link
> attributes, thus no room for errors.
> 
> 2. the above allows the single-QP implementation, which is deemed by
> many as a requirement.
> 
> 3. the draft continues to be silent on the use of multiple QPs. It is
> completely left to the implementations.
> 
> 4. IPv4 broadcast address (255.255.255.255), as well as all the
> net-directed, subnet-directed and all-subnets-directed broadcast
> addresses will map to this combined broadcast group.
> 
> 5. The current multicast mapping stays unchanged.
> 
> 6. The broadcast group takes the following format:
> 
> FF1X:401B:<P_Key>:0::1

My suggestion is to allow the broadcast-GID to be independent of
multicast mappings. Therefore, it would take the form:

FF1x:0000:<P_Key>::255.255.255.255.

Or, if a signature is preferred:

FF1x:001B:<P_Key>::255.255.255.255

The multicast mapping can have the protocol signature in it. As I
noted earlier we can view this as equal to the all 1's ethernet
address. The multicast mapping can continue to, again analogous to
Ethernet, as :

FF1x:401B:<P_Key>::<28 bits>
FF1x:601B:<P_Key>: <80 bits>


Vivek



> 
> This allows the IPv4 all-node multicast address (224.0.0.1) to map
> naturally to this broadcast group too, saving one IB MC group.
> 
> If people don't like mixing L3 all-node multicast with L2 broadcast,
> we can use the existing v4 broadcast group, or even something else.
> 
> 7. So long as we don't pick FF1X:601B:<P_Key>:0::1 as the combined
> broadcast group, the IPv6 all-node multicast address will not share
> broadcast traffic with IPv4. Thus we continue to maintain some
> level of traffic separation. This addresses my previous concern.
> 
> Comments are welcomed.
> 
> Jerry
> 
> >
> >
> >> >
> >> > >
> >> > > 2) One common broadcast-GID
> >> > >
> >> > >    a) One broadcast-GID that the driver joins. [the protocol 
> >> signatures can
> >> > >    still be included in the multicast mappings].
> >> > >
> >> > >    This is architecturally cleaner and may be for implementation too --
> >> > >    only one broadcast-GID to be created and joined.
> >> >
> >> > Yes, its does seem cleaner and simpler, and makes fabric configuration
> >> > errors less likely (note that the fabric administrator still can create
> >> > a multicast group corresponding to one of the v4/v6 IP multicast addresses
> >> > with the wrong qkey).
> >>
> >>Yes that is possible. However, in general I would expect the setup to be such
> >>that the multicast GIDs are created by the nodes based on the broadcast-GID
> >>attributes. Since first full-join is a create this shouldn't be much of a
> >>problem for creating the all-nodes v6 multicast GID.
> >>
> >> >
> >> > If this common group is adopted, one possibility to think about is making
> >> > the link broadcast address the same as the link multicast address for
> >> > 224.0.0.1 or FF02:0:0:0:0:0:0:1, resulting in a reduction of one
> >> > multicast group required for v4 and/or v6 operation on the fabric.
> >>
> >>I prefer your alternative suggestion. I am viewing the common broadcast-GID
> >>to be the same as the all-1's link address in Ethernet. The multicast
> >>mapping is only for IP multicast addresses same as the ethernet
> >>model (the mapping itself is different from ethernet of course).
> >>[nit: it is FF12::1 and not FF02::1].
> >>
> >> >
> >> > Alternatively, what might work best (ie requires minimal changes
> >> > to existing implementations) is naming the current v4 broadcast group as
> >> > the common group, and eliminating any implications that the v6 all-node
> >> > group should be created by the fabric administrator.
> >>
> >>Yes, this works well. I'm ok with that though would still prefer that, if
> >>this option is chosen, we drop the protocol signature from the
> >>broadcast GID.
> >
> >Mike
> >
> >
> >_______________________________________________
> >IPoverIB mailing list
> >[email protected]
> >https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> 
> _______________________________________________
> IPoverIB mailing list
> [email protected]
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 
>