RE: GID clarifications

At 12:44 PM 1/5/2004 -0800, bill wrote:
>Folks,
>
>Looks like there are good reasons to clean up the definition of the GID
>to be used (Making it SHOULD use GID0)
>
>I think that is concensus that everyone can live with.
>
>Objections ?

None.  Some comments in-lined below.

Mike


>Bill
>
>-----Original Message-----
>From: [email protected] [mailto:[email protected]] On Behalf
>Of Kanoj Sarcar
>Sent: Monday, December 08, 2003 7:06 PM
>To: Michael Krause
>Cc: [email protected]
>Subject: Re: [Ipoverib] GID clarifications
>
>
>Michael Krause wrote:
> >
> > At 08:22 PM 12/7/2003 -0800, H.K. Jerry Chu wrote:
> > >Mike,
>
>Hi,
>
>Michael, thanks for providing the IBTA context about multiple GIDs and
>path selection.
>
> > >
> > > >At 06:04 PM 12/4/2003 -0800, bill wrote:
> > > >>Ok, let me ask a few questions to lead us toward concensus
> > > >
> > > >As the author of the IB spec for chapter 4, I'll attempt to provide
>
> > > >some clarifications.
> > > >
> > > >>1) Does anyone give a compelling reason to use anything but GID0
> > > >>for IPoIB
> > > >>- what is the benefit of doing this ?
> > > >
> > > >Multiple GIDs were provided as a method to:
> > > >
> > > >- Provide traffic segregation between subnets conceptually similar
> > > >to multi-LID usage within a single subnet.
> > >
> > >According to IBA 1.0a spec (i don't have 1.1 spec at hand), "A subnet
>
> > >shall have only one assigned GID prefix at any given time (at GID
> > >index 0)." It seems to me unless a CA port can be simultaneously
> > >connected to more than one IB subnet, the unique subnet prefix plus
> > >the default GID prefix (the link-local 0xFF80 prefix) are the only
> > >two possible values to have for the upper 64 bits of a GID.
> >
> > Correct.  One can have either the default or an assigned thus all GID
> > share one or the other.
> >
> > >  I wonder what the traffic seggregation between subnets you have in
> > > mind.
> >
> > The basic idea was one could encode in the lower 64 bits (these are
> > admin assigned if not the port EUI64 value) or through a combination
> > of other GRH attributes (e.g. TClass, Flow Label, etc.) one could
> > segregate traffic to take different paths when passing through a
> > router. The path selection would be implemented via selecting an
> > egress port and assigning an appropriate next router hop / endnode
> > LID.
>
>It seems to me that for endports to influence which path to
>use for each UD packet, the SM and the endnodes need to exchange
>some information. I was under the impression that SL/VL information
>within a IBA subnet, and additionally Tclass etc between IBA
>subnets, would be sufficient; ie, segregation could be achieved with
>just these parameters, and would not need to extend to source GID. I am
>not sure whether your use of the word "or" above means that this is
>still a viable option.

It is still viable.

> > > >- Provide a method to support software / OS virtualization and
> > > >device sharing via using a unique global identifier independent of
> > > >the number of LIDs assigned (though one can take a similar approach
>
> > > >if the LMC value is greater than 0.
> > > >
> > > >I will note that there are many OSVs offering or starting to offer
> > > >virtualization services and while some initial implementations may
> > > >be focused on GID 0, this may not hold true in the coming few
> > > >years.
> > >
> > >Not sure what flavor of "virtualization" you're alluding to. If the
> > >virtualization is achieved through the use of multiple IB partitions,
>
> > >different IB partitions can safely share the same GID much like
> > >different Ethernet VLAN sharing the same MAC address. I don't see the
>
> > >need for multiple GIDs.
> >
> > Virtualization is nominally a function of address while partitions are
>
> > a function of segregation enforcement ala 802.1Q.  While one can use
> > partitions, they are not seen by the underlying software while a GRH
> > is seen thus depending upon how it is implemented, how the remote peer
>
> > "sees" each target peer, etc. a GID is more flexible and may align
> > better with application usage.  Please keep in mind I'm explaining how
>
> > the motivation for multiple GID from an IBA perspective.  Now, can one
>
> > use a single GID with multiple P_Key and then have a unique IP-to-QP
> > mapping per resulting partition to achieve something similar?  The
> > answer is yes, it is possible but that was not the intent of the IBTA
> > when this was first being debated.
> >
> > >Perhaps you meant virtualization like the hosting of multiple IP
> > >addressses with the same subnet prefix but different host numbers on
> > >the same NIC, this can be achieved through the use of multiple
> > >locally-assigned EUI-64s in IB, and I agree it is a useful feature to
>
> > >support.
>
>At least from the ipoib perspective, it is also possible to
>virtualize based on the QPN (using same Pkey), since that is
>part of the mac address. I do agree that virtualization based on GIDs is
>attractive.

It is possible to use multiple QPN and one would expect that there would be 
a unique QP per GID or per whatever is the attribute being used to 
differentiate traffic.


> > >
> > > >
> > > >- Provide a method to constrain out some data flows are routed -
> > > >local vs. non-local subnet access given a GID can use either the
> > > >default or the SM assigned subnet prefix.
> > >
> > >Note that the draft currently recommends the use of the IB subnet
> > >prefix for the GID in IPoIB. Although the real requirement is that
> > >the GID must be reachable (i.e. routable) at the IB layer from any
> > >node within the scope of the IP link it belongs to, recommending the
> > >IB subnet prefix is less error prone as it continues to work even
> > >after the underlying IP link has been expanded to cover multiple IB
> > >subnets.
> >
> > Agreed.
> >
> > > >
> > > >>2) Does anyone have an implementation that uses anything but GID0
> > > >>for
> > > IPoIB ?
> > > >
> > > >I believe it would be a mistake to limit support to just GID 0.  IB
>
> > > >technology was designed to support a diverse set of usage models
> > > >that will evolve over time.  The only potential problem was a DLPI
> > > >application raw level usage model which is (a) somewhat outside of
> > > >the scope of the IP over IB specification and (b) rather limited in
>
> > > >terms of usage within the industry.  I don't really want to debate
> > > >the merits of raw DLPI but do acknowledge it is in use today on
> > > >Ethernet and other layer 2 interconnects.
> > > >
>
>Note that as has been pointed out, it is also possible for non DLPI apps 
>to want to know the sender's L2 address (trivially possible on ethernet by 
>looking at the received packet and picking out the source ethernet 
>address). As you point out, this might be slightly out of scope, but 
>here's an example that is within scope: imagine a node A with mac address 
>M1 wants to know what IP address the RARP server will assign for a 
>different mac address M2; it can fabricate a RARP message and broadcast it 
>out; the RARP server must send the RARP response to M1 and not to M2. For 
>this, the RARP server must be able to determine that the sender's L2 
>address is M1. Similarly, an ARP implementation might want to
>determine the sender of an ARP request and validate against the "sender 
>hardware address" in the ARP message. I can't make a call on how extensive 
>this type of usage is though.

A lesson I've learned over the years in developing new technology is that 
"one can envision" many things which often are quite cool in various 
aspects.  However, the world often moves to a different beat as technology 
commodization (economic) combined with "good enough" technology eliminates 
the "envision" cool idea for KISS.


>Thanks.
>
>Kanoj
>
>
> > > >>If I do not hear any counter arguments by next Friday - I would
> > > >>like to ask for concensus on MUST use GID0
> > > >
> > > >Please consider this a counter argument and a vote against such a
> > > >movement.
> > >
> > >You are opposing requiring GID0, but are you ok with the current
> > >wording of RECOMMENDING the GID formed by the IB subnet prefix and
> > >the portGUID (the one assigned by the manufacturer)?
> >
> > I can live with RECOMMENDING not with MUST.
> >
> > >BTW, I'm confused by the use of GID0 as I thought in IBA spec. it
> > >means the one with the link-local prefix but I thought people wanted
> > >to use the one with the IB subnet prefix.
> >
> > GID 0 is just the guaranteed GID to be created via the combination of
> > the port GUID (manufacturer assigned GUID) and the subnet prefix.
> >
> > Mike
> >
> > >Jerry
> > >
> > > >
> > > >Mike
> > > >
> > > >
> > > >_______________________________________________
> > > >IPoverIB mailing list
> > > >[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
> >
> > _______________________________________________
> > IPoverIB mailing list
> > [email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
>
>_______________________________________________
>IPoverIB mailing list
>[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>_______________________________________________
>IPoverIB mailing list
>[email protected]
>https://www1.ietf.org/mailman/listinfo/ipoverib