RE: GID clarifications

Folks,

Looks like there are good reasons to clean up the definition of the GID
to be used (Making it SHOULD use GID0)

I think that is concensus that everyone can live with.

Objections ?

Bill

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf
Of Kanoj Sarcar
Sent: Monday, December 08, 2003 7:06 PM
To: Michael Krause
Cc: [email protected]
Subject: Re: [Ipoverib] GID clarifications


Michael Krause wrote:
> 
> At 08:22 PM 12/7/2003 -0800, H.K. Jerry Chu wrote:
> >Mike,

Hi,

Michael, thanks for providing the IBTA context about multiple GIDs and
path selection.

> >
> > >At 06:04 PM 12/4/2003 -0800, bill wrote:
> > >>Ok, let me ask a few questions to lead us toward concensus
> > >
> > >As the author of the IB spec for chapter 4, I'll attempt to provide

> > >some clarifications.
> > >
> > >>1) Does anyone give a compelling reason to use anything but GID0 
> > >>for IPoIB
> > >>- what is the benefit of doing this ?
> > >
> > >Multiple GIDs were provided as a method to:
> > >
> > >- Provide traffic segregation between subnets conceptually similar 
> > >to multi-LID usage within a single subnet.
> >
> >According to IBA 1.0a spec (i don't have 1.1 spec at hand), "A subnet

> >shall have only one assigned GID prefix at any given time (at GID 
> >index 0)." It seems to me unless a CA port can be simultaneously 
> >connected to more than one IB subnet, the unique subnet prefix plus 
> >the default GID prefix (the link-local 0xFF80 prefix) are the only 
> >two possible values to have for the upper 64 bits of a GID.
> 
> Correct.  One can have either the default or an assigned thus all GID 
> share one or the other.
> 
> >  I wonder what the traffic seggregation between subnets you have in 
> > mind.
> 
> The basic idea was one could encode in the lower 64 bits (these are 
> admin assigned if not the port EUI64 value) or through a combination 
> of other GRH attributes (e.g. TClass, Flow Label, etc.) one could 
> segregate traffic to take different paths when passing through a 
> router. The path selection would be implemented via selecting an 
> egress port and assigning an appropriate next router hop / endnode 
> LID.

It seems to me that for endports to influence which path to
use for each UD packet, the SM and the endnodes need to exchange 
some information. I was under the impression that SL/VL information 
within a IBA subnet, and additionally Tclass etc between IBA 
subnets, would be sufficient; ie, segregation could be achieved with
just these parameters, and would not need to extend to source GID. I am
not sure whether your use of the word "or" above means that this is
still a viable option.

> 
> > >- Provide a method to support software / OS virtualization and 
> > >device sharing via using a unique global identifier independent of 
> > >the number of LIDs assigned (though one can take a similar approach

> > >if the LMC value is greater than 0.
> > >
> > >I will note that there are many OSVs offering or starting to offer 
> > >virtualization services and while some initial implementations may 
> > >be focused on GID 0, this may not hold true in the coming few 
> > >years.
> >
> >Not sure what flavor of "virtualization" you're alluding to. If the 
> >virtualization is achieved through the use of multiple IB partitions,

> >different IB partitions can safely share the same GID much like 
> >different Ethernet VLAN sharing the same MAC address. I don't see the

> >need for multiple GIDs.
> 
> Virtualization is nominally a function of address while partitions are

> a function of segregation enforcement ala 802.1Q.  While one can use 
> partitions, they are not seen by the underlying software while a GRH 
> is seen thus depending upon how it is implemented, how the remote peer

> "sees" each target peer, etc. a GID is more flexible and may align 
> better with application usage.  Please keep in mind I'm explaining how

> the motivation for multiple GID from an IBA perspective.  Now, can one

> use a single GID with multiple P_Key and then have a unique IP-to-QP 
> mapping per resulting partition to achieve something similar?  The 
> answer is yes, it is possible but that was not the intent of the IBTA 
> when this was first being debated.
> 
> >Perhaps you meant virtualization like the hosting of multiple IP 
> >addressses with the same subnet prefix but different host numbers on 
> >the same NIC, this can be achieved through the use of multiple 
> >locally-assigned EUI-64s in IB, and I agree it is a useful feature to

> >support.

At least from the ipoib perspective, it is also possible to 
virtualize based on the QPN (using same Pkey), since that is 
part of the mac address. I do agree that virtualization based on GIDs is
attractive.

> >
> > >
> > >- Provide a method to constrain out some data flows are routed - 
> > >local vs. non-local subnet access given a GID can use either the 
> > >default or the SM assigned subnet prefix.
> >
> >Note that the draft currently recommends the use of the IB subnet 
> >prefix for the GID in IPoIB. Although the real requirement is that 
> >the GID must be reachable (i.e. routable) at the IB layer from any 
> >node within the scope of the IP link it belongs to, recommending the 
> >IB subnet prefix is less error prone as it continues to work even 
> >after the underlying IP link has been expanded to cover multiple IB 
> >subnets.
> 
> Agreed.
> 
> > >
> > >>2) Does anyone have an implementation that uses anything but GID0 
> > >>for
> > IPoIB ?
> > >
> > >I believe it would be a mistake to limit support to just GID 0.  IB

> > >technology was designed to support a diverse set of usage models 
> > >that will evolve over time.  The only potential problem was a DLPI 
> > >application raw level usage model which is (a) somewhat outside of 
> > >the scope of the IP over IB specification and (b) rather limited in

> > >terms of usage within the industry.  I don't really want to debate 
> > >the merits of raw DLPI but do acknowledge it is in use today on 
> > >Ethernet and other layer 2 interconnects.
> > >

Note that as has been pointed out, it is also possible for non DLPI apps
to want to know the sender's L2 address (trivially possible on ethernet
by looking at the received packet and picking out the source ethernet
address). As you point out, this might be slightly out of scope, but
here's an example that is within
scope: imagine a node A with mac address M1 wants to know what IP
address the 
RARP server will assign for a different mac address M2; it can fabricate
a RARP 
message and broadcast it out; the RARP server must send the RARP
response to M1 and not to M2. For this, the RARP server must be able to
determine that the sender's L2 address is M1. Similarly, an ARP
implementation might want to 
determine the sender of an ARP request and validate against the "sender
hardware address" in the ARP message. I can't make a call on how
extensive this type of usage is though.

Thanks.

Kanoj


> > >>If I do not hear any counter arguments by next Friday - I would 
> > >>like to ask for concensus on MUST use GID0
> > >
> > >Please consider this a counter argument and a vote against such a 
> > >movement.
> >
> >You are opposing requiring GID0, but are you ok with the current 
> >wording of RECOMMENDING the GID formed by the IB subnet prefix and 
> >the portGUID (the one assigned by the manufacturer)?
> 
> I can live with RECOMMENDING not with MUST.
> 
> >BTW, I'm confused by the use of GID0 as I thought in IBA spec. it 
> >means the one with the link-local prefix but I thought people wanted 
> >to use the one with the IB subnet prefix.
> 
> GID 0 is just the guaranteed GID to be created via the combination of 
> the port GUID (manufacturer assigned GUID) and the subnet prefix.
> 
> Mike
> 
> >Jerry
> >
> > >
> > >Mike
> > >
> > >
> > >_______________________________________________
> > >IPoverIB mailing list
> > >[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
> >
> >
> >_______________________________________________
> >IPoverIB mailing list
> >[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib
> 
> _______________________________________________
> IPoverIB mailing list
> [email protected] https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib