RE: GID clarifications

Ok, let me ask a few questions to lead us toward concensus

1) Does anyone give a compelling reason to use anything but GID0 for
IPoIB - what is the benefit of doing this ?

2) Does anyone have an implementation that uses anything but GID0 for
IPoIB ?

If I do not hear any counter arguments by next Friday - I would like to
ask for concensus on MUST use GID0

Bill

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf
Of Kanoj Sarcar
Sent: Thursday, December 04, 2003 4:44 PM
To: Ajoy C Siddabathuni
Cc: [email protected]; [email protected]; [email protected];
[email protected]; Kanoj Sarcar
Subject: Re: [Ipoverib] GID clarifications


Ajoy C Siddabathuni wrote:
> 
> ! > As far as RECOMMENDED vs. MUST, going by the IETF standards, 
> RECOMMENDED is ! > compelling enough for implementors to follow it 
> blindly, still leaving room ! > for some clever implementations. ! >
> ! > <snip>
> ! > 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that
there
> ! >    may exist valid reasons in particular circumstances to ignore a
> ! >    particular item, but the full implications must be understood
and
> ! >    carefully weighed before choosing a different course.
> !
> ! Just to be careful, if a non DLPI system (eg Linux) uses GID1 for
the mac
> ! address, the Linux system itself will function perfectly (ignoring
whatever
> ! uniqueness problems there might be in choosing GID1 over GID0). The
problem
> ! will be that any _other_ DLPI based system on the subnet will
possibly not be
> ! able to interact 100% with the Linux box, even though the DLPI
system itself
> ! uses GID0 per the draft recommendations. IE, for the Linux box, the
"full
> ! implications" are not local, but have to do with interop.
> !
> ! It seems a bit much to expect non DLPI systems (like Linux) to
understand
> ! what impact their using GID1 might have on _other_ DLPI based
system. That
> ! is why this issue should be addressed in the draft/rfc.
> 
> The analysis provided seems to rely on two assumptions -
> 

Hi,

> 1) Linux/non-dlpi systems are free to use GID1 and thus will not
interoperate
>    with Solaris
> 2) We expect implementations to interoperate in spite of the fact
>    that they willfully deviated from the spec.

"Willfully deviated" might be too strong, given that the working group
currently chooses to allow the flexibility of using other GIDs; what
seems to be under discussion is the relative benefit of allowing that
flexibility, if there is a cost to be paid in terms of interop.

> 
> Given the current definition of the ipoib pkt format, the issue on 
> hand does not necessarily only apply to DLPI systems, any application 
> talking to the ipoib driver directly (for whatever purposes) would 
> have to deal with src/dst(IB) address lookup problems; when that 
> information is not provided explicitly by the driver when passing up 
> pkts. Thus, even Linux and other OSes should be implementing the rfc 
> by its letter and intent.

Here is the difference between DLPI and nonDLPI systems: on non DLPI
systems, if the os allows raw access to the interface, then programs
utilizing the raw interface must neccesarily be aware of the underlying
packet format, and either deal with sender address not being present, or
know how to decipher it from the packet header. If you took a program
that runs on raw Ethernet, there is a very good chance you will need to
modify the program to run on raw IPoIB.

With DLPI, there is an attempt to reduce the media awareness of
programs, by requiring the os to provide some meta information for every
packet received (sender's mac address in this case). In practice, this
does not guarantee that 
a program that runs on ethernet will run unmodified on IPoIB, but there
is a 
good chance that it might. 

As you point out, determining the sender's address is problematic in
either case (dlpi or non dlpi): in the case of dlpi though, the onus is
on the driver to provide the information (dlpi apps expect it), whereas
in non dlpi case, 
programs can choose to solve the problem in their own unique ways. Using
GID0 always is one way to solve the problem for everyone, the other is
to require 
using GRH for every packet transmitted, but that has a performance
impact and is thus not desirable.

> 
> Finally, if we were to make a case with such assumptions as above, do 
> we then reconsider all uses of "RECOMMENDED/SHOULD" in the rfc.
>

If there is an interop issue, or a case where some established practice
or software breaks (due to an implementation not following
recommendation), in my 
opinion, we should think about the item seriously. 

Is there a reason you are opposed to changing the RECOMMENDED to a
REQUIRED in this specific case? Vivek mentioned there was inclination
towards allowing the flexibility, and I generally agree to that; but in
this case, curtailing the flexibility is solving a problem, without
creating any new ones I am aware 
of.

Thanks.

Kanoj



> thx,
> ~Ajoy
> 
> !
> ! Thanks.
> !
> ! Kanoj
> !
> ! >
> ! > thx,
> ! > ~Ajoy
> ! >
> ! > !
> ! > ! Vivek
> ! > !
> ! > ! On Wed, 3 Dec 2003, Kanoj Sarcar wrote:
> ! > !
> ! > ! > Vivek Kashyap wrote:
> ! > ! > >
> ! > ! > > Kanoj,
> ! > ! > >
> ! > ! > > The port's GUID is it's EUI-64 identifier. A GID, in turn 
> can be formed ! > using this GUID, the CA's GUID or some other value. 
> Any GID is unique in the IB
> ! > fabric however only by using the port's GUID/EUI-64 identifier can
one
> uniquely
> ! > identify the port. The other GID's, as noted, can be applied to
other ports
> too
> ! > - though not concurrently. The GID formed using the EUI-64
identifier is the
> ! > 'GID at index 0'. I've made note of your comment and can clarify
further as
> ! > needed.
> ! > ! > >
> ! > ! > > wrt, using only GID at index 0 as in the MAC address, there
have been
> ! > proponents of using the other GIDs (non EUI-64 GID).
> ! > ! > >
> ! > ! > > Vivek
> ! > ! >
> ! > ! > Vivek,
> ! > ! >
> ! > ! > Per IBA 1.1 section 4.1 EUI-64 subsection and 4.1.1 item 2,
manufacturer
> ! > assigned and
> ! > ! > SM assigned values are required to be EUI-64 compliant (they
differ only
> in
> ! > u/l bits).
> ! > ! > Thus, all GIDs contain an EUI-64 in the lower 64 bits; thus
saying "GUID
> is
> ! > its EUI-64
> ! > ! > identifier" might be confusing. Saying "GUID using
manufacturer assigned
> ! > EUI-64 identifier"
> ! > ! > or better "GID index 0" removes all confusion. I think we
agree on the
> ! > intent, I just
> ! > ! > want to make sure if you use the term GUID, you do
specifically define
> what
> ! > you mean
> ! > ! > by it.
> ! > ! >
> ! > ! > Regarding MUST vs RECOMMENDED, is it worthwhile reconsidering
given that
> ! > ! >
> ! > ! > 1. SM assigned GIDs are not that popular (anymore?)
> ! > ! > 2. it allows determination of sender's ipoib address, thus
allowing
> ! > (receiver side)
> ! > ! > implementations to be dlpiv2 conformant?
> ! > ! >
> ! > ! > Thanks.
> ! > ! >
> ! > ! > Kanoj
> ! > ! >
> ! > ! >
> ! > ! >
> ! > ! > > --
> ! > ! > > Vivek Kashyap
> ! > ! > > Linux Technology Center, IBM
> ! > ! > > [email protected]
> ! > ! > > [email protected]
> ! > ! > > Ph: 503 578 3422 T/L: 775 3422
> ! > ! > >
> ! > ! > >     Kanoj Sarcar <[email protected]>
> ! > ! > >     Sent by: [email protected]
> ! > ! > >
> ! > ! > >     12/02/2003 03:54 PM
> ! > ! > >     Please respond to Kanoj.Sarcar
> ! > ! > >
> ! > ! > > Hi,
> ! > ! > >
> ! > ! > > It seems that certain parts of the
> ! > ! > > draft-ietf-ipoib-ip-over-infiniband-05.txt draft might
> ! > ! > > require clarification. For this discussion, please refer to
> ! > ! > >
> ! > ! > > 1. section 9.1 Link Information says:
> ! > ! > > "The GID, if formed using the GUID, can be used to
unambiguously
> ! > ! > > identify an endpoint."
> ! > ! > >
> ! > ! > > 2. section 9.1.1 Link Layer Address/Hardware Address, part
c) says:
> ! > ! > > "It is RECOMMENDED that the GID formed by the combination of
the
> ! > ! > > IB subnet prefix and the port's GUID be included in the
> ! > ! > > link-layer/hardware address."
> ! > ! > >
> ! > ! > > Can comment #2 be safely taken to mean that GID index 0 is
> ! > ! > > being RECOMMENDED? Port GUID seems somewhat ambiguous, since
I
> ! > ! > > am not sure (after reading IBA 1.1 section 4.1) whether the
> ! > ! > > term GUID refers only to the manufacturer assigned value
that appears
> ! > ! > > in lower 64 bits of GID index 0, or whether it also includes
> ! > ! > > lower 64 bits of SM assigned GIDs at higher indices. If GID
index 0
> ! > ! > > was really intended, the comment should explicitly state so
to prevent
> ! > ! > > confusion.
> ! > ! > >
> ! > ! > > Also, to make things simple, is it reasonable to REQUIRE
> implementations
> ! > ! > > to always use GID index 0 (maybe this depends on whether
existing
> ! > ! > > implementations all use GID0 only)? The reason this might be
> ! > ! > > interesting is that when a node receives a packet without a
GRH
> ! > ! > > (whereby it already knows the sender's LID and QPN), it can
possibly
> ! > ! > > reverse lookup the sender's LID to identify the sender's GID
index
> ! > ! > > 0 and concatenate the sender's QPN to figure out the
sender's link
> ! > ! > > address. Thus, a receiver can always figure out the sender's
address,
> ! > ! > > even when a GRH is not present. This would be similar to
ethernet,
> ! > ! > > where a receiver can always determine a sender's address,
and thus
> ! > ! > > is more DLPIv2 (unitdata_indication) friendly.
> ! > ! > >
> ! > ! > > Regarding comment #1, what does "unambiguously identify an
endpoint"
> ! > ! > > mean? GIDs are unique in the subnet; and if an IP link
contains
> ! > ! > > members in multiple IBA subnets, the subnet prefixes will
disambiguate
> ! > ! > > which endport is being referred.
> ! > ! > >
> ! > ! > > Thanks.
> ! > ! > >
> ! > ! > > Kanoj
> ! > ! > >
> ! > ! > > _______________________________________________
> ! > ! > > IPoverIB mailing list
> ! > ! > > [email protected]
> ! > ! > > https://www1.ietf.org/mailman/listinfo/ipoverib
> ! > ! >
> ! > ! > _______________________________________________
> ! > ! > IPoverIB mailing list
> ! > ! > [email protected]
> ! > ! > https://www1.ietf.org/mailman/listinfo/ipoverib
> ! > ! >
> ! > ! >
> ! > !
> ! > ! __
> ! > !
> ! > ! Vivek Kashyap
> ! > ! Linux Technology Center, IBM
> ! > !
> ! > !
> ! > ! _______________________________________________
> ! > ! IPoverIB mailing list
> ! > ! [email protected]
> ! > ! https://www1.ietf.org/mailman/listinfo/ipoverib
> !
> ! _______________________________________________
> ! IPoverIB mailing list
> ! [email protected]
> ! https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email protected] https://www1.ietf.org/mailman/listinfo/ipoverib