Re: GID clarifications

Ajoy C Siddabathuni wrote:
> 
> ! > As far as RECOMMENDED vs. MUST, going by the IETF standards, RECOMMENDED is
> ! > compelling enough for implementors to follow it blindly, still leaving room
> ! > for some clever implementations.
> ! >
> ! > <snip>
> ! > 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
> ! >    may exist valid reasons in particular circumstances to ignore a
> ! >    particular item, but the full implications must be understood and
> ! >    carefully weighed before choosing a different course.
> !
> ! Just to be careful, if a non DLPI system (eg Linux) uses GID1 for the mac
> ! address, the Linux system itself will function perfectly (ignoring whatever
> ! uniqueness problems there might be in choosing GID1 over GID0). The problem
> ! will be that any _other_ DLPI based system on the subnet will possibly not be
> ! able to interact 100% with the Linux box, even though the DLPI system itself
> ! uses GID0 per the draft recommendations. IE, for the Linux box, the "full
> ! implications" are not local, but have to do with interop.
> !
> ! It seems a bit much to expect non DLPI systems (like Linux) to understand
> ! what impact their using GID1 might have on _other_ DLPI based system. That
> ! is why this issue should be addressed in the draft/rfc.
> 
> The analysis provided seems to rely on two assumptions -
> 

Hi,

> 1) Linux/non-dlpi systems are free to use GID1 and thus will not interoperate
>    with Solaris
> 2) We expect implementations to interoperate in spite of the fact
>    that they willfully deviated from the spec.

"Willfully deviated" might be too strong, given that the working group currently
chooses to allow the flexibility of using other GIDs; what seems to be under
discussion is the relative benefit of allowing that flexibility, if there is a
cost to be paid in terms of interop.

> 
> Given the current definition of the ipoib pkt format, the issue on hand does
> not necessarily only apply to DLPI systems, any application talking to the ipoib
> driver directly (for whatever purposes) would have to deal with src/dst(IB)
> address lookup problems; when that information is not provided explicitly by the
> driver when passing up pkts. Thus, even Linux and other OSes should be
> implementing the rfc by its letter and intent.

Here is the difference between DLPI and nonDLPI systems: on non DLPI systems, if
the os allows raw access to the interface, then programs utilizing the raw
interface must neccesarily be aware of the underlying packet format, and either
deal with sender address not being present, or know how to decipher it from the
packet header. If you took a program that runs on raw Ethernet, there is a very
good chance you will need to modify the program to run on raw IPoIB.

With DLPI, there is an attempt to reduce the media awareness of programs, by
requiring the os to provide some meta information for every packet received
(sender's mac address in this case). In practice, this does not guarantee that 
a program that runs on ethernet will run unmodified on IPoIB, but there is a 
good chance that it might. 

As you point out, determining the sender's address is problematic in either
case (dlpi or non dlpi): in the case of dlpi though, the onus is on the driver
to provide the information (dlpi apps expect it), whereas in non dlpi case, 
programs can choose to solve the problem in their own unique ways.
Using GID0 always is one way to solve the problem for everyone, the other is to require 
using GRH for every packet transmitted, but that has a performance impact and
is thus not desirable.

> 
> Finally, if we were to make a case with such assumptions as above, do we then
> reconsider all uses of "RECOMMENDED/SHOULD" in the rfc.
>

If there is an interop issue, or a case where some established practice or
software breaks (due to an implementation not following recommendation), in my 
opinion, we should think about the item seriously. 

Is there a reason you are opposed to changing the RECOMMENDED to a REQUIRED
in this specific case? Vivek mentioned there was inclination towards allowing
the flexibility, and I generally agree to that; but in this case, curtailing
the flexibility is solving a problem, without creating any new ones I am aware 
of.

Thanks.

Kanoj



> thx,
> ~Ajoy
> 
> !
> ! Thanks.
> !
> ! Kanoj
> !
> ! >
> ! > thx,
> ! > ~Ajoy
> ! >
> ! > !
> ! > ! Vivek
> ! > !
> ! > ! On Wed, 3 Dec 2003, Kanoj Sarcar wrote:
> ! > !
> ! > ! > Vivek Kashyap wrote:
> ! > ! > >
> ! > ! > > Kanoj,
> ! > ! > >
> ! > ! > > The port's GUID is it's EUI-64 identifier. A GID, in turn can be
> formed
> ! > using this GUID, the CA's GUID or some other value. Any GID is unique in the
> IB
> ! > fabric however only by using the port's GUID/EUI-64 identifier can one
> uniquely
> ! > identify the port. The other GID's, as noted, can be applied to other ports
> too
> ! > - though not concurrently. The GID formed using the EUI-64 identifier is the
> ! > 'GID at index 0'. I've made note of your comment and can clarify further as
> ! > needed.
> ! > ! > >
> ! > ! > > wrt, using only GID at index 0 as in the MAC address, there have been
> ! > proponents of using the other GIDs (non EUI-64 GID).
> ! > ! > >
> ! > ! > > Vivek
> ! > ! >
> ! > ! > Vivek,
> ! > ! >
> ! > ! > Per IBA 1.1 section 4.1 EUI-64 subsection and 4.1.1 item 2, manufacturer
> ! > assigned and
> ! > ! > SM assigned values are required to be EUI-64 compliant (they differ only
> in
> ! > u/l bits).
> ! > ! > Thus, all GIDs contain an EUI-64 in the lower 64 bits; thus saying "GUID
> is
> ! > its EUI-64
> ! > ! > identifier" might be confusing. Saying "GUID using manufacturer assigned
> ! > EUI-64 identifier"
> ! > ! > or better "GID index 0" removes all confusion. I think we agree on the
> ! > intent, I just
> ! > ! > want to make sure if you use the term GUID, you do specifically define
> what
> ! > you mean
> ! > ! > by it.
> ! > ! >
> ! > ! > Regarding MUST vs RECOMMENDED, is it worthwhile reconsidering given that
> ! > ! >
> ! > ! > 1. SM assigned GIDs are not that popular (anymore?)
> ! > ! > 2. it allows determination of sender's ipoib address, thus allowing
> ! > (receiver side)
> ! > ! > implementations to be dlpiv2 conformant?
> ! > ! >
> ! > ! > Thanks.
> ! > ! >
> ! > ! > Kanoj
> ! > ! >
> ! > ! >
> ! > ! >
> ! > ! > > --
> ! > ! > > Vivek Kashyap
> ! > ! > > Linux Technology Center, IBM
> ! > ! > > [email protected]
> ! > ! > > [email protected]
> ! > ! > > Ph: 503 578 3422 T/L: 775 3422
> ! > ! > >
> ! > ! > >     Kanoj Sarcar <[email protected]>
> ! > ! > >     Sent by: [email protected]
> ! > ! > >
> ! > ! > >     12/02/2003 03:54 PM
> ! > ! > >     Please respond to Kanoj.Sarcar
> ! > ! > >
> ! > ! > > Hi,
> ! > ! > >
> ! > ! > > It seems that certain parts of the
> ! > ! > > draft-ietf-ipoib-ip-over-infiniband-05.txt draft might
> ! > ! > > require clarification. For this discussion, please refer to
> ! > ! > >
> ! > ! > > 1. section 9.1 Link Information says:
> ! > ! > > "The GID, if formed using the GUID, can be used to unambiguously
> ! > ! > > identify an endpoint."
> ! > ! > >
> ! > ! > > 2. section 9.1.1 Link Layer Address/Hardware Address, part c) says:
> ! > ! > > "It is RECOMMENDED that the GID formed by the combination of the
> ! > ! > > IB subnet prefix and the port's GUID be included in the
> ! > ! > > link-layer/hardware address."
> ! > ! > >
> ! > ! > > Can comment #2 be safely taken to mean that GID index 0 is
> ! > ! > > being RECOMMENDED? Port GUID seems somewhat ambiguous, since I
> ! > ! > > am not sure (after reading IBA 1.1 section 4.1) whether the
> ! > ! > > term GUID refers only to the manufacturer assigned value that appears
> ! > ! > > in lower 64 bits of GID index 0, or whether it also includes
> ! > ! > > lower 64 bits of SM assigned GIDs at higher indices. If GID index 0
> ! > ! > > was really intended, the comment should explicitly state so to prevent
> ! > ! > > confusion.
> ! > ! > >
> ! > ! > > Also, to make things simple, is it reasonable to REQUIRE
> implementations
> ! > ! > > to always use GID index 0 (maybe this depends on whether existing
> ! > ! > > implementations all use GID0 only)? The reason this might be
> ! > ! > > interesting is that when a node receives a packet without a GRH
> ! > ! > > (whereby it already knows the sender's LID and QPN), it can possibly
> ! > ! > > reverse lookup the sender's LID to identify the sender's GID index
> ! > ! > > 0 and concatenate the sender's QPN to figure out the sender's link
> ! > ! > > address. Thus, a receiver can always figure out the sender's address,
> ! > ! > > even when a GRH is not present. This would be similar to ethernet,
> ! > ! > > where a receiver can always determine a sender's address, and thus
> ! > ! > > is more DLPIv2 (unitdata_indication) friendly.
> ! > ! > >
> ! > ! > > Regarding comment #1, what does "unambiguously identify an endpoint"
> ! > ! > > mean? GIDs are unique in the subnet; and if an IP link contains
> ! > ! > > members in multiple IBA subnets, the subnet prefixes will disambiguate
> ! > ! > > which endport is being referred.
> ! > ! > >
> ! > ! > > Thanks.
> ! > ! > >
> ! > ! > > Kanoj
> ! > ! > >
> ! > ! > > _______________________________________________
> ! > ! > > IPoverIB mailing list
> ! > ! > > [email protected]
> ! > ! > > https://www1.ietf.org/mailman/listinfo/ipoverib
> ! > ! >
> ! > ! > _______________________________________________
> ! > ! > IPoverIB mailing list
> ! > ! > [email protected]
> ! > ! > https://www1.ietf.org/mailman/listinfo/ipoverib
> ! > ! >
> ! > ! >
> ! > !
> ! > ! __
> ! > !
> ! > ! Vivek Kashyap
> ! > ! Linux Technology Center, IBM
> ! > !
> ! > !
> ! > ! _______________________________________________
> ! > ! IPoverIB mailing list
> ! > ! [email protected]
> ! > ! https://www1.ietf.org/mailman/listinfo/ipoverib
> !
> ! _______________________________________________
> ! IPoverIB mailing list
> ! [email protected]
> ! https://www1.ietf.org/mailman/listinfo/ipoverib