RE: draft-carrier-rddp-rnic-interop-00.txt

Hi Mike,
 
thanks for your comments.  
 
On re-reading the drafts, I agree that the DDP draft does state very clearly that the version fields must match (sec 4.1).  However, I find no such statement in the RDMAP draft (other than stating the field must be 01b, there is no corresponding statement that the receiver must check or that, if it checks, what 'valid' means).   My error for over-simplifying to introduce the problem.
 
The interoperability issue I addressed in the RDDP WG meeting was that I knew there were schemes being considered (external to the IETF) for allowing RDMAC RNICs to interoperate with IETF RNICs and that unless the WG offered some informational text describing an interoperability mechanism, then the results would be open to vendor implementation.  
 
In the draft, Jim & I suggest changing the MPA version from 0 to 1 so that a ULP (or other entity) could do the MPA negotiation for an RDMAC RNIC (which would use the version of 0).  Very clearly an RDMAC-only RNIC without this other entity could not attempt to interoperate with a "Permissive" IETF RNIC.  For an IETF-only RNIC, the change has no effect: it can only interoperate with other IETF RNICs.  
 
The goal of the draft, then, is to provide a mechanism for ULPs or other entities (such as the OpenGroup's IT API & RNIC PI) to create a common interface to RDMAC as well as IETF RNICs that enables interoperability between the RNICs where possible.
 
--jc

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Michael Krause
Sent: Monday, November 29, 2004 11:11 AM
To: [email protected]
Subject: RE: [rddp] draft-carrier-rddp-rnic-interop-00.txt


At 06:40 PM 11/23/2004, Jim Pinkerton wrote:




Feedback is appreciated on this draft. This was a result of an informal
group working over beers after the RDDP work group meeting gave us
direction to try to solve interoperability as an informative appendix to
the MPA specification. 

John and I tried to be very specific about the exact recommended text
changes for the various drafts, to hopefully expedite the review process
with something concrete to shoot at.


Initial comments.

Mike

(1) It is not clear to me where there is ambiguity within the specifications.  The RDMAP specification (draft-ietf-rddp-rdmap-02.txt) section 6.1 states that the RDMA version MUST be 01b.  The table in section 6.8 states that a remote operation error is generated for a RDMAP version error.  The RDMAC RDMAP specification contains the same language but states the version MUST be 00b.  As such, it clear what the version must be and the error that should be returned if there is a version mismatch which contradicts the assertions made in section 3 of the interop draft.

(2) The DDP specification (draft-ietf-rddp-ddp-03.txt) section 4.1 states the DV bits must be set to 1.  Section 7.2 states that an invalid version must be returned upon detecting a mismatch and section 7.1 states that upon error detection, DDP  must stop placing segments and return an error to the ULP.  In this case, it would be to surface an error to RDMA.  The RDMAC DDP specification contains the same language but states the version MUST be 00b.  Again, this contradicts the assertions made in section 3 of the interop draft.

(3) I agree that there are three cases of implementations that can exist.  However, it seems that the specifications are relatively clear on what the versions must be for each set of specifications.  It is also clear what the expected behavior should be as well, i.e. the generation / surfacing of an error.  As such, a RDMAC implementation would require the version to be 0 to be compliant while the IETF implementations would require the version to be 1.  The only way to achieve interoperability is to set the appropriate version number prior to a connection transition to RDMA / DDP mode.  

(4) A RDMAC-only implementation would not negotiate MPA since this is not part of the RDMAC specifications.  The draft implies that any RDMAC compliant implementation would be augmented to provide MPA negotiation.  I don't see how this can be required - at best, it is an option to implement and may or may not be present on a given endnode.  If this is not implemented, the two sides will simply fail the RDMA stream establishment.  In general, the tables showing the negotiation at the MPA level look reasonable.

(5) It is not clear that new verbs extensions are required for interoperability.  The verbs specification section 9.3 already states how a version mismatch (DDP or RDMA) should be surfaced.  There is value in being able to configure a RNIC (on a per session basis) as to what version to support.  This should be done once and cause all layers to use a consistent version for a given QP.  The value should be set at QP creation and then updated upon session establishment if the RNIC can support both RDMAC and IETF specifications.  I also see value in having the query what the current version is.  I don't see why a ModifyQP is required as one knows up front what is desired and the implementation if permissive, would negotiate accordingly.  Therefore all that is needed is to understand the result.

_______________________________________________
rddp mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rddp