| Newsgroups |
gmane.ietf.rddp |
| Message-ID |
<[email protected]> |
John,
"Single shot" automatic invalidation of STags wouldn't break other things in addition to breaking apps written for Infiniband. For inster it would break iSER and modifying iSER to allow it would lead to very inefficient undesireable operation characterists. One needs to be able to have one side set up a large buffer for a transfer and allow the other side to use smaller buffers to handle that transaction a piece at a time as happens when a SCSI initiator sets up a buffer for a large Read and the SCSI target uses multiple RDMA Writes to transfer the data.
In iSER we did put in a specific requirement that iSER check that the Send carrying the Status for a command also invalidated the STag for that command and doing the invalidation if the Send didn't.
It would probably be appropriate to put a SHOULD statement into RDMAP and/or DDP about the ULP ensuring that an STag was invalidated for remote access before it tells the application that the data is in the buffer. This would say that other ULP's should do something similar to what we put into iSER. Given that "single shot" would be undesireable, there is no way for DDP or RDMAP to enforce the invalidation because they don't know when the ULP operation has been completed. The requirement for ULPs should be stated. If one can put a MUST on a ULP, then it could be a MUST.
Regards,
Pat
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of John Hufferd
Sent: Tuesday, June 08, 2004 11:06 AM
To: [email protected]
Subject: Re: [rddp] Long-lived Stags: Why?
My statements about InfiniBand were targeted at the exact point that Michael makes below. There is a lot of industry work going on to use iSCSI and the infrastructure around it on InfiniBand. That work is focused on taking the iSCSI/iSER and having it operate on InfiniBand, so that we do not have to reinvent all the management and discovery stuff on InfiniBand for use by SRP (especially since iSCSI/iSER has more capabilities than SRP).
SRP is NOT an "Official" InfiniBand Architecture, but is an IEEE thing and the folks that have defined the InfiniBand Spec only refer to SRP as an "Example" of a storage protocol on InfiniBand.
With respect to the point here however, the reason that I was concerned about the InfiniBand was that most of the RDMA/iWARP operations can be directly mapped against InfiniBand, and therefore we only need to have single versions of applications that can operate on both RDMA/iWARP and InfiniBand. However, I was worried that if this "Single Shot" discussion got going such that the RNIC was expected to automatically invalidate an STag when ever it received or extracted data, significant incompatibilities would exist between RDMA/iWARP and InfiniBand which have otherwise been resolved.
If we are moving away from any such consideration for the RNIC, then this whole issue of RDMA/iWARP can be put aside and handled in what ever other venue's that are approprate.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: [email protected]
Michael Krause <[email protected]>
Sent by: [email protected]
06/08/2004 07:06 AM
To
[email protected]
cc
Subject
Re: [rddp] Long-lived Stags: Why?
At 09:37 PM 6/7/2004, [email protected] wrote:
>Dwight,
>
> > This compromize includes two components, one a successful spoofing of
> > the transport layer, in which RDDP is no more or less vulernable (same
> > transports) and a second component which is the demarcation point at
> > which the ULP can be assured that the buffer is no longer available for
> > modification by a remote peer. I believe that iSER has correctly
> > addressed this issue (see John Hufferd's clarification). The Stag will
> > be un-allocated before notification/completion of the SCSI command.
>
>iSER is one way to design/implement iSCSI over RDMA - it is not
>the only one, and my posts were based on a different architectural
>approach. Nonetheless, the fact that iSER considers STag reuse to
>be sufficiently dangerous that it applies several "MUST" requirements
>to STag invalidation ought to be (IMHO) a visible nail in the coffin
>of long-lived STags.
>
>Would someone remind us why long-lived STag support should be retained
>given the security pain and suffering it's causing (e.g., what is the
>compelling use for this functionality)? The simplest solution here
>is to take it out, especially if there's no compelling use for it ...
>
>Looking back over the list emails, the reasons I think I see for
>long-lived STags are:
>
>(1) Allow storage protocols to stage transfer based on available
> resources. (John Hufferd) Also applies to HPC protocols like
> MPI. (Jim Pinkerton).
>(2) Compatibility with InfiniBand. (John Hufferd)
>(3) Avoid limiting flexibility. (Caitlin Bestler)
>(4) Allow applications to innovate around long-lived STags.
> (Jim Pinkerton)
Add my name and I'm sure many others who have spoken up about this issue to
all of these values.
>I believe that staged transfer (1) is compatible with a suitable
>definition of "one-shot" STags - after completion of data transfer,
>the buffer becomes not writable from the network before completion
>is reported to the ULP application (there are details to be worked
>out in the definition of "one-shot" that may impact the protocols).
>
>I have a hard time with InfiniBand compatibility as a motivation
>given the serious incompatibility between iSER and SRP (the native
>SCSI encapsulation on InfiniBand).
There is an effort within an open source group to provide iSCSI and iSCSI /
iSER over InfiniBand. These are already well underway.
As for compatibility with InfiniBand, need to keep in mind that much of the
existing ULP that use RDMA as well as many user applications already
operate over InfiniBand. Various OSV already have a solid RDMA
infrastructure capable of supporting InfiniBand. Many ISV have products
already using InfiniBand. In general, there is a reasonably sized and
solid ecosystem that supports the InfiniBand operational semantics which
were used as the basis for RDDP semantics. The basic RDDP semantics have
been incorporated back into InfiniBand as well enabling the industry to
have a single set that semantics that are interconnect independent. As
such, it would be very bad for the industry and customers to state
compatibility with InfiniBand (an interconnect that is recognized and has
associated IETF specifications) as insufficient motivation based solely on
SRP / iSER compatibility.
>Flexibility and innovation in the absence of other compelling uses of long
>lived STag functionality strike me as reasons to put the RDDP protocols
>(or possibly the long-lived STag extensions to them) on the experimental
>RFC track rather than the standards track (i.e., publish them as
>Experimental RFCs rather than Proposed Standard) until the compelling
>benefits that outweigh the security risks can
>be demonstrated.
This is just coming down to whether to mandate IPSec or not and proceed
with the existing specifications through the standards track. I don't
think we are to the point of just tossing the specs to experimental over
this one question.
Mike
_______________________________________________
rddp mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rddp