Security Draft Comments

Overall, the security draft is coming together with no real major issues to 
resolve from what I've reviewed to date thus we should be able to get it 
cleaned up in a relatively short period of time.

Mike


(1) Draft might benefit from a section that describes current RDMA / DDP 
techniques that limit exposures.  For example, a description of the basic 
error semantics and how these lead to hard errors and connection 
shutdown.  While some of this is implied or spread out to varying extents 
within the draft specification, consolidation into a single section would 
simplify the subsequent discussion as well as add clarity.  Applicable 
error conditions to document: protection violation, STag violation, base / 
bounds violation, etc.  In addition, a brief description of window vs. 
region and how the fine grain exposure of a window has benefits to the ULP 
in constraining potential memory to target.

(2) Unclear if there is a need for a glossary but some specification terms 
are repeated here while others are not.  May be simpler to just reference 
the specs with the agreed to terminology and definitions rather than 
redefine in potentially imprecise terms.

(3) Figure 1.  One might consider having a line between the admin and the 
privileged application given the resource manager is not required but its 
essence may be integrated into the privileged application.  For example, a 
storage subsystem may already have defined interfaces to administration 
services and if it has exclusive control of the RNIC would not require a 
separate resource manager.

(4) The draft often repeats information making the flow / structure 
somewhat awkward.  It would be better to have a given interface, 
definition, etc. defined once with all relevant material than to spread the 
material out and thus repeat it throughout the document.

(5) Section 4 is a bit rough in its flow / structure.  Subsections could be 
simplified into a somewhat standard structure.  This builds upon the 
previous comment.  Perhaps each component could be:
	- Defined
	- List of responsibilities as they relate to RDMA / DDP
	- List of resources and what is local and what is visible from the remote
	- Is the resource shared or not
	- How each component is initialized
	- List of potential issues w.r.t. security exposures / attacks / risks, 
i.e. threats
	- List of techniques to eliminate / reduce security threats

This would provide more focus on each component given the material is 
consolidated.  I tend to prefer such an approach because it helps bring 
together all of the relevant information required to define and protect a 
resource making it more likely developers will get it right.

(6) Request proxy interface definition in 4.1 really does not provide much 
information.  It provides a high-level statement on counter measures but 
due to organization, such a statement does not have much in terms of value.

(7) 4.2.4 Is there any need to discuss the 8-bit key portion of the STag 
and how an application / ULP can help control the wire-view of the STag?

(8) 4.28 There is "Event Queue" and one other line but not really anything 
else stated here.  This seems almost like a fragment or a TBD area.

(9) 4.3 Is 4.3 just a place holder?  Not clear its relevance to the draft 
as many system properties can be attacked but these are outside the scope 
of RDMA itself.

(10) 5.0 The pros / cons of sharing resources should be focused on security 
issues.  Any application / subsystem  that does not share obviously can be 
limited or denied requested resources - this is a standard problem for an 
OS / policy manager to resolve.  It is simpler to just state that the 
system operates in one of two modes:
	- Use of a resource policy manager that dynamically determines what, if 
any, resources are provided and the amount of resources.  How dynamic 
resource management is an implementation-specific problem and outside the 
scope of the draft specification.

	- Fixed resource assignment to a given application.  The application knows 
what its limits are through whatever means (could be an embedded 
environment thus there is no administration interface).

Some of this is covered in various areas but consolidating this all into 
one section focused on "to share or share not, that is the question?" would 
simplify the document.

(11) It would be good to have consistent usage of "privileged" when 
describing the resource manager.

(12) Page 15 under NS-RT. There is a note attached but not really 
sufficient information provided w.r.t. the key-based authentication per 
packet sentence.  Either provide more information here or add a pointer to 
the appropriate specification or subsequent section where this is described 
further.

(13) 7.1.1.   Add "each" to , it is recommend that Protection Domain be 
limited to the number....

(14) 7.1.2 Second bullet should use the term window or cross reference to 
the appropriate specification so that the reader can understand how this is 
somewhat a local issue and that there are tools defined to help provide 
this service.

(15) 7.1.2 fourth bullet,  STag needs to be properly capitalized.  It would 
also be good to cross reference back to the error description (per previous 
comment) the effects of a STag violation.

(16) 7.1.5 Need to emphasize that this is an application problem not a RDMA 
or DDP problem.

(17) 7.2  My reaction was "so".  Not clear why this section is there given 
the limited information provided.

(18) 7.2.1 Need to think about this more but it seems there could be nasty 
middle-box attacks which are outside the scope of RDMA and DDP.  It would 
also be good to separate authentication from integrity protection as the 
two provide different protections and the associated protections should be 
clearly attributed to each if you are going to list out the various attacks.

(19) 7.2.2 Repeats earlier information on RDMA Read attack.  The spec might 
benefit from simply having a set of ladder diagrams of operations for 
different sequences with a list of associated attack and how one provides 
countermeasures.

(20) 7.2.2. STag of zero is a local, not remote value and thus should not 
be part of the allocation algorithm.

(21) 7.3.2 This may be a bit of implementation but it may be possible that 
a RNIC could have multiple execution paths such that the invalidate 
executes in parallel or serial to other operations thus a subsequent RDMA 
operation could access memory prior to the invalidate being 
completed.  Same applies if the invalidate or access rights changes are 
implemented in software which is asynchronous the subsequent hardware 
packet processing operations.  This might lead to a race condition and thus 
exposure.

One question: What if the remote stated it would invalidate but does not do 
so?  It is clear that remote peers should not be trusted but one may appear 
to be fully compliant with the ULP needs and then simply not invalidate 
thus enabling the memory to be read / updated.  This can be dealt with via 
the implementation - a couple of simple techniques come to mind - but it 
may need to be discussed a bit further.

(22) 7.4.2 Zeroing of a data buffer or providing valid data with all 
non-used buffer memory being invalidated / zero should be noted.

(23) 7.5.2.2 Note that this is only for a shared CQ in the header though 
the subsequent header does make this more specific.

(24) 7.5.3 It is not clear to me that a RDMA MIB is required.  One can 
understand whether a LLP or a physical interface are making forward 
progress through the existing MIBs.  Providing additional statistics isn't 
really of value as one knows the connections / streams and layer 2 are 
making forward progress.  After that, one checks to see if the ULP is 
making forward progress and therefore we have a complete picture without 
requiring yet another MIB to implement / manage.

(25) Not sure if I agree with the note in 7.5.3 that indicates that there 
isn't risk for some resources.  For example, a ULP may negotiate use of 
large receive buffers and potentially large number of receive buffers and 
then not make forward progress.  These types of consumptions are problems 
for both local and remote attacks.

(26) 7.5.4 One could also see an attack where one purposely transmits out 
of order packets or delays the  packet retransmission quite often. This 
could force some implementations to take the slow path quite often thus 
causing a performance degradation attack.  This is an 
implementation-specific problem that is also applicable to non-RDMA network 
stacks so it isn't necessarily new but might be beneficial to note.

(27) 7.5.5 The last sentence says that the remote peer just can't use the 
STag but it can also cause a connection to be shut down on error thus while 
true, there is another potential consequence.

(28) 7.5.6 Should note the high and low-water marks per connection sharing 
a receive queue to detect and allow corrective action to take place.

(29) 7.6 Not sure if this applies but if the system supports dynamic 
patching of a subsystem, then one might replace the executable with a 
trojan horse thus enabling one to take over the system / resources.  Might 
be worthwhile to have a note that such patching solutions should have all 
RDMA / DDP resources released so that the LLP establishment process is 
required thereby providing some level of new authentication to occur to the 
previous remote peers.

(30) Most of sections 8 and 9 are works in progress so no additional 
comments made.