Re: Long-lived Stags: Why?

John Hufferd said:
> Just one caution here, I am not sure you really meant "Receive with
> invalidate" but I think that might be problematical with Shared Receive
> Queue (SRQ).
>

Yes it would require that a "Receive Operation" be posted to the QP
that had the SRQ, but the Receive Operation would only provide the
authorization to receive and indicate the desire to invalidate -- it
would not supply an untagged buffer. That would still come from the SRQ.

Alternately, this "QP-with-SRQ receive operation" could actually be
a queue for a shim layer, which would simply match completions
based upon the QP and perform any required invalidation. It could
also ex-post-facto notice excessive completions and trigger an
automatic response -- dealing with the already idenfitifed security
vulnerability for shared resource queues.

> Also I think I agree that unless the HW or OS enforces an invalidate after
> each simple operation, which I strongly disagree with since it would not
> match many "transaction" models, I do not see how the HW or the OS, can
> enforce the invalidation after each "transaction".  And if neither the HW
> or the OS can enforce it, I am not sure what the words should say, and how
> compliance could be certified.  Further, I am not certain that I
> understand the full purpose of the words, in that case, what ever they
> are.
> .

A "transaction" is not a transaction until the Data Sink reaps a completion.
All time between completions is an area of quantum uncertainty. The
application should not be processing the associated tagged buffer
until its availability has been communicated through a ULP speciific
mechanism that *must* involve a completion over the same DDP stream.

The hardware cannot enforce such rules. Like ULP flow control, it can
only enable compliant ULPs to do so easily.

What is proposed are improvements to the local interface which would
make proper usage of these tools the default, so the applications would
inadvertantly leave themselves vulnerable.

The key is that the Data Sink ULP will not apply its own validation
to the buffer until *after* the STag exposure for it has been eliminated.
Now obviously, hardware will not run around trying to ensure that a
different STag has exposed the same buffer any more than C run-time
libraries check to make sure that a destination buffer isn't the stack
location that marks the return from the call.

Applications that insist on doing very stupid things can already kill
themselves in numerous ways, this is not a *new* vulnerability.

But neglecting to check that your peer invalidated an STag in its
response, or even leaving it open to use "next time" is something
a developer new to RDMA could easily do without understanding
the consequences. What is suggested is that the local interface
MUST NOT/SHOULD NOT make such practices the default.

Appications that wish to post the kernel stack as the target
of an RDMA Read cannot expect the RNIC to stop them.