Re: Invalidation

On Monday, September 29, 2003, at 02:12 PM, [email protected] wrote:

> (2) Invalidation.  We have a lingering issue from way back
> 	when that RDMAP is handling invalidation of STags,
> 	which are DDP level resources.  This needs some
> 	serious examination in this forum.

I'm not sure I remember anything that qualifies as an
"issue". There are certainly some quirks that relate to
the fact that the invalidation is inherently asynchronous
because it is processed on the "completion track" and at
least theoretically by a different layer (and hence potentially
as well).

Quirk #1: Invalidation is not a fence. "Later" DDP Segments
may be placed prior to the invalidation -- even if received
in order. Invalidation offers no guarantees to the sender,
only to the receiver. And the receiver is only guaranteed
that there will be no further placements *when the notice
of the invalidation is delivered*. The only guarantee about
what was placed at that point is that *earlier* DDP segments
will have been placed. There is no guarantee that *later*
DDP segments were *not* placed.

Quirk #2: Invalidation may or may not disrupt an in-process
RDMA Read. That is, there is no specification on *when* an
RDMA Read engine translates STags.



The last issue is that the current wording does not allow
a local implementation to suppress remote invalidation on
specific tags. This essentially prevents exporting STags
that are associated with permanent data structures such
as Memory Regions (as opposed to windows). That should be
fixed.

In general, remote invalidation should not be viewed as
providing any guarantees to the remote side. It is a
service to the local ULP. While it ought to remain
mandatory to implement, the local ULP should have
the option of not using it for a specific STag.