Re: Re: Invalidation

I agree with Mallikarjun's comments below.

I also agree with Caitlin's earlier comments.  Leaving the invalidation
function in the RDMAP layer allows the DDP layer to perform placement and
gives the RDMAP layer flexibility regarding how the invalidation function
is implemented.  The current layering, even allows the DDP layer to be
implemented on an adapter and the RDMAP layer on the host OS. Though that
type of implementation may be inefficient (compared to others), it may be
easier to implement and is possible with the current layering.

Thanks,

Renato J Recio
Chief Architect, eServer I/O
IBM Distinguished Engineer
Member IBM Academy of Technology
Tel 512-838-3685, T/L 678-3685



                                                                                                                            
                      "Mallikarjun C."                                                                                      
                      <[email protected]>        To:       <[email protected]>, <[email protected]>                             
                      Sent by:                 cc:                                                                          
                      [email protected]        Subject:  Re: [rddp] Re: Invalidation                                        
                      rg                                                                                                    
                                                                                                                            
                                                                                                                            
                      10/03/2003 08:45                                                                                      
                      PM                                                                                                    
                                                                                                                            
                                                                                                                            




Since I'm one of the few who advocated this feature
for the benefit of ULPs (such as iSER), let me add my
comments.

> is invalidation sufficiently ULP (wrt DDP, e.g., RDMAP)-specific

Yes, I believe so.  Ultimately, it is the ULP that decides how
long remote access for an STag (app buffer) should be allowed
based on application considerations such as an "application transaction"
(an example "transaction" for iSCSI/iSER is the exchange of
cmd-data-status for one I/O), and the risk in not invalidating.

>that
> it needs to be left to DDP's ULPs to implement and customize in their
> own fashion?
>
> If yes, the next question will be why it belongs in the RDMAP layer,
> as opposed to RDMAP providing local access to the DDP interface for
> STag invalidation, and letting ULPs figure out how to
> communicate STags when remote invalidation is in order.

Today, the RDMAP spec only provides the semantics and wire protocol
specifics around invalidation.  It is left to the ULPs (of RDMAP) to
figure out how to use the protocol feature to achieve the desired
results - again they know best when to disallow access to the underlying
buffers.

Now, a natural follow-up question might be why cannot the ULPs
do the invalidation themselves.  It appeared to us that it's more efficient
to have the auto-invalidation take place in the RDMAP layer, so the
ULP can simply sanity-check the invalidated STag value post-fact (that
could
be part of the completion message delivering the Send Message payload),
rather than having to turn around right away and ask the RDMAP/DDP
layers to do the invalidation.  IOW, one round-trip through the RDMAP/DDP
layers (right after processing an inbound Send Message) is eliminated
by the auto-invalidation.  In a hardware RNIC implementation where
the ULP-RDMAP boundary is also the software-hardware boundary,
the savings resulting out of eliminating this round trip are much more
substantial.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668
Roseville CA 95747
[email protected]

----- Original Message -----
From: <[email protected]>
To: <[email protected]>
Sent: Friday, October 03, 2003 5:07 PM
Subject: RE: [rddp] Re: Invalidation


> > From reading your email, it sounds like you are thinking there is a
> > layering violation? Is that your point?
>
> No.  It appears that invalidation is a rather important mechanism
> for actually using STags, and so, aside from historical reasons, what's
> the rationale for it being an RDMAP operation rather than a DDP
operation?
>
> Another way to ask this is: does absence of STag invalidation from DDP
> significantly reduce the usefulness of DDP without RDMAP?  Alternatively,
> is invalidation sufficiently ULP (wrt DDP, e.g., RDMAP)-specific that
> it needs to be left to DDP's ULPs to implement and customize in their
> own fashion?
>
> If yes, the next question will be why it belongs in the RDMAP layer,
> as opposed to RDMAP providing local access to the DDP interface for
> STag invalidation, and letting ULPs figure out how to
> communicate STags when remote invalidation is in order.
>
> I don't have a preconceived notion of the answers, but there's some
> design rationale that needs to be brought out and discussed here.
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> [email protected]        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
>
> _______________________________________________
> rddp mailing list
> [email protected]
> https://www1.ietf.org/mailman/listinfo/rddp
>


_______________________________________________
rddp mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rddp