Re: Changes to rfc5661sesqui suggested by errata 2006.

----- Original Message -----
> From: "Dave Noveck" <[email protected]>
> To: "Magnus Westerlund" <[email protected]>, "Chuck Lever" <[email protected]>
> Cc: "NFSv4" <[email protected]>
> Sent: Wednesday, September 18, 2019 7:36:14 PM
> Subject: [nfsv4] Changes to rfc5661sesqui suggested by errata 2006.

> I've been looking at the changes it makes sense to make to rfc5611sesqui to
> adapt to the changes suggested by errata 2006.  You can find the
> actual changed text below but first let me summarize the changes.
> 
> The largest change consists of a rewritten section 15.1,1.3.  This include
> the changes suggested by errata 2006 but also has the following additional
> changes:
> 
>   - It adds to the list of possible reasons to return NFS4ERR_DELAY an
>   example (session migration described in section 11.13.3) since none of the
>   existing examples covers the case of returning NFS4ERR_DELAY on a SEQUENCE
>   operation.
>   - It deals with the possible effect of NFS4ERR_DELAY on replies fetched
>   from the replay cache, including issues related to non-idempotent requests.
>   - It explicitly deals with requests that do not include a SEQUENCE
>   operation.
> 
> 
> In addition, there are some limited changes:
> 
>   - An additional bullet item is added to the list of things that stiil
>   need to be done in rfc5661bis (in Section 1.1), covering the erratas other
>   than 2006.
>   - A reference to section 15.1.1.3 is added to Section 11.12.2.
>   - A number of references to Section 15.1.1.3 are added to Section
>   11.12.3.
>   - A description of the changes made to Section 15.1.1.3 is added to
>   Appendix B.3.
> 
> If people are OK with these changes, i can submit a srfc5661sesqui-nsns-02
> whenever it is
> appropriate.
> 
> *Replaced Section 15.1.1.3 <http://15.1.1.3>:*
> 
> 15.1.1.3.  NFS4ERR_DELAY (Error Code 10008)
> 
>   For any of a number of reasons, the replier could not process this
>   operation in what was deemed a reasonable time.  The client should
>   wait and then try the request with a new slot and sequence value.
> 
>   Some examples of scenarios that might lead to this situation:
> 
>   o  A server that supports hierarchical storage receives a request to
>      process a file that had been migrated.

We should not forget, that there is a pNFS usecase as well. Our server will return
NFS4ERR_LAYOUTTRYLATER in such situation. May be a sentance should be added to
point that some operations, namely LAYOUTGET, may return an alternative error code.

Tigran.


> 
>   o  An operation requires a delegation recall to proceed, so that the
>      need to wait for for this delegation to be recalled makes
>      processing this request in a timely fashion impossible.
> 
>   o  A request is being performed on a session being migrated from
>      another server as described in Section 11.13.3, and the lack of
>      full information about the state of the session on the source
>      makes it impossible to process the request immediately
> 
>   In such cases, returning the error NFS4ERR_DELAY allows necessary
>   preparatory operations to proceed without holding up requester
>   resources such as a session slot.  After delaying for period of time,
>   the client can then re-send the operation in question, often as part
>   of a nearly identical request.  Because of the need to avoid spurious
>   reissues of non-idempotent operations and to avoid acting in response
>   to NFS4ERR_DELAY errors returned on responses returned from the
>   replier's replay cache, integration with the session-provided replay
>   cache is necessary.  There are a number of cases to deal with, each
>   of which requires different sorts of handling by the requester and
>   replier:
> 
>   o  If NFS4ERR_DELAY is returned on a SEQUENCE operation, the request
>      is retried in full with the SEQUENCE operation containing the same
>      slot and sequece values.  In this case, the replier MUST avoid
>      returning a response containing NFS4ERR_DELAY as the response to
>      SEQUENCE solely on the basis of its presence in the replay cache.
>      If the replier did this, the retries would not be effective as
>      there would be no opportunity for the replier to see whether the
>      condition that generated the NFS4ERR_DELAY had been rectified
>      during the interim between the original request and the retry.
> 
>   o  If NFS4ERR_DELAY is returned on a operation other than SEQUENCE
>      which validly appears as the first operatoion of a request,
>      handling is similar.  The request can be retired in full without
>      modification.  In this case as well, the replier MUST avoid
>      returning a response containing NFS4ERR_DELAY as the response to
>      an intial operation of a request solely on the basis of its
>      presence in the replay cache.  If the replier did this, the
>      retries would not be effective as there would be no opportunity
>      for the replier to see whether condition that generated the
>      NFS4ERR_DELAY had been rectified during the interim between the
>      original request and the retry.
> 
>   o  If NFS4ERR_DELAY is returned on an operation other than the first
>      in the request, the request when retried MUST contain a SEQUENCE
>      operation which is different than the original one, with either
>      the bin id or the sequence value different from that in the
>      original request.  Because requesters do this, there is no need
>      for the replier to take special care to avoid returing an
>      NFS4ERR_DELAY error, obtained from the replay cache.  When no non-
>      idempotent operations have been processed before the NFS4ERR_DELAY
>      was returned, the requester should retry the request in full, with
>      the only difference from the original request being the
>      modfication to the slot id or sequence value in the reissued
>      SEQUENCE operation.
> 
>   o  When NFS4ERR_DELAY is returned on an operation other than the
>      first within a request and there has been a non-idempotent
>      operation processed before the NFS4ERR_DELAY was returned, the
>      reissued request should avoid the non-idempotent operation.  The
>      request still must use a SEQUENCE operation with either a
>      different slot id or sequence value from the SEQUENCE in the
>      original request.  Because this is done, there is no way the
>      replier could avoid spuriously re-eecuting the non-idempotent
>      operation since the different SEQUENCE parameters prevent the
>      requester from recognizing that the non-idempotent operation is
>      being retried.
> 
>   Note that without the ability to return NFS4ERR_DELAY and the
>   requester's willingness to re-send when receiving it, deadlock might
>   result.  For example, if a recall is done, and if the delegation
>   return or operations preparatory to delegation return are held up by
>   other operations that need the delegation to be returned, session
>   slots might not be available.  The result could be deadlock.
> 
> *Addition to Section 1.1:*
> 
> The following is to be added as the third bullet in the list in this
> section:
> 
>   o  Work would have to be done to address many erratas relevant to RFC
>      5661, other than errata 2006 which is addressed in this document.
>      That errata was not deferrable because of the interaction of the
>      changes suggested in that errata and handling of state and session
>      migration.  The erratas that have been deferred include changes
>      originally suggested by a p[articular errata, which change
>      consensus decisions made in RFC5661, which need to be changed to
>      ensure compatibility with existing implementations that do not
>      follow the handling delineated in RFC 5661.  Note that it is
>      expected that such erratas will remain relevant to implementers
>      and the authors of an eventual rfc5661bis, despite the fact that
>      this document, when approved, will obsolete RFC 5661
> 
> *Modifications in Section 11.12.2:*
> 
> The fifth non-bulleted paragraph is modified to add a new last sentence and
> now reads as follows:
> 
>      When transferring state between the source and destination, the
>   issues discussed in Section 7.2 of [63] must still be attended to.
>   In this case, the use of NFS4ERR_DELAY may still necessary in
>   NFSv4.1, as it was in NFSv4.0, to prevent locking state changing
>   while it is being transferred.  See Section 15.1.1.3 for information
>   about appropriate client retry approaches inn the event that
>   NFS4ERR_DELAY is returned.
> 
> *Modifications in Section 11.12.3:*
> 
> The sixth non-bulleted paragraph is modified to add a new last sentence and
> now reads as follows:
> 
>      An important issue is that the specification needs to take note of
>   all potential COMPOUNDs, even if they might be unlikely in practice.
>   For example, a COMPOUND is allowed to access multiple file systems
>   and might perform non-idempotent operations in some of them before
>   accessing a file system being migrated.  Also, a COMPOUND may return
>   considerable data in the response, before being rejected with
>      NFS4ERR_DELAY or NFS4ERR_MOVED, and may in addition be marked as
>   sa_cachethis.  However, note that if the client and server adhere to
>   rules in Section 15.1.1.3, there is no possibility of non-idempotent
>   operations being spuriouly reissued after receiving NFS4ERR_DELAY
>   response.
> 
> The seventh non-bulleted paragraph is modified to read as it appear below.
> Added
> material is italicized:
> 
>   Because of the considerations mentioned above
> *including the rules for   the handling of NFS4ERR_DELAY included in
> Section 15.1.1.3*, the
>   destination server can respond appropriately to SEQUENCE operations
>   received from the client by adopting the three policies listed below:
> 
> *Addition to Appendex B.3:*
> 
> The following new bulleted paragraph needs to be added at the end of
> Appendix B.3:
> 
>   o  Because of the need to provide the clarifications in errata 2006
>      and to adapt these to properly explain the interaction of
>      NFS4ERR_DELAY with the replay cache, a revised description of
>      NFS4ERR_DELAY appears in Section 15.1.1.3.  This errata, unlike
>      manay other RFC5661 erratas is addressed in this document because
>      of the extensive use of NFS4ERR_DELAY in connection with state
>      migration and session migration.
> 
> _______________________________________________
> nfsv4 mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4