Re: NFSv4.2 server replies to Copy with length == 0

I guess the IETF would prefer an individual draft in the .xml format
accepted
by xml2rfc.    However, if you don't have the time to learn all that, I
think an email
saying what the spec should say and why would be good to move the process
forward by providing a focus for working group discussion.

Ultimately we would need a replacement document arrived at by editing
the .xml for rfc7862.   However, there isn't much point in doing that
until the working group decides that a significant change is needed.

On Mon, Oct 21, 2019 at 10:27 AM Rick Macklem <[email protected]> wrote:

> David Noveck wrote:
> >Tom Talpey wrote:
> > > The IETF process should define its path.
> >
> >I agree.   For us to do that, I suggest that Rick send out a proposal
> >for what the spec should say and why it should say that that the working
> >group can discuss.
> Ok, I will try. Not having done this before, what is the preferred format?
> (I have no experience with the ietf editing tools, but will take a stab at
> it.)
>
> Thanks, rick
>
> Any such proposal would need to mention that the proposed
> replacement text is compatible with all existing clients and
> that no existing client depends on the server obeying the current
> "MUST" regarding EINVAL.   However, those facts, while relevant,
> cannot be the focus of the proposal.
>
> Given the realities of the IETF process, it would take a while
> for any such proposal to effectively change the current spec
> language.   During that time. clients might reasonably be
> adapted to both the existing spec, and the new proposal, just as
> Rick is expecting to do with his client.
>
> With regard to servers developed before the change becomes
> effective implementers will have to decide whether to obey the
> existing "MUST".   Here I think the relevant fact is not that there are
> particular clients that do not depend on the server obeying the "MUST",
> but that it is hard to imagine any client depending on this "MUST",
> which is part of the RFC2119 definition of "MUST", making use of the
> word "MUST"  in this case dubious.  Perhaps an errata can be submitted to
> warn potential implemeners that this requirement is under active
> discussion.
> I am assuming the errata would still be visible to potential implementers,
> even if it is "HELD OVER", as I expect it to be.
>
> On Sun, Oct 20, 2019 at 3:21 PM Tom Talpey <[email protected]<mailto:
> [email protected]>> wrote:
> On 10/18/2019 10:37 PM, Rick Macklem wrote:
> > Oh, I should clarify that I think building a client that works with both
> the
> > Linux server and one that follows the RFC is possible and that is what I
> > am doing w.r.t. client.
> > - For example, it is clipping the length to file_size - offset, not
> doing a Copy
> >     on the wire when offset >= file_size and never putting a Copy with
> length
> >     equal 0 on the wire. (Not ideal, but seems to work ok.)
> >
> > It is the server that can only be Linux style or RFC style and that is
> where the
> > tunable to choose between them comes in. (It needs to be "Linux style" to
> > work well with a Linux server.)
>
> I applaud the effort, but I ask please don't deploy such a client. It
> will serve only to prolong the indecision of what the protocol should
> be. The IETF process should define its path.
>
> Tom.
>
> >
> > rick
> >
> >
> > ________________________________________
> > From: nfsv4 <[email protected]<mailto:[email protected]>> on
> behalf of Rick Macklem <[email protected]<mailto:[email protected]>>
> > Sent: Friday, October 18, 2019 7:27 PM
> > To: Tom Talpey; [email protected]<mailto:[email protected]>
> > Subject: Re: [nfsv4] NFSv4.2 server replies to Copy with length == 0
> >
> > First off, I hope you don't mind a top post.
> > Just to make sure it is clear, in my opinion, the discussion below is not
> > addressing the "real problem".
> >
> > The "real problem" is that, even if the Linux folks patched their code
> > tomorrow (obviously an optimistic bound;-), there are who knows how many
> > installed systems out there doing it the Linux way and working for their
> users
> > just fine.
> > --> Now, maybe the client coders can be clever enough to make their
> >        clients work both ways, but that isn't going to be trivial.
> >        - For example, if a client sends one of these Linux servers a
> Copy with
> >           length == 0, it simply replies with 0 bytes copied. Then if
> the client
> >           thinks the offset was < file_size, it could try again with
> >           length = INT_MAX and see if that works. (If it can figure out
> which works,
> >           then it can use that for that mount/server.)
> > But as you can see, it could get messy (and fail for cases where the
> file_size
> > does change, resulting in the client choosing the wrong method, etc).
> >
> > To me, the problem isn't whether or not Linux can be patched to conform
> > to the RFC, it is "what do you do to keep the installed base that does
> not
> > conform to the RFC working?".
> >
> > Right now we seem to have a Linux client which works fine with a Linux
> server
> > and works fine with other servers that don't yet support NFSv4.2.
> >
> > What will I do. Make my code interoperate with Linux and not just
> > Linux-future (or whatever it's called) because that's what most users
> > will care about. (There will probably be some tunable that can be enabled
> > to conform to the RFC, but I'll bet no one ever uses it.)
> >
> > So what can be done. Well, I'm a pragmatist, so I say "if there isn't an
> installed
> > base of NFSv4.2 implementations that conform to the RFC, then just
> redefine
> > the RFC to conform to the installed base, aka Linux". Anything else is
> going to
> > be messy and, unless the Linux folks buy into it, irrelevant imho.
> > (That's why I've asked other implementors to put their hands up and,
> sofar, there
> > hasn't been any hands up. I thought Ganesha was working on NFSv4.2, but
> > no one has posted on [email protected]<mailto:[email protected]> about it.)
> >
> > rick
> >
> > ________________________________________
> > From: nfsv4 <[email protected]<mailto:[email protected]>> on
> behalf of Tom Talpey <[email protected]<mailto:[email protected]>>
> > Sent: Friday, October 18, 2019 5:49 PM
> > To: [email protected]<mailto:[email protected]>
> > Subject: Re: [nfsv4] NFSv4.2 server replies to Copy with length == 0
> >
> > On 10/18/2019 4:50 PM, Olga Kornievskaia wrote:
> >> On Fri, Oct 18, 2019 at 4:26 PM Tom Talpey <[email protected]<mailto:
> [email protected]>> wrote:
> >>>
> >>> On 10/17/2019 5:34 PM, Rick Macklem wrote:
> >>>> Olga Kornievskaia wrote:
> >>>>> On Thu, Oct 17, 2019 at 12:20 PM Rick Macklem <[email protected]
> <mailto:[email protected]>> >wrote:
> >>>>>>
> >>>>>> Tom Talpey wrote:
> >>>>>>> On 10/17/2019 11:22 AM, J. Bruce Fields wrote:
> >>>>>>>> On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
> >>>>>>>>> I have now found two cases where the Linux NFSv4.2 server does
> not
> >>>>>>>>> conform to RFC-7862. One is as above and the other is a reply to
> Seek
> >>>>>>>>> of NFS4ERR_NXIO when the sa_offset argument == file_size
> (instead of
> >>>>>>>>> replying NFS_OK along with sr_eof == true).
> >>>>>>>>
> >>>>>>>> Huh.  Looks like that's documented behavior of Linux's seek.
> (See the
> >>>>>>>> ERRORS section of the lseek(2) man page.)  Looks like Solaris also
> >>>>>>>> returns -ENXIO in this case:
> >>>>>>>>
> >>>>>>>>
> https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
> >>>>>>>>
> >>>>>>>> And freebsd too:
> >>>>>>>>
> >>>>>>>>
> https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
> >>>>>>>>
> >>>>>>>> I wonder where that spec language came from?
> >>>>>>>
> >>>>>>> Those manpages look like ENXIO comes back only on sparse files.
> Perhaps
> >>>>>>> this is boilerplate from v4.0 before this kind of thing was common.
> >>>>>> As far as I know, a non-sparse file (no holes) behaves the same as
> a file with
> >>>>>> holes in it. (I'm not sure if the POSIX draft is clear for the case
> where the
> >>>>>> file system does not support holes, but I think most
> implementations handle
> >>>>>> that the same way as a file with no holes on a file system that
> supports holes.)
> >>>>>>
> >>>>>>> This should at least be discussed on [email protected]...
> >>>>>> I think there needs to be a discussion on how to best deal with
> cases where
> >>>>>> implementations have shipped to users with these glitches in them.
> >>>>>>
> >>>>>> I think it might be better to document them, so that client coders
> can
> >>>>>> implement work arounds (I am doing so for the three cases I've
> found)
> >>>>>> instead of the server being patched to change its behaviour, causing
> >>>>>> potentially more interoperability issues.
> >>>>>> (That's why I re-posted this to [email protected]<mailto:
> [email protected]>.)
> >>>>>>
> >>>>>>>> Our NFS server could translate an -ENXIO return into 0 and sr_eof
> ==
> >>>>>>>> true easily enough, assuming -ENXIO is really only ever returned
> in that
> >>>>>>>> case.
> >>>>>>>>
> >>>>>>>> I haven't tested, but from a quick check of the Linux client code
> I
> >>>>>>>> think that would require a matching fix on the client side to
> translate
> >>>>>>>> sr_eof == 0 *back* to ENXIO.
> >>>>>>>>
> >>>>>>>> I don't know if it's worth it.
> >>>>>>>
> >>>>>>> What Bad Thing would happen for the difference?
> >>>>>> Well, for a POSIX draft style client, the only effect is that a
> client may be
> >>>>>> broken (not handle the sr_eof == true reply correctly) without the
> >>>>>> implementors knowing that, since they tested against the Linux
> server.
> >>>>>> (I believe this is the current status of the Linux client.)
> >>>>>>
> >>>>>> For a server implementor (I get to wear both hats;-), the server
> can only
> >>>>>> generate one reply or the other. I've implemented both with a
> tunable
> >>>>>> that can be used to flip between them. I default to the NFS4ERR_NXIO
> >>>>>> reply, since that is what the Linux client expects.
> >>>>>>
> >>>>>> As for the Copy issues, the client can easily handle them, but the
> client
> >>>>>> coder needs to know about them.
> >>>>>> At this time the FreeBSD server code only implements what is in the
> RFC.
> >>>>>> This seems sufficient for the testing I've done sofar.
> >>>>>> The case that I think will break would be Linux code that does a
> >>>>>> copy_file_range(infd, NULL, outfd, NULL, INT_MAX, 0) to copy an
> entire
> >>>>>> file. I'll test this case to-day, but I'm think it will just get a
> NFS4ERR_INVAL
> >>>>>> reply from the FreeBSD server.
> >>>>>
> >>>>> this might be problematic for the linux client because though there
> >>>>> isn't an official use of it but the libc cp might implement this
> doing
> >>>>> exactly what you say here copy_file_range(infd, NULL, outfd, NULL,
> >>>>> INT_MAX, 0). It doesn't try to determine the size of the file before
> >>>>> the copy and
> >>>>> just copy max bytes assuming that it'll get a short read at some
> point.
> >>>> I just tested this and the Linux client does send a Copy with a
> >>>> ca_count == INT_MAX.
> >>>>
> >>>> Truth be told, it worked fine, because my server code neglected to do
> the
> >>>> required ca_src_offset + ca_count > file_size --> NFS4ERR_INVAL.
> >>>> I'm now adding the check, but it will only be enabled by a tunable not
> >>>> set by default.
> >>>>
> >>>> I am going to post on [email protected]<mailto:[email protected]>, to see
> if we can just "clarify" the RFC
> >>>> to do what Linux already does.
> >>>
> >>> Technically speaking, this is not Linux behavior but libc's. Of course,
> >>> this may be a moot distinction, because changing applications can be
> >>> even harder than changing kernels. But for IETF discussion purposes
> >>> I don't think it's relevant to lump it into "Linux behavior".
> >>>
> >>> Question though. Could the Linux kernel client recognize this type of
> >>> API request, and morph it into a valid NFS wire operation? Such "torque
> >>> conversion" is basically the client's job, after all.
> >>
> >> I believe the reason the (userland) client would do a copy of INT_MAX
> >> bytes is because it want to avoid trying to query the size of the file
> >> then copying only that amount of bytes since the file size could have
> >> changed between the query and the copy. Same reason even in the kernel
> >> we'd rather not query and then copy.
> >
> > The application (which is not the same as the client) is simply
> > expressing its request to copy the entire file. That's perfectly
> > natural for a local API.
> >
> > The kernel client could just as easily translate this to a valid NFS
> > copy by pulling the size from its attribute cache, or sending a getattr
> > and using the result. After all, the copy operation is expensive and
> > long-running, it's basically no extra overhead.
> >
> > In other words, I don't buy the argument that this can't and shouldn't
> > be done.
> >
> >> While linux server doesn't implement count=0 copy and in theory it
> >> probably should, what the (nfs kernel) client can do if it receives a
> >> request from VFS to copy INT_MAX it can translate that to count=0
> >> copy.
> >>
> >> The problem we currently face of having two different ways of doing a
> >> copy: a traditional copy which if at some point requests beyond the
> >> end of the file it doesn't expect to fail and expects a short read vs
> >> doing something like copy_file_range() that uses the nfs copy_offload
> >> that if that were to read beyond the end of the input file and if the
> >> NFS spec reqs are enforced it would fail with EINVAL. That creates
> >> inconsistent results using different copy mechanisms.
> >
> > Ok, this is significant, the operations arguably should have similar
> > semantics. But, if there's a simple set of required processing that
> > the client isn't doing, which will erase the difference, why not do
> > those?
> >
> > Tom.
> >
> >>
> >>
> >>
> >>>
> >>> Tom.
> >>>
> >>>> rick
> >>>>
> >>>>> (I can make this work for the Linux client, but it will take another
> "cheat"
> >>>>>     enabled via a tunable.)
> >>>>> --> Part of the confusion here comes from the fact that the Linux
> syscall
> >>>>>           semantics for copy_file_range() has changed. (I also where
> the
> >>>>>           copy_file_range(2) hat for FreeBSD, so I've got some work
> to do there, too.)
> >>>>>
> >>>>> In summary, I think that, since Linux has been shipping this for
> some time,
> >>>>> documenting workarounds is a practical approach.
> >>>>>
> >>>>> rick
> >>>>>
> >>>>> _______________________________________________
> >>>>> nfsv4 mailing list
> >>>>> [email protected]<mailto:[email protected]>
> >>>>> https://www.ietf.org/mailman/listinfo/nfsv4
> >>>>
> >>>> _______________________________________________
> >>>> nfsv4 mailing list
> >>>> [email protected]<mailto:[email protected]>
> >>>> https://www.ietf.org/mailman/listinfo/nfsv4
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> nfsv4 mailing list
> >>> [email protected]<mailto:[email protected]>
> >>> https://www.ietf.org/mailman/listinfo/nfsv4
> >>
> >> _______________________________________________
> >> nfsv4 mailing list
> >> [email protected]<mailto:[email protected]>
> >> https://www.ietf.org/mailman/listinfo/nfsv4
> >>
> >>
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [email protected]<mailto:[email protected]>
> > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [email protected]<mailto:[email protected]>
> > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [email protected]<mailto:[email protected]>
> > https://www.ietf.org/mailman/listinfo/nfsv4
> >
> >
>
> _______________________________________________
> nfsv4 mailing list
> [email protected]<mailto:[email protected]>
> https://www.ietf.org/mailman/listinfo/nfsv4
>

_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4