Re: NFS links die with garbage files.

On Sat, Apr 4, 2026 at 12:44 PM Zaphod Beeblebrox <[email protected]> wrote:
>
> For this discussion, I have two machines.  The server is running 15.0p4 and is serving nfs to a couple dozen machines. The client is 15.0p5 and mounting a few dozen mounts.  They also run NFSv3 as I have never gotten around to implementing NIS for my home network.  All filesystems are ZFS.
>
> TLDR;
> With very light use, one particular mount watched by rtorrent for new files and filled by rtorrent with incoming files (that are not, in general, read in situ) ... fails with one element in the NFS root visible, but not really.  Shows up in ls, but fails "no such file or directory" for stat.
>
> All of my NFS filesystems are put in /d/<machine>/<share> naming scheme and soft links pick that up and put them where they go in the local filesystem (if any).  In this case, the failing filesystem is /d/vr/torrent/in ... which has a subdirectory watch.
>
> Watch is filled by moving files into it on the client.  This is noticed by rtorrent which runs on the server (kqueue directory watch) and rtorrent then fills in the files in the root NFS directory with the torrent data.  When a torrent is finished, it is moved by rtorrent to another ZFS and NFS filesystem (all local to the server).
>
> umount -f on /d/vr/torrent/in followed by a re-mount fixes the problem.
Since an unmount/mount fixes the problem, that hints that it is a
caching problem.

>
> Since this happens over days, I'm at a bit of a loss as to how to proceed with diagnosis.
Since the above doesn't really give me any details w.r.t. exactly what
is happening
and in what order, I'll just make a few generic comments..

#1 - The NFS client is not (and will never be, given the protocol) a
POSIX compliant
        file system. Various tricks are used to make it appear POSIX-like.
#2 - For performance reasons, file attributes and data blocks
(including directory
        blocks) are cached in the client. Since there is no cache
coherency protocol
        in NFS, the caches can easily be "out of date".
        How out of date?
        Well, there are a bunch of mount options (see "man mount_nfs")
that adjust
        how long caches are considered valid (ie, it is time based).
The attributes
        mtime (and in some cases ctime) then decide if the cached data
is still valid.

Caching is usually managed alright when only one client is manipulating an
exported subtree, but as soon as more than one client is accessing a subtree
concurrently, all bets are off.

If multiple changes occur within the same clock tick (such that mtime/ctime
does not change for each of the changes), then a change can get completely
missed by other clients. (NFSv4 has a change attribute that is supposed to
change when any data/metadata change is made to a file, but that is not in
NFSv3.)

You mention things like "soft links" and I'll admit I have no idea what that
means. Once a file is looked up in one location it is represented by a file
handle which has nothing to do with it's name/name's location.
Also, the mapping from name->file handle will be cached. If another
client moves/changes the name, this cached entry will not be recognized
until sometime later (the time based caching...).

All I can suggest is trying the various mount options related to cache
timing (for example, the ac* ones, nametimeo, negnametimeo.).

It may just be that NFS does not do what you want, because it
only approximates POSIX.

A final note, I do not do any work on NFSv3 these days, so nothing
has changed w.r.t. NFSv3 in years.

rick