Re: nfs server issues

On Sat, Jul 4, 2026 at 1:05 AM Richard Purdie
<[email protected]> wrote:
>
> On Fri, 2026-07-03 at 20:22 -0700, Rick Macklem wrote:
> > Another thing you can do that might give you a hint w.r.t. what is
> > going on is to repeatedly do..
> > # nfsstat -E -s
> > - and look to see what operations are being done frequently.
> > (If you see lots of FreeStateID's happening, I think that is a known
> > bug in the Linux client that was only fixed fairly recently, for
> > example.)
>
> I've pasted the current numbers after the system reset below. What
> would be considered "lots"? :)
>
> Unfortunately we're going to struggle to change the linux kernels
> connecting to this as those differing versions are part of our test
> setup.
>
> Cheers,
>
> Richard
>
> nfsstat -E -s
> Server Info:
>       Getattr      Setattr       Lookup     Readlink         Read        Write
>   17354100116   1674996998   5136769057        40544  18429670397    986443058
>        Create       Remove       Rename         Link      Symlink        Mkdir
>             0     98646613    119865835     17071853            0            0
>         Rmdir      Readdir     RdirPlus       Access        Mknod       Fsstat
>             0   1655544711            0   4129535844            0            0
>        FSinfo     pathConf       Commit      LookupP      SetClId    SetClIdCf
>             0            0      8723436            0            0            0
One more thing to note. The # of Commits is much smaller than the # of Writes
on the top line. This "hints" that a lot of the Writes are being done File_sync.
(To check that, you'd need to capture packets and look at them in wireshark.)

If a lot of the Writes are File_sync (which means the server must commit the
data/metadata changes to stable storage before replying), doing the "mirrored
pair of storage devices dedicated to the ZIL" could be what you need to get
writes to work well. (As I mentioned, the "cheat" alternative is to
set "sync=disabled",
but that runs a risk of data loss when the NFS server crashes/reboots.)

rick

>          Open     OpenAttr    OpenDwnGr     OpenCfrm    DelePurge       DelRet
>   33736862240            0           16            0            0            0
>         GetFH         Lock        LockT        LockU        Close       Verify
>    3212316334    620514754            0     91450845   1343565521            0
>       NVerify        PutFH     PutPubFH    PutRootFH        Renew    RestoreFH
>             0  72840514777            0         2939            0     17071831
>        SaveFH      Secinfo   RelLockOwn     V4Create
>     136947415            0            0       771785
> BackChannelCtBindConnToSes   ExchangeID   CreateSess  DestroySess  FreeStateID
>             0           45         2951         1538         1437     87475206
>   GetDirDeleg   GetDevInfo   GetDevList layoutCommit    LayoutGet LayoutReturn
>           468            0            0            0            0            0
>  SecInfNoName     Sequence       SetSSV  TestStateID    WantDeleg  DestroyClId
>          1445  72791198609            0         1318            0         9210
>  ReclaimCompl
>          1489
>      Allocate         Copy   CopyNotify   Deallocate     IOAdvise  LayoutError
>             0         9713            0         4666            0            0
>   LayoutStats  OffloadCncl  OffloadStat     ReadPlus         Seek    WriteSame
>             0            0            0          112        48469            0
>         Clone   GetExtattr   SetExtattr  ListExtattr    RmExtattr
>            14            0            0        65153            0
> Server:
>       Clients    OpenOwner        Opens    LockOwner        Locks       Delegs
>            44           50        15677          186          186            0
>       Layouts
>             0
> Server Cache Stats:
>        Inprog     Non-idem       Misses    CacheSize      TCPPeak
>             0            0            0            0            0