Re: Suggestions to reduce FreeBSD nfsd I/O latency on ZFS?

On Tue, May 12, 2026 at 3:09 PM vermaden <[email protected]> wrote:
>
> > Are there suggestions on how to reduce
> > FreeBSD nfsd I/O latency on ZFS?
> > Can putting nfsd into a realtime process class help?
> >
> > Lionel
>
> Putting nfsd(8) in 'realtime' class with rtprio(8) command
> will surely guarantee You the CPU time ... but on I/O part:
>
> These ZFS settings should help:
> - atime -> off
> - primarycache -> all
> - redundant_metadata -> none
> - sync -> standard
Another thing to consider here is the possibility of putting the
ZIL on a mirrored pair dedicated storage devices.
If you look at a packet trace in wireshark, see if the writes
are FILE_SYNC or UNSTABLE. (If FILE_SYNC, getting good
ZIL performance will be more important.)

Also, you might want to try making the recordsize the same
as the I/O size the client is using. As noted below,
the default NFS maximum I/O size is 128K, but can be increased
to 1Mbyte (or 4Mbytes if running "main").

> - logbias -> latency
> - prefetch -> none
> - xattr -> sa
>
> On nfsd(8) part ... you may want to enable more threads with '-n' option in 'nfs_server_flags'.
> # grep nfs_server /etc/rc.conf
> nfs_server_enable="YES"
> nfs_server_flags="-n 32"
If you have a lot of client mounts (or a client that can do a lot
of concurrent I/Os, you could bump this up a lot more.
(They are kernel threads, so they don't have a lot of overhead,
mostly just a kernel stack.  If you have too many, the extras
just sit idle.)

If you bump it way up and then do a "ps axHl" after the server
has been running under load for a while, the unused ones will
have 0:00.00 listed for TIME for them.

rick

>
> You may also want to increase Max I/O to 1MB.
> # sysctl vfs.nfsd.srvmaxio
> vfs.nfsd.srvmaxio: 1048576
>
> Hope that helps.
>
> Regards,
> vermaden
>
>