Re: Increase FreeBSD NFSv4 nfsd buffer size? Re: Increase default NFSv4 server size "max_block_size" to 4MB

On Thu, May 14, 2026 at 3:39 PM Rick Macklem <[email protected]> wrote:
>
> On Wed, May 13, 2026 at 4:38 PM Rick Macklem <[email protected]> wrote:
> >
> > On Wed, May 13, 2026 at 1:37 AM Cedric Blancher
> > <[email protected]> wrote:
> > >
> > > On Wed, 13 May 2026 at 01:02, Rick Macklem <[email protected]> wrote:
> > > >
> > > > On Tue, May 12, 2026 at 6:18 AM Cedric Blancher
> > > > <[email protected]> wrote:
> > > > >
> > > > > On Tue, 17 Mar 2026 at 00:35, Rick Macklem <[email protected]> wrote:
> > > > > >
> > > > > > On Mon, Mar 16, 2026 at 5:41 AM Chuck Lever <[email protected]> wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Mar 16, 2026, at 3:51 AM, Cedric Blancher wrote:
> > > > > > > > As debated a while ago, can the default NFSv4 server size for
> > > > > > > > "max_block_size" be increased to 4MB, please?
> > > > > > >
> > > > > > > There is an administrative setting to raise this limit for
> > > > > > > recent versions of the kernel. Can you report your experience
> > > > > > > when you raise the limit? Hiccups, performance issues, etc? I
> > > > > > > would kind of like this exercise to be data-driven.
> > > > > > >
> > > > > > > What is still unknown to me is which NFS client implementations
> > > > > > > can support 4MB or 8MB. Without client support, an increase in
> > > > > > > the default in NFSD doesn't mean anything. Rick, Anna, Roland?
> > > > > > Although it has not seen much testing, it is possible to do a > 1Mbyte NFSv4
> > > > > > mount in FreeBSD.
> > > > > > For a 2Mbyte mount, (the only size > 1Mbyte I've tried) the settings would be..
> > > > > > In /boot/loader.conf
> > > > > > kern.maxphys=2097152
> > > > > > vfs.maxbcachebuf=2097152
> > > > > >
> > > > > > and in /etc/sysctl.conf
> > > > > > kern.ipc.maxsockbuf=9455616
> > > > > >
> > > > > > Then a mount will use 2Mbytes if the server supports it.
> > > > >
> > > > > How can I verify that the FreeBSD NFSv4 nfsd now uses 2M for NFS buffers?
> > > > The default is 128K. (You can see what it is via "sysctl vfs.nfsd.srvmaxio".)
> > > >
> > > > With "main",
> > >
> > > Which FreeBSD version is "main" exactly, i.e. which 16.0 snapshot
> > > date? Is this supported for 15.0?
> The commit just hit "main" as b92b9da33006. So any snapshot after
> to-day should have it. (It will be MFC'd to stable/15 in 2 weeks and will
> be in 15.2.)
>
> To enable it, put this line in /etc/rc.conf:
> nfs_server_maxio=4194304
> and this line in /etc/sysctl.conf:
> kern.ipc.maxsockbuf=18892800 (or larger)
Oh, and I should also point out that, for the low end hardware I have,
I have never seen a performance improvement for a maxio setting
above 256Kbytes.  The story may be different for faster hardware,
but I wouldn't just assume that bumping it up will get you better
performance. (There is, of course, also a difference between getting
better performance for a single client vs fair distribution of server
resources across many clients.)

rick

>
> rick
>
> > >
> > > > you can increase that up to 4M by putting
> > > > nfs_server_maxio=N (where N can be up to 4194304 for "main" and 1048576 for 15)
> > >
> > > What line do I have to put into /etc/sysctl.conf and /etc/rc.conf?
> > Oops, my bad. It isn't committed yet. I forgot that I have to convince the
> > FreeBSD collective to increase UIO_MAXIOV or convince the ZFS folk
> > to not check for that limit in ZFS.
> >
> > I will email when it gets into "main".
> >
> > It can currently be set to 1Mbyte via
> > nfs_server_maxio=1048576
> > in /etc/rc.conf and
> > kern.ipc.maxsockbuf=5242880  (the exact minimum setting is printed
> >                                                     out if you boot
> > without this set large
> >                                                      enough in
> > /etc/sysctl.conf.)
> >                                                    --> it must be at
> > least a little more than
> >                                                         4 *
> > nfs_server_maxio so that 4 reads
> >                                                         or writes fit
> > in the socket's buffers.
> >
> > rick
> >
> > >
> > > > - adding an entry in /etc/sysctl.conf for a larger value for kern.ipc.maxsockbuf
> > > >   (It will tell you the recommended minimum setting if you boot after
> > > >    putting nfs_server_maxio=N in /etc/rc.conf.)
> > >
> > > How can I verify that the settings work, on the NFS server side?
> > Look in a wireshark packet trace for maxread and maxwrite in a
> > GETATTR reply.
> >
> > > Can
> > > we programmatically probe which values are supported?
> > The sysctl is vfs.nfsd.srvmaxio and, when you try and set this
> > to too large a value, it will fail with EINVAL.
> > (You can do this from C via the sysctl(3) library call. See "man 3 sysctl".)
> >
> > rick
> >
> > >
> > > Ced
> > > --
> > > Cedric Blancher <[email protected]>
> > > [https://plus.google.com/u/0/+CedricBlancher/]
> > > Institute Pasteur
> > >