Re: I/O read and write *latency*, why is it so much higher on NFSv4.2, compared to P9 and iSCSI?

On Fri, 5 Jun 2026 at 20:43, Dan Shelton <[email protected]> wrote:
>
> Except from the freebsd hackers mailing list:
>
> On Tue, 19 May 2026 at 00:39, Bakul Shah <[email protected]> wrote:
> >
> > On May 18, 2026, at 2:26 PM, Dan Shelton <[email protected]> wrote:
> > >
> > > On Fri, 20 Feb 2026 at 01:21, Bakul Shah <[email protected]> wrote:
> > >>
> > >> On Feb 19, 2026, at 11:45 AM, Dan Shelton <[email protected]> wrote:
> > >>>
> > >>> On Wed, 18 Feb 2026 at 22:45, Dan Shelton <[email protected]> wrote:
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> Has anyone tried a BHYVE with a disk as file on a NFSv4.2 mount?
> > >>
> > >> Yes. [I tried this on a 15.0-RELEASE-p3 host, nfsv4.2 mounting
> > >> a filesystem from a 15.0-STABLE machine]
> > >
> > > How about the performance? Is it better than iSCSI?
> >
> > I don't know about iSCSI but comparing with p9fs:
> >
> > Test1:
> > dd bs=1m count=4000 > /dev/null < large-file
> >
> > nfsV3:
> > 32.3
> > 46.3
> > 51.3
> >
> > nfsV4:
> > 129.1
> > 59.9
> > 48.8
> >
> > p9fs:
> > 17.7
> > 17.5
> > 17.6
> >
> > Test2:
> > find /usr/src/ > /dev/null
> >
> > nfsV3:
> > 60.0
> > 39.0
> > 30.9
> >
> > nfsV4:
> > 54.0
> > 17.9
> > 35.8
> >
> > p9fs:
> > 6.9
> > 6.5
> > 6.6
> >
> >
> > So slower in all cases. In addition the variability in nfs numbers is concerning!
> >
> > p9fs doesn't cache but nfs does, so anything cached is served much faster.
>
> How is the situation on Linux? How is read/write *latency* NFSv4.2
> compared to iSCSI and P9FS?

Colleague of mine said Linux NFS is much worse, because of multiple
CPU context switch for each NFS write to disk, NFS read from disk. Is
that accurate?

Dan
-- 
Dan Shelton - Cluster Specialist Win/Lin/Bsd