Re: [REGRESSION] nfsd TCP socket lockup on 14.3-RELEASE-p9/p10 - Confirmed on Multiple Systems

<<On Mon, 20 Apr 2026 10:26:23 -0400, Mark Johnston <[email protected]> said:

> Would you be able to share full "procstat -kka" output from an affected
> system?

For what it's worth, I have found the following recipe to be helpful:

# procstat -hk NFSD_PID | | cut -c 54- | sort | uniq -c | sort -n

since many nfsd threads will often be sleeping with the same stack
trace, this helps to collapse them and call out the interesting
activity.

I ran this on our one active 14.3-p10 server, and the result is like
this:

   1 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dbuf_read dbuf_hold_impl dbuf_hold dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal 
   1 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_run nfsrvd_nfsd nfssvc_nfsd 
   1 soiolock sosend_generic sosend svc_vc_reply svc_sendreply_common svc_sendreply_mbuf nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 
   1 uma_zalloc_arg tcp_m_copym tcp_default_output tcp_usr_send sosend_generic_locked sosend_generic sosend svc_vc_reply svc_sendreply_common svc_sendreply_mbuf nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 
   2 mi_switch _cv_wait dbuf_read dbuf_hold_impl dbuf_hold dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit 
   2 mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig svc_run_internal svc_thread_start fork_exit fork_trampoline 
   7 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 
  19 mi_switch _cv_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 
  46 mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt svc_run_internal svc_thread_start fork_exit fork_trampoline 

-GAWollman