Re: [PATCH 0/5] RFC: Stop NFSv4.1 slot-growth heuristic from rewarding busy clients
Jeff Layton <[email protected]>
| Newsgroups | gmane.linux.nfs |
|---|---|
| Message-ID | <[email protected]> |
On Wed, 2026-06-10 at 21:58 -0400, Chuck Lever wrote: > This series grew out of Ben Coddington's fair-queueing RFC and the > discussion it prompted with Neil Brown: a single busy NFSv4.1 client, > opening many connections against a large slot table, can keep enough > requests in flight to monopolize the nfsd thread pool and starve its > peers [1]. Where fair queueing itself belongs is still being worked > out. One change, though, holds no matter which design wins -- the > server should not actively widen the gap, and today it does. > > nfsd4_sequence() grows a session's slot table by 20% every time the > client reaches its highest slot, so it hands the client already > keeping the most threads busy a still larger table; one session can > climb toward 2048 slots on a server with far fewer threads to run > them. This series caps that growth where added slots stop buying > concurrency, and leaves the dynamic-slot machinery otherwise intact. > > Two design points are worth checking against the diffs. > > The cap is per session, not per namespace, and deliberately so. A > budget shared across a namespace's sessions breaks at the floor: > every active session permanently holds slot 0, which the shrinker > never reclaims, so once the session count alone reaches the thread > ceiling those floors exhaust a shared budget and pin the one busy > client small while the pool sits idle. Making that floor explicit is > a self-contained accounting cleanup -- nfsd_total_target_slots gains > the full-target meaning its name implies, and the never-reclaim-slot-0 > correction moves to the one place that reads it, the shrinker's count > callback. Since a session cannot use another's slots, capping each > independently against the ceiling is the natural choice. > > The ceiling is a sunrpc-owned quantity. Maximum sustainable > concurrency is a property of svc_serv and svc_pool, so > svc_serv_maxthreads() lives in sunrpc, and nfsd_nrthreads()'s > open-coded sum is converted to it rather than nfsd carrying a second > copy. It sums each pool's configured maximum, not the running thread > count: nfsd sizes its pool dynamically, and gating on the live count > would deny a client resuming from idle the slots it needs before the > pool scales back up. > > This removes a perverse incentive; it is not slot admission control. > A client still sizes its sessions at CREATE_SESSION, and per-client > fairness against thread starvation belongs in the dispatch layer, > where the larger discussion continues. > > [1] https://lore.kernel.org/linux-nfs/[email protected]/ > > --- > Chuck Lever (5): > SUNRPC: Add svc_serv_maxthreads() to report the thread ceiling > NFSD: Count slot 0 in nfsd_total_target_slots > NFSD: Clean up documenting comment for reduce_session_slots() > NFSD: Document and rename the NFSv4.1 session slot shrinker callbacks > NFSD: Bound on-demand DRC slot growth by the thread ceiling > > fs/nfsd/nfs4state.c | 115 +++++++++++++++++++++++++++++++-------------- > fs/nfsd/nfssvc.c | 12 +++-- > include/linux/sunrpc/svc.h | 1 + > net/sunrpc/svc.c | 23 +++++++++ > 4 files changed, 113 insertions(+), 38 deletions(-) > --- > base-commit: 4549871118cf616eecdd2d939f78e3b9e1dddc48 > change-id: 20260610-nfsd-slot-growth-clamp-ca03e338678c > > Best regards, > -- > Chuck Lever <[email protected]> This makes a lot of sense. It'd be nice to clean up or comment the nit Neil pointed out with the ternary assignment, but otherwise you can add: Reviewed-by: Jeff Layton <[email protected]>