Re: [PATCH 0/5] RFC: Stop NFSv4.1 slot-growth heuristic from rewarding busy clients
"Chuck Lever" <[email protected]>
| Newsgroups | gmane.linux.nfs |
|---|---|
| Message-ID | <[email protected]> |
On Mon, Jun 15, 2026, at 2:12 PM, Benjamin Coddington wrote: > On 10 Jun 2026, at 21:58, Chuck Lever wrote: > >> This series grew out of Ben Coddington's fair-queueing RFC and the >> discussion it prompted with Neil Brown: a single busy NFSv4.1 client, >> opening many connections against a large slot table, can keep enough >> requests in flight to monopolize the nfsd thread pool and starve its >> peers [1]. Where fair queueing itself belongs is still being worked >> out. One change, though, holds no matter which design wins -- the >> server should not actively widen the gap, and today it does. >> >> nfsd4_sequence() grows a session's slot table by 20% every time the >> client reaches its highest slot, so it hands the client already >> keeping the most threads busy a still larger table; one session can >> climb toward 2048 slots on a server with far fewer threads to run >> them. This series caps that growth where added slots stop buying >> concurrency, and leaves the dynamic-slot machinery otherwise intact. >> >> Two design points are worth checking against the diffs. >> >> The cap is per session, not per namespace, and deliberately so. A >> budget shared across a namespace's sessions breaks at the floor: >> every active session permanently holds slot 0, which the shrinker >> never reclaims, so once the session count alone reaches the thread >> ceiling those floors exhaust a shared budget and pin the one busy >> client small while the pool sits idle. Making that floor explicit is >> a self-contained accounting cleanup -- nfsd_total_target_slots gains >> the full-target meaning its name implies, and the never-reclaim-slot-0 >> correction moves to the one place that reads it, the shrinker's count >> callback. Since a session cannot use another's slots, capping each >> independently against the ceiling is the natural choice. >> >> The ceiling is a sunrpc-owned quantity. Maximum sustainable >> concurrency is a property of svc_serv and svc_pool, so >> svc_serv_maxthreads() lives in sunrpc, and nfsd_nrthreads()'s >> open-coded sum is converted to it rather than nfsd carrying a second >> copy. It sums each pool's configured maximum, not the running thread >> count: nfsd sizes its pool dynamically, and gating on the live count >> would deny a client resuming from idle the slots it needs before the >> pool scales back up. >> >> This removes a perverse incentive; it is not slot admission control. >> A client still sizes its sessions at CREATE_SESSION, and per-client >> fairness against thread starvation belongs in the dispatch layer, >> where the larger discussion continues. >> >> [1] https://lore.kernel.org/linux-nfs/cover.1780498019.git.bcodding-F/[email protected]/ >> >> --- >> Chuck Lever (5): >> SUNRPC: Add svc_serv_maxthreads() to report the thread ceiling >> NFSD: Count slot 0 in nfsd_total_target_slots >> NFSD: Clean up documenting comment for reduce_session_slots() >> NFSD: Document and rename the NFSv4.1 session slot shrinker callbacks >> NFSD: Bound on-demand DRC slot growth by the thread ceiling >> >> fs/nfsd/nfs4state.c | 115 +++++++++++++++++++++++++++++++-------------- >> fs/nfsd/nfssvc.c | 12 +++-- >> include/linux/sunrpc/svc.h | 1 + >> net/sunrpc/svc.c | 23 +++++++++ >> 4 files changed, 113 insertions(+), 38 deletions(-) >> --- >> base-commit: 4549871118cf616eecdd2d939f78e3b9e1dddc48 >> change-id: 20260610-nfsd-slot-growth-clamp-ca03e338678c > > Nice! > > Reviewed-by: Benjamin Coddington <bcodding-F/[email protected]> > > I'm pretty late to review this, especially as it lands right in the same > zone I was exploring. Sorry about that.. No worries. These are sitting in nfsd-testing, and will be for a while since the merge window is now open. I added your R-b. > Do you want me to rebase the fair-queue series on top of this once it's in > nfsd-testing and re-run the A/B through the harness with this as the > baseline? Rebase on nfsd-testing whenever you are ready to post for me to merge. > Happy to follow up with a Tested-by. That would be great if you can! > The work I have yet to do on the fair-queue series is to show a prototyped > burst-allowance. -- Chuck Lever