[PATCH v5 3/5] sunrpc: guarantee a thread per pool when auto-distributing
Jeff Layton <[email protected]>
| Newsgroups | gmane.linux.nfs,gmane.linux.kernel |
|---|---|
| Message-ID | <[email protected]> |
svc_set_num_threads() spreads the requested thread count evenly across the service's pools. In pernode mode each pool maps to a NUMA node, and svc_pool_for_cpu() steers an incoming transport to the pool for the node it arrived on. When fewer threads than pools are requested, even distribution leaves some pools empty, and a transport steered to an empty pool has no thread to service it. Floor each pool at one thread when auto-distributing a non-zero count, so no pool is left empty. Every pool maps to a node that had CPUs when the pool map was built (svc_pool_map_init_pernode() only creates pools for nodes returned by for_each_node_with_cpus()), so there is no pool that should be left threadless. The resulting total may exceed the requested count. This only affects the auto-distribute path (a single-value array, i.e. svc_set_num_threads()); callers that set per-pool counts explicitly via svc_set_pool_threads() are unchanged and may still set a pool to zero. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Jeff Layton <[email protected]> Reviewed-by: NeilBrown <neil-+NVA1uvv1dVBDLzU/[email protected]> --- net/sunrpc/svc.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index b942845f82a3..2d1cdf55c561 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -840,6 +840,12 @@ EXPORT_SYMBOL_GPL(svc_set_pool_threads); * are multiple pools then the new threads or victims will be distributed * evenly among them. * + * When @nrservs is non-zero but smaller than the number of pools, even + * distribution would leave some pools empty. Since each pool maps to a + * NUMA node and only services transports steered to that node, every + * pool is instead guaranteed at least one thread. The resulting total + * may therefore exceed @nrservs. + * * Caller must ensure mutual exclusion between this and server startup or * shutdown. * @@ -855,6 +861,16 @@ svc_set_num_threads(struct svc_serv *serv, unsigned int min_threads, unsigned int remain = nrservs % serv->sv_nrpools; int i, err = 0; + /* + * Don't let a pool sit empty while threads are being + * auto-distributed: a transport steered to its node would have + * nothing to service it. Every pool maps to a CPU-bearing node, + * so hand each one a thread. This may push the total above + * @nrservs. + */ + if (base == 0 && nrservs != 0) + remain = serv->sv_nrpools; + for (i = 0; i < serv->sv_nrpools; ++i) { struct svc_pool *pool = &serv->sv_pools[i]; int threads = base; -- 2.55.0