[PATCH v3 3/4] sunrpc: guarantee a thread per CPU-bearing node when auto-distributing
Jeff Layton <[email protected]>
| Newsgroups | gmane.linux.nfs,gmane.linux.kernel |
|---|---|
| Message-ID | <[email protected]> |
svc_set_num_threads() spreads the requested thread count evenly across the service's pools. In pernode mode each pool maps to a NUMA node, and svc_pool_for_cpu() steers an incoming transport to the pool for the node it arrived on. When fewer threads than pools are requested, even distribution leaves some nodes' pools empty, and a transport steered to an empty pool has no thread to service it. Floor each CPU-bearing node's pool at one thread when auto-distributing a non-zero count, so no such pool is left empty. The resulting total may exceed the requested count. This only affects the auto-distribute path (a single-value array, i.e. svc_set_num_threads()); callers that set per-pool counts explicitly via svc_set_pool_threads() are unchanged and may still set a pool to zero. Nodes without CPUs (e.g. memory-only nodes) get no thread, as nothing is steered to them. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Jeff Layton <[email protected]> --- net/sunrpc/svc.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 2f6938fe28b2..99a4fd62399b 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -826,6 +826,12 @@ EXPORT_SYMBOL_GPL(svc_set_pool_threads); * are multiple pools then the new threads or victims will be distributed * evenly among them. * + * When @nrservs is non-zero but smaller than the number of pools, even + * distribution would leave some pools empty. Since each pool maps to a + * NUMA node and only services transports steered to that node, every + * pool whose node has CPUs is instead guaranteed at least one thread. + * The resulting total may therefore exceed @nrservs. + * * Caller must ensure mutual exclusion between this and server startup or * shutdown. * @@ -850,6 +856,15 @@ svc_set_num_threads(struct svc_serv *serv, unsigned int min_threads, --remain; } + /* + * Don't let a node's pool sit empty while threads are + * being auto-distributed: a transport steered there would + * have nothing to service it. + */ + if (threads == 0 && nrservs && + nr_cpus_node(svc_pool_map_get_node(pool->sp_id))) + threads = 1; + err = svc_set_pool_threads(serv, pool, min_threads, threads); if (err) break; -- 2.54.0