[PATCH v6] sched/psi: Skip CPUs with zero non-idle delta in per-CPU aggregation

Zhan Xusheng <[email protected]>
Newsgroups gmane.linux.kernel
Message-ID <[email protected]>
collect_percpu_times() iterates over every possible CPU to build a
non-idle-weighted average of the PSI state times. When a CPU has no
PSI_NONIDLE delta for the current sampling interval:

    nonidle    = nsecs_to_jiffies(times[PSI_NONIDLE]) = 0
    deltas[s] += times[s] * nonidle                /* += 0 */

so the weighted accumulation contributes nothing.

get_recent_times() already sets the PSI_NONIDLE bit in cpu_changed_states
iff the PSI_NONIDLE delta is non-zero. Use that bit to skip such CPUs
early, as suggested by Johannes, avoiding the nsecs_to_jiffies() call and
the PSI_NONIDLE * u64 mul-adds that follow.

No functional change: on the skipped path the old code adds zero to
deltas[] and zero to nonidle_total, which is exactly the result of not
iterating. The PSI_NONIDLE bit is folded into changed_states before the
skip, so the aggregator's reschedule decision is unaffected.

The cost is worth trimming because collect_percpu_times() is
O(nr_possible_cpus) per group and is not only called from the 2s
averaging work: when a PSI trigger is armed, psi_rtpoll_work() calls it at
the trigger cadence. On a 12-thread host with a "some 50000 500000"
trigger armed, bpftrace measured collect_percpu_times() running ~160
times/s (vs ~0.5/s on the averaging path), and on a partially loaded box a
large fraction of the per-CPU iterations per call had no PSI_NONIDLE delta
and are skipped.

read(/proc/pressure/cpu) median latency, QEMU/KVM A/B on mainline
v7.1-rc2+ (identical config, the patch the only difference), idle guest,
100k iterations, varying -smp:

    -smp    baseline   patched      delta
      2      1834 ns    1799 ns     -1.9%
      4      1957 ns    1897 ns     -3.1%
      8      2204 ns    2103 ns     -4.6%
     12      2369 ns    2272 ns     -4.1%
     16      2596 ns    2363 ns     -9.0%
     24      3083 ns    2677 ns    -13.2%
     32      3740 ns    3050 ns    -18.4%

The saving scales with CPU count, toward the many-CPU systems that run
pressure-monitoring agents. (The high -smp points use KVM oversubscription
on a 12-thread host and carry scheduling noise; the trend is the signal.
The all-busy case is within run-to-run noise.)

Suggested-by: Johannes Weiner <[email protected]>
Signed-off-by: Zhan Xusheng <[email protected]>
---
Changes in v6:
 - Add the benchmark data requested on v4: the -smp scaling A/B table and
   the psi_rtpoll_work() call-frequency measurement. No code change vs v5.
v5: reword changelog (consistency rationale; "No functional change")
    https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/

 kernel/sched/psi.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c
index e2e825dcd088..e2805d32743c 100644
--- a/kernel/sched/psi.c
+++ b/kernel/sched/psi.c
@@ -386,6 +386,9 @@ static void collect_percpu_times(struct psi_group *group,
 				&cpu_changed_states);
 		changed_states |= cpu_changed_states;
 
+		if (!(cpu_changed_states & (1 << PSI_NONIDLE)))
+			continue;
+
 		nonidle = nsecs_to_jiffies(times[PSI_NONIDLE]);
 		nonidle_total += nonidle;
 
-- 
2.43.0
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.