Re: [RFC PATCH 0/1] psi: Introduce in-kernel PSI auto monitor feature
Pintu Kumar Agarwal <[email protected]> Mon, 20 Jul 2026 19:59:50 +0530
| Newsgroups | org.kernel.vger.linux-embedded,org.kernel.vger.linux-kernel,org.kernel.vger.linux-trace-kernel |
|---|---|
| Message-ID | <CANqwyPhJ82Vy9_BUzBumVJKxdMySv+YVaMHNTCOtz2sGOh_h2A@mail.gmail.com> |
Dear Johannes, On Mon, Jul 13, 2026 at 5:29=E2=80=AFPM Johannes Weiner <[email protected]= > wrote: > > On Thu, Jul 02, 2026 at 10:46:05PM +0530, Pintu Kumar Agarwal wrote: > > Hi all, > > > > This RFC introduces an in-kernel PSI auto monitor aimed at improving > > root-cause visibility for resource pressure events in Linux systems. > > > > Motivation: > > > > PSI already provides an excellent mechanism to detect CPU, memory and > > I/O pressure and includes trigger-based notifications via pollable > > interfaces. However, it deliberately avoids attributing pressure to > > individual tasks. > > Why is this necessary? > > > In real-world systems, this creates a gap: when a PSI trigger fires, > > users still need to determine *which tasks caused the stall* by combini= ng > > multiple tools (top, meminfo, vmstat, perf, tracing, etc.), often after > > the event has already passed. > > I've never found myself needing to identify which specific task was > stalled. Tasks competing over a shared resource are interdependent. > > Consider this scenario: Task A, B, C are allocating memory and > creating pressure together; B randomly becomes the sucker to hit > direct reclaim and making room for the other too as well. Why is it > meaningful to know that B stalled? It caused the resource contention > no more than the other two. > > Then A and C refault: A is fast, but C happens to hit when the flash > drive is running garbage collection. Why is it meaningful to know C? > Again, C is no more the culprit in that situation than the others. > > The meaningful thing you can say is that the domain as a whole is > under pressure. Who exactly becomes the lightning rod is noise. > > That said, if you do need it, why not use delayacct? It already tracks > the reclaim, swap, thrashing time and events that also go into psi on > a per-task basis. It's not sampled, either. Thank you so much for your detailed feedback. This is very helpful. Your point about resource pressure being a domain-level phenomenon rather is well taken. I agree that pressure is often created collectively by multiple workloads, and that the task experiencing the stall is not necessarily the root cause. The goal of the RFC was not to identify a single culprit, but to provide additional attribution context and surrounding tasks around PSI events by capturing the major resource consumers and participants exactly at the time pressure becomes significant. Your suggestion regarding delayacct is very insightful. I think there is a utility under tools/accounting/delaytop.c which already provides task-level delay information in user space when run manually. I am currently studying delayacct, taskstats and the existing delaytop utility to understand how I can correlate PSI auto-monitor attribution using delayacct. The idea could be: PSI threshold breach -> PSI Auto Monitor trigger -> Delayacct provides delayed task list -> Auto monitor dump / create trace events PSI threshold breach -> PSI Auto Monitor is triggered -> Collect per-task delayacct/taskstats data -> Identify tasks with significant reclaim / swap / I/O / CPU scheduling delay -> Emit trace events / logs / structured report One additional motivation for the in-kernel prototype was early-boot observability. Unlike a userspace monitor, it can capture pressure events and attribution context before userspace services are available, which proved useful in several embedded boot-time investigations. One more point is that the psi auto monitor can run and trigger on its own during early boot. It can quickly help to identify bottlenecks during boot and aid in boot time optimization at both kernel and user space level. As a side note, please see one of the snapshots below captured during boot. # dmesg | grep psi [ 30.948653][ T420] psi_monitor: pressure high: cpu=3D50% mem=3D1% io=3D11% (thresh cpu=3D50 mem=3D60 io=3D40) [ 30.958342][ T420] psi_monitor: logging top 8 tasks under pressure: [ 30.964959][ T420] psi_monitor: pid=3D2637 comm=3Dpcid-loc-api psi_flag=3D12 oncpu=3D0 cputime(ms)=3D443 rss(kB)=3D21976 io(kB)=3D0 score= =3D13203 [ 30.977392][ T420] psi_monitor: pid=3D1974 comm=3Dsyslogd psi_flag=3D0 oncpu=3D3 cputime(ms)=3D643 rss(kB)=3D2156 io(kB)=3D272 score=3D4565 [ 30.989137][ T420] psi_monitor: pid=3D2597 comm=3Dqwesd psi_flag=3D4 oncpu=3D3 cputime(ms)=3D104 rss(kB)=3D7964 io(kB)=3D0 score=3D4502 [ 31.000540][ T420] psi_monitor: pid=3D1493 comm=3Ddlt-system psi_flag=3D0 oncpu=3D3 cputime(ms)=3D370 rss(kB)=3D1984 io(kB)=3D136 score= =3D2978 [ 31.014588][ T420] psi_monitor: pid=3D15 comm=3Drcu_preempt psi_flag=3D= 0 oncpu=3D2 cputime(ms)=3D443 rss(kB)=3D0 io(kB)=3D0 score=3D2215 [ 31.033593][ T420] psi_monitor: pid=3D420 comm=3Dkworker/2:9+events psi_flag=3D12 oncpu=3D2 cputime(ms)=3D351 rss(kB)=3D0 io(kB)=3D0 score=3D17= 55 [ 31.051276][ T420] psi_monitor: pid=3D2713 comm=3Dsdir psi_flag=3D0 oncpu=3D1 cputime(ms)=3D124 rss(kB)=3D0 io(kB)=3D20 score=3D640 [ 31.065800][ T420] psi_monitor: pid=3D41 comm=3Dkworker/u8:1-memlat_wq psi_flag=3D0 oncpu=3D3 cputime(ms)=3D52 rss(kB)=3D0 io(kB)=3D0 score=3D260 Thank you once again for your review and valuable feedback. I will explore more and improve the design for v2. Thanks, Pintu