Re: [RFC PATCH 0/1] psi: Introduce in-kernel PSI auto monitor feature

Pintu Kumar Agarwal <[email protected]> Mon, 20 Jul 2026 19:59:50 +0530
Newsgroups org.kernel.vger.linux-embedded,org.kernel.vger.linux-kernel,org.kernel.vger.linux-trace-kernel
Message-ID <CANqwyPhJ82Vy9_BUzBumVJKxdMySv+YVaMHNTCOtz2sGOh_h2A@mail.gmail.com>
Dear Johannes,

On Mon, Jul 13, 2026 at 5:29=E2=80=AFPM Johannes Weiner <[email protected]=
> wrote:
>
> On Thu, Jul 02, 2026 at 10:46:05PM +0530, Pintu Kumar Agarwal wrote:
> > Hi all,
> >
> > This RFC introduces an in-kernel PSI auto monitor aimed at improving
> > root-cause visibility for resource pressure events in Linux systems.
> >
> > Motivation:
> >
> > PSI already provides an excellent mechanism to detect CPU, memory and
> > I/O pressure and includes trigger-based notifications via pollable
> > interfaces. However, it deliberately avoids attributing pressure to
> > individual tasks.
>
> Why is this necessary?
>
> > In real-world systems, this creates a gap: when a PSI trigger fires,
> > users still need to determine *which tasks caused the stall* by combini=
ng
> > multiple tools (top, meminfo, vmstat, perf, tracing, etc.), often after
> > the event has already passed.
>
> I've never found myself needing to identify which specific task was
> stalled. Tasks competing over a shared resource are interdependent.
>
> Consider this scenario: Task A, B, C are allocating memory and
> creating pressure together; B randomly becomes the sucker to hit
> direct reclaim and making room for the other too as well. Why is it
> meaningful to know that B stalled? It caused the resource contention
> no more than the other two.
>
> Then A and C refault: A is fast, but C happens to hit when the flash
> drive is running garbage collection. Why is it meaningful to know C?
> Again, C is no more the culprit in that situation than the others.
>
> The meaningful thing you can say is that the domain as a whole is
> under pressure. Who exactly becomes the lightning rod is noise.
>
> That said, if you do need it, why not use delayacct? It already tracks
> the reclaim, swap, thrashing time and events that also go into psi on
> a per-task basis. It's not sampled, either.

Thank you so much for your detailed feedback. This is very helpful.
Your point about resource pressure being a domain-level phenomenon
rather is well taken.
I agree that pressure is often created collectively by multiple
workloads, and that the task experiencing the stall is not necessarily
the root cause.

The goal of the RFC was not to identify a single culprit, but to
provide additional attribution context and surrounding tasks around
PSI events
by capturing the major resource consumers and participants exactly at
the time pressure becomes significant.

Your suggestion regarding delayacct is very insightful.
I think there is a utility under tools/accounting/delaytop.c which
already provides task-level delay information in user space when run
manually.

I am currently studying delayacct, taskstats and the existing delaytop
utility to understand how I can correlate PSI auto-monitor attribution
using delayacct.
The idea could be:
PSI threshold breach -> PSI Auto Monitor trigger -> Delayacct provides
delayed task list -> Auto monitor dump / create trace events
PSI threshold breach -> PSI Auto Monitor is triggered -> Collect
per-task delayacct/taskstats data
-> Identify tasks with significant reclaim / swap / I/O / CPU
scheduling delay -> Emit trace events / logs / structured report

One additional motivation for the in-kernel prototype was early-boot
observability.
Unlike a userspace monitor, it can capture pressure events and
attribution context before userspace services are available,
which proved useful in several embedded boot-time investigations.
One more point is that the psi auto monitor can run and trigger on its
own during early boot.
It can quickly help to identify bottlenecks during boot and aid in
boot time optimization at both kernel and user space level.

As a side note, please see one of the snapshots below captured during boot.
# dmesg | grep psi
[   30.948653][  T420] psi_monitor: pressure high: cpu=3D50% mem=3D1%
io=3D11% (thresh cpu=3D50 mem=3D60 io=3D40)
[   30.958342][  T420] psi_monitor: logging top 8 tasks under pressure:
[   30.964959][  T420] psi_monitor: pid=3D2637 comm=3Dpcid-loc-api
psi_flag=3D12 oncpu=3D0 cputime(ms)=3D443 rss(kB)=3D21976 io(kB)=3D0 score=
=3D13203
[   30.977392][  T420] psi_monitor: pid=3D1974 comm=3Dsyslogd psi_flag=3D0
oncpu=3D3 cputime(ms)=3D643 rss(kB)=3D2156 io(kB)=3D272 score=3D4565
[   30.989137][  T420] psi_monitor: pid=3D2597 comm=3Dqwesd psi_flag=3D4
oncpu=3D3 cputime(ms)=3D104 rss(kB)=3D7964 io(kB)=3D0 score=3D4502
[   31.000540][  T420] psi_monitor: pid=3D1493 comm=3Ddlt-system
psi_flag=3D0 oncpu=3D3 cputime(ms)=3D370 rss(kB)=3D1984 io(kB)=3D136 score=
=3D2978
[   31.014588][  T420] psi_monitor: pid=3D15 comm=3Drcu_preempt psi_flag=3D=
0
oncpu=3D2 cputime(ms)=3D443 rss(kB)=3D0 io(kB)=3D0 score=3D2215
[   31.033593][  T420] psi_monitor: pid=3D420 comm=3Dkworker/2:9+events
psi_flag=3D12 oncpu=3D2 cputime(ms)=3D351 rss(kB)=3D0 io(kB)=3D0 score=3D17=
55
[   31.051276][  T420] psi_monitor: pid=3D2713 comm=3Dsdir psi_flag=3D0
oncpu=3D1 cputime(ms)=3D124 rss(kB)=3D0 io(kB)=3D20 score=3D640
[   31.065800][  T420] psi_monitor: pid=3D41 comm=3Dkworker/u8:1-memlat_wq
psi_flag=3D0 oncpu=3D3 cputime(ms)=3D52 rss(kB)=3D0 io(kB)=3D0 score=3D260


Thank you once again for your review and valuable feedback.
I will explore more and improve the design for v2.


Thanks,
Pintu