[PATCH mm-unstable v2 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttling for MGLRU

From: Hui Zhu <[email protected]>

The legacy reclaim path updates the NR_ISOLATED_ANON/FILE node
counters around isolation and throttles direct reclaimers via
too_many_isolated() when isolated folios pile up.  The MGLRU eviction
path does neither: evict_folios() isolates folios without touching
the counters and never consults too_many_isolated().

Patch 1 updates NR_ISOLATED_ANON/FILE around isolation in
evict_folios(), reusing the existing nr_isolated.  Without this the
counters stay at zero while MGLRU reclaim is active, so compaction's
too_many_isolated() cannot see the pages MGLRU has isolated.

Patch 2 extracts the throttling loop from shrink_inactive_list() into
throttle_isolated() and calls it from evict_folios() as well, keeping
the two reclaim paths unified, in line with what was done previously
for writeback reclaim.  This way the MGLRU eviction path is throttled
when isolated folios pile up instead of thrashing the shrinking LRU
lists - the scenario the too_many_isolated() check exists for.  A
dying task fakes reclaim progress exactly like the legacy path so it
exits reclaim quickly.

Testing
=======

Test on 8G RAM qemu.
The reproducer confines stress-ng workers in a 192M memcg and swaps
through dm-delay (300ms write latency) so pageout is slow and isolated
folios pile up; the workload is intentionally extreme to force the
throttle path.
Throttle events are counted via the mm_vmscan_throttled tracepoint.
The test scripts and test log is in [1].

Test 1, reclaim throttling, parallel direct reclaim in the memcg:

                            before        after
throttle events (ISOLATED)       0       512472
  - from kswapd                  0            0
nr_isolated_anon peak            0         3670
pswpout                    3383673        87134

Without the series MGLRU reclaim swaps heavily while nr_isolated_*
stays at 0 and nothing is throttled.  With the series the counters
are updated and direct reclaimers are throttled (the pswpout drop is
the reclaimers backing off); kswapd stays exempt.

Test 2, counters visible to compaction, same pressure plus
compact_memory in parallel:

                            before        after
nr_isolated peak                59         1093

The "before" 59 is compaction's own transient isolation; reclaim's
isolation is invisible.  With patch 1 it becomes visible to
compaction's too_many_isolated().  (Compaction's own throttling
threshold, (inactive + active) / 16, is ~23k pages on this box and
needs more pile-up than the box can generate; test 1 exercises the
same throttling mechanism end to end on the reclaim side.)

[1] https://gist.github.com/teawater/d3968aac92eb6bd1378beb54a82933f4

Changelog:
v2:
Accoding to the commens of Kairun, Rebased on mm-unstable.
Split into two patches; patch 2 is new and adds the
too_many_isolated() throttling to the MGLRU eviction path, which v1
did not cover.
Add test infomations.

Hui Zhu (2):
  mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim
    path
  mm/vmscan: apply too_many_isolated() throttling to MGLRU eviction

 mm/vmscan.c | 74 +++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 63 insertions(+), 11 deletions(-)

-- 
2.53.0