[RFC PATCH 7/7] exit: route exit_mm()'s final mmput() through mmput_exit()

Aditya Sharma <[email protected]>
Newsgroups org.kernel.vger.linux-trace-kernel,org.kernel.vger.linux-doc,org.kernel.vger.linux-kernel,org.kvack.linux-mm
Message-ID <[email protected]>
This makes the series operative. So far, nothing called mmput_exit().
Switch exit_mm() from mmput() to mmput_exit(), so the exiting task's final
reference drop can defer the address-space teardown to mm_reaper
instead of running exit_mmap() on the exiting CPU.

Only the exit path is routed. Every other mmput() caller (get_task_mm()
users like ptrace and /proc, kthread_unuse_mm(), etc.) still tears down
inline, and if one of those ends up holding the actual last reference,
behavior is unchanged -- the deferral applies only when the exiting
task's own drop is the final one.

The deferral triggers only when all of the following hold, and falls
back to inline __mmput() otherwise:

- vm.async_mm_teardown=1 (static key, default off)
- RSS >= vm.async_mm_teardown_thresh_pages (default 64MB)
- the mm is not an OOM target (MMF_OOM_TARGETED) and has not already
been reaped (MMF_OOM_SKIP)
- charging the RSS to the pending budget stays under
vm.async_mm_teardown_max_pending_pages

Results, bare metal, 32 CPUs / 32GB RAM (31 GiB usable), single
NUMA node, performance governor, medians of 10 reps, THP off unless noted,
vm.async_mm_teardown_max_pending_pages raised to 3/4 of RAM for the
async runs (the RAM/4 default would reject a single 16GB mm on this
box; see the cover letter's open question on the cap default):

reap latency (SIGKILL -> waitpid() returns), async off -> on:
1GB    18.01 ms  ->  0.14 ms
4GB    73.71 ms  ->  0.14 ms
16GB  282.16 ms  ->  0.15 ms
16GB   27.68 ms  ->  0.13 ms   (THP on)

voluntary exit (_exit() -> reap) matches within noise:
16GB  271.27 ms  ->  0.12 ms

redis-server with a 16GB populated dataset (real heap, VmRSS ~16.2GB):
      292.97 ms  ->  0.24 ms

32 simultaneous 0.5GB exits, tail until all reaped:
      157.42 ms  ->  0.32 ms

below-threshold 30MB process with the feature enabled:
        0.56 ms  ->  0.60 ms   (sync path, within noise)

A CONFIG_ASYNC_MM_TEARDOWN=n build of the same tree reproduces the
async-off column within noise (18.01 vs 18.01 ms at 1GB, 282.14 vs
282.16 ms at 16GB), so the config itself costs nothing when dark.

The work is moved, not eliminated: with async on, the 16GB region is
fully back in the buddy allocator ~240 ms after the kill (vs ~283 ms
when torn down inline), and the reaper spends ~1.18x the inline CPU on
the same teardowns (nice-19 kthread, cache-cold on another CPU). The
cover letter carries the full matrix: freeing-latency checks,
backpressure fallback, OOM-killer interaction (an OOM victim is never
queued, tracepoint-verified, including the CLONE_VM-sharer case),
tmpfs inode eviction in reaper context, sysctl toggling under load,
and a freezer cycle with a queued backlog.

Signed-off-by: Aditya Sharma <[email protected]>
---
 kernel/exit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 1056422bc..6f97d204f 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -607,7 +607,7 @@ static void exit_mm(void)
 	task_unlock(current);
 	mmap_read_unlock(mm);
 	mm_update_next_owner(mm);
-	mmput(mm);
+	mmput_exit(mm);
 	if (test_thread_flag(TIF_MEMDIE))
 		exit_oom_victim();
 }
-- 
2.34.1
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.