[PATCH] LoongArch: rethook: Do not save/restore percpu base register in trampoline
Wentao Guan <[email protected]>
| Newsgroups | dev.linux.lists.loongarch,org.kernel.vger.linux-kernel,org.kernel.vger.stable |
|---|---|
| Message-ID | <[email protected]> |
The rethook trampoline saves $r21 ($u0), the percpu base, into its
frame at entry and restores it at exit. In between,
rethook_trampoline_handler() may schedule via preempt_enable_notrace();
if the task migrates to another CPU, the frame's $r21 names the old
CPU's percpu base, and restoring it poisons $r21 on the new CPU. Until
the next user->kernel transition heals $r21, this_cpu_*() accesses
(runqueues, RCU per-CPU data, timer tick programming, FPU ownership)
hit the wrong CPU's percpu area.
Under kretprobe-heavy preemptible load this corrupts scheduler and
timer state: scheduling-while-atomic splats, wrong-CPU RCU warnings,
WARN_ON_ONCE(rq != this_rq()) in nohz_balance_exit_idle(), and CPUs
parking in the idle loop with the constant timer never re-armed (hard
lockup). Reproduces on a Loongson-3A6000 with kretprobes on VFS paths
plus heavy file churn (OS install / unsquashfs).
By convention $r21 always holds the current CPU's percpu base in kernel
mode: exception entries reload it only when coming from user mode, and
RESTORE_SOME() restores it only when returning to user mode; the
context-switch path never writes it. The live $r21 at trampoline exit
is therefore already correct, and nothing in between can legitimately
change it (kernel C code cannot write a global register variable). The
same flaw existed in the pre-rethook kretprobe trampoline since v6.3;
it was carried over when rethook replaced it. Drop both the save and
the restore.
Fixes: 3f5536860086d ("LoongArch: Add kretprobes support")
Cc: [email protected] # v6.5+
Assisted-by: Kimi:Kimi-K3 # debug and root-cause analysis
Signed-off-by: Wentao Guan <[email protected]>
---
arch/loongarch/kernel/rethook_trampoline.S | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/arch/loongarch/kernel/rethook_trampoline.S b/arch/loongarch/kernel/rethook_trampoline.S
index 2e009fbea53f2..5efe0268b3143 100644
--- a/arch/loongarch/kernel/rethook_trampoline.S
+++ b/arch/loongarch/kernel/rethook_trampoline.S
@@ -24,7 +24,12 @@
cfi_st t6, PT_R18
cfi_st t7, PT_R19
cfi_st t8, PT_R20
- cfi_st u0, PT_R21
+ /*
+ * $r21 ($u0, percpu base) is deliberately not saved/restored: in
+ * kernel mode it must always hold the current CPU's percpu base,
+ * and restoring it from the frame would poison it with the old
+ * CPU's base if the handler scheduled and we migrated.
+ */
cfi_st fp, PT_R22
cfi_st s0, PT_R23
cfi_st s1, PT_R24
@@ -59,7 +64,7 @@
cfi_ld t6, PT_R18
cfi_ld t7, PT_R19
cfi_ld t8, PT_R20
- cfi_ld u0, PT_R21
+ /* $r21 not restored; see comment in save_all_base_regs. */
cfi_ld fp, PT_R22
cfi_ld s0, PT_R23
cfi_ld s1, PT_R24
--
2.30.2