[PATCH] drm/amdgpu: drain SDMA before process eviction during GPU reset

Host kernel panics (Oops in free_swap_cache) when GPU reset is
triggered under multi-process load. The crash is caused by SDMA engines
writing to memory while munmap clears page table entries during process
eviction.

The race condition:
amdgpu_device_halt_activities() loops over all devices in the reset
list and calls amdgpu_amdkfd_pre_reset() on each device. The first call
to amdgpu_amdkfd_pre_reset() triggers global process eviction across
all GPUs via kgd2kfd_suspend_process(), which starts munmap clearing
PTEs. However, SDMA engines on other devices in the reset list still
have in-flight DMA transactions that have not been drained.

Fix by draining ALL SDMA rings on ALL devices in the reset list BEFORE
calling amdgpu_amdkfd_pre_reset() on any device. Also flush the HDP
write cache to ensure SDMA writes reach system memory before PTEs are
cleared. This guarantees no in-flight DMAs exist when process eviction
begins.

Signed-off-by: Sunday Clement <[email protected]>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 +++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 472e96ae884e..0cd3dc4cffb2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5340,7 +5340,36 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
 					  bool need_emergency_restart)
 {
 	struct amdgpu_device *tmp_adev = NULL;
-	int i;
+	int i, r;
+	struct amdgpu_ring *ring;
+
+    /*
+     * Drain all SDMA on all devices before any KFD pre_reset.
+     *
+     * amdgpu_amdkfd_pre_reset() triggers global process eviction across
+     * all GPUs, which starts munmap clearing PTEs. We must ensure no
+     * in-flight SDMA DMAs on any device when eviction begins, otherwise
+     * SDMA writes race with munmap, corrupting PTEs.
+     */
+	list_for_each_entry(tmp_adev, device_list, reset_list) {
+		for (i = 0; i < tmp_adev->sdma.num_instances; i++) {
+			ring = &tmp_adev->sdma.instance[i].ring;
+			if (ring->sched.timeout == MAX_SCHEDULE_TIMEOUT)
+				continue;
+
+			r = amdgpu_fence_wait_empty(ring);
+			if (r)
+				dev_warn(tmp_adev->dev,
+					"SDMA%d ring drain timeout before reset, proceeding\n", i);
+		}
+
+		/*
+		 * Flush HDP write cache to ensure SDMA writes reach system
+		 * memory before munmap. Prevents cached writes from arriving
+		 * after pages are freed.
+		 */
+		amdgpu_device_flush_hdp(tmp_adev, NULL);
+	}
 
 	/* block all schedulers and reset given job's ring */
 	list_for_each_entry(tmp_adev, device_list, reset_list) {
-- 
2.43.0