[PATCH 0/5] drm/amdgpu: fix userq crash and lockdep violations during GPU reset

From: Vitaly Prosyak <[email protected]>

This series fixes two bugs exposed by running the amd_basic IGT test with
AMDGPU_ENABLE_USERQTEST=1 on a GFX12 GPU with CONFIG_LOCKDEP=y enabled.

Bug 1: kernel BUG at drivers/dma-buf/dma-resv.c:319
-----------------------------------------------------
amdgpu_evf_mgr_attach_fence() calls ttm_bo_validate() then
dma_resv_add_fence() without reserving a fence slot first.
ttm_bo_validate() consumes the pre-allocated slots for its own fences,
leaving none for the eviction fence:

  kernel BUG at drivers/dma-buf/dma-resv.c:319!
  BUG_ON(fobj->num_fences >= fobj->max_fences)
  RIP: dma_resv_add_fence+0x294/0x2b0
  Workqueue: events amdgpu_userq_restore_worker

Patch 1 adds the missing dma_resv_reserve_fences(resv, 1) call.

Bug 2: four lockdep violations during GPU reset (CONFIG_LOCKDEP=y)
-------------------------------------------------------------------
With lockdep enabled a MES timeout during cs-compute-fwm-UMQ triggers
a GPU reset, exposing four lock ordering violations. All involve
reset_domain->sem being held while other locks that must come before it
in the established ordering are acquired.

The violations (each addressed by one patch):

1. halt_activities() -> drm_client_dev_suspend() takes clientlist_mutex
   while reset_domain->sem is already held. Fixed in patch 3 by moving
   the reset lock acquisition to after halt_activities().

2. psp_resume() takes firmware.mutex while reset_domain->sem is held.
   Fixed in patch 4 by skipping firmware.mutex in reset context, since
   down_write(reset_domain->sem) already provides exclusive protection.

3. amdgpu_device_reinit_after_reset() calls drm_client_dev_resume()
   which takes clientlist_mutex while reset_domain->sem is held. Fixed
   in patch 5 by moving the call to after recovery_put_reset_lock().

4. amdgpu_userq_destroy() calls cancel_delayed_work_sync() inside
   userq_mutex; the work itself takes userq_mutex. Fixed in patch 2 by
   moving the cancel before mutex_lock().

After all 5 patches the amd_basic UMQ subtests pass with no lockdep
warnings on a CONFIG_LOCKDEP=y / CONFIG_PROVE_LOCKING=y kernel.

Tested-on: GFX12 (Radeon RX 9070 XT / device 0x7550)
           IGT amd_basic with AMDGPU_ENABLE_USERQTEST=1
           Kernel: 7.1.0+, CONFIG_LOCKDEP=y, CONFIG_PROVE_LOCKING=y

Vitaly Prosyak (5):
  drm/amdgpu: reserve dma_resv slot before adding eviction fence
  drm/amdgpu: cancel hang_detect_work before taking userq_mutex
  drm/amdgpu: take reset lock after halt_activities
  drm/amdgpu/psp: skip firmware.mutex in psp_resume during GPU reset
  drm/amdgpu: move drm_client_dev_resume outside reset_domain->sem

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c         |  9 ++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_eviction_fence.c |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c            | 10 ++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c          |  4 +++-
 4 files changed, 17 insertions(+), 8 deletions(-)

Vitaly Prosyak (5):
  drm/amdgpu: reserve dma_resv slot before adding eviction fence
  drm/amdgpu: cancel hang_detect_work before taking userq_mutex
  drm/amdgpu: take reset lock after halt_activities
  drm/amdgpu/psp: skip firmware.mutex in psp_resume during GPU reset
  drm/amdgpu: move drm_client_dev_resume outside reset_domain->sem

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 21 ++++++++++++++-----
 .../drm/amd/amdgpu/amdgpu_eviction_fence.c    |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 13 +++++++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c     |  5 ++++-
 4 files changed, 32 insertions(+), 9 deletions(-)

-- 
2.54.0