Re: [PATCH] drm/amdgpu/mes: tear down the rs64mem bitmap in sw_fini
"Chen, Michael" <[email protected]> Tue, 4 Aug 2026 19:14:21 +0000
| Newsgroups | org.freedesktop.lists.amd-gfx |
|---|---|
| Message-ID | <DM6PR12MB361149752AE3CA5C022B41C194D42@DM6PR12MB3611.namprd12.prod.outlook.com> |
--_000_DM6PR12MB361149752AE3CA5C022B41C194D42DM6PR12MB3611namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable AMD General Reviewed-by: Michael Chen <[email protected]> ________________________________ From: Liang, Prike <[email protected]> Sent: Tuesday, August 4, 2026 3:49 AM To: [email protected] <[email protected]> Cc: Deucher, Alexander <[email protected]>; Koenig, Christian <Chri= [email protected]>; Chen, Michael <[email protected]>; Liang, Prike <= [email protected]> Subject: [PATCH] drm/amdgpu/mes: tear down the rs64mem bitmap in sw_fini It's more reasonable to tear down the rs64mem bitmap software context at MES sw_fini phase. This also can avoid the following lock order issue. [ 2529.150983] kworker/u64:1/3134 is trying to acquire lock: [ 2529.151206] ffff8aebd9a3fa10 (reservation_ww_class_mutex){+.+.}-{4:4}, a= t: amdgpu_bo_free_kernel+0x4e/0x130 [amdgpu] [ 2529.151989] but task is already holding lock: [ 2529.152227] ffff8aebd2411648 (&reset_domain->sem){++++}-{4:4}, at: amdgp= u_device_lock_reset_domain+0x20/0x30 [amdgpu] [ 2529.153008] which lock already depends on the new lock. [ 2529.153342] the existing dependency chain (in reverse order) is: [ 2529.153645] -> #2 (&reset_domain->sem){++++}-{4:4}: [ 2529.153913] down_read+0x4a/0x240 [ 2529.154081] amdgpu_userq_destroy+0xd8/0x3a0 [amdgpu] [ 2529.154635] amdgpu_userq_ioctl+0x3e2/0xe30 [amdgpu] [ 2529.155157] drm_ioctl_kernel+0xaf/0x110 [drm] [ 2529.155401] drm_ioctl+0x290/0x510 [drm] [ 2529.155612] amdgpu_drm_ioctl+0x52/0x90 [amdgpu] [ 2529.156068] __x64_sys_ioctl+0xa0/0xf0 [ 2529.156249] x64_sys_call+0x1278/0x21c0 [ 2529.156434] do_syscall_64+0xbe/0x5f0 [ 2529.156611] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 2529.156842] -> #1 (&userq_mgr->userq_mutex){+.+.}-{4:4}: [ 2529.157127] __mutex_lock+0xb2/0x11a0 [ 2529.157304] mutex_lock_nested+0x1f/0x30 [ 2529.157492] amdgpu_userq_vm_validate_and_restore_queue+0x710/0x9b= 0 [amdgpu] [ 2529.158098] amdgpu_userq_restore_worker+0x39/0x290 [amdgpu] [ 2529.158646] process_one_work+0x23e/0x6f -> #0 (reservation_ww_class_mutex){+.+.}-{4:4}: [ 2529.159832] __lock_acquire+0x14c4/0x2210 [ 2529.160021] lock_acquire+0xc6/0x310 [ 2529.160193] __ww_mutex_lock.constprop.0+0xd9/0x1a30 [ 2529.160419] ww_mutex_lock+0x40/0xb0 [ 2529.160590] amdgpu_bo_free_kernel+0x4e/0x130 [amdgpu] [ 2529.161077] amdgpu_mes_rs64mem_fini+0x35/0x60 [amdgpu] [ 2529.161601] mes_v11_0_suspend+0x2d/0x40 [amdgpu] [ 2529.162102] amdgpu_ip_block_suspend+0x2b/0x70 [amdgpu] [ 2529.162609] amdgpu_device_ip_suspend_phase2+0xa4/0x270 [amdgpu] [ 2529.163116] amdgpu_device_pre_asic_reset+0x130/0x1f0 [amdgpu] [ 2529.163617] amdgpu_device_asic_reset+0x55/0x530 [amdgpu] [ 2529.164101] amdgpu_device_gpu_recover+0x1e5/0x410 [amdgpu] [ 2529.164592] amdgpu_userq_mgr_reset_work+0x80/0xa0 [amdgpu] [ 2529.165140] amdgpu_userq_hang_detect_work+0xbb/0x1c0 [amdgpu] [ 2529.165693] process_one_work+0x23e/0x6f0 [ 2529.165884] worker_thread+0x1c4/0x380 [ 2529.166065] kthread+0x10c/0x150 [ 2529.166226] ret_from_fork+0x314/0x390 [ 2529.166408] ret_from_fork_asm+0x1a/0x30 [ 2529.166595] other info that might help us debug this: [ 2529.166917] Chain exists of: reservation_ww_class_mutex --> &userq_mgr->userq_mutex -->= &reset_domain->sem Signed-off-by: Prike Liang <[email protected]> --- drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 4 ++++ drivers/gpu/drm/amd/amdgpu/mes_v11_0.c | 4 ---- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c b/drivers/gpu/drm/amd/= amdgpu/amdgpu_mes.c index 5998e05ecd79..b96f94e5169f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c @@ -308,8 +308,12 @@ void amdgpu_mes_fini(struct amdgpu_device *adev) amdgpu_mes_doorbell_free(adev); + if (adev->mes.use_rs64mem) + amdgpu_mes_rs64mem_fini(&adev->mes); + ida_destroy(&adev->mes.doorbell_ida); mutex_destroy(&adev->mes.mutex_hidden); + } int amdgpu_mes_suspend(struct amdgpu_device *adev, u32 xcc_id) diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/a= mdgpu/mes_v11_0.c index 37985c37be18..31722bb10038 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c @@ -2038,10 +2038,6 @@ static int mes_v11_0_hw_init(struct amdgpu_ip_block = *ip_block) static int mes_v11_0_hw_fini(struct amdgpu_ip_block *ip_block) { - struct amdgpu_device *adev =3D ip_block->adev; - - if (adev->mes.use_rs64mem) - amdgpu_mes_rs64mem_fini(&adev->mes); return 0; } -- 2.34.1 --_000_DM6PR12MB361149752AE3CA5C022B41C194D42DM6PR12MB3611namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo= ttom:0;} </style> </head> <body dir=3D"ltr"> <div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo= nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c= olor: black;"> <div> <div style=3D"font-family: Calibri; text-align: left; color: rgb(0, 0, 255)= ; margin-left: 5pt; font-size: 10pt;"> AMD General</div> <br> </div> Reviewed-by: Michael Chen <[email protected]></div> <div id=3D"appendonsend"></div> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st= yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Liang, Prike <Prik= [email protected]><br> <b>Sent:</b> Tuesday, August 4, 2026 3:49 AM<br> <b>To:</b> [email protected] <[email protected]&= gt;<br> <b>Cc:</b> Deucher, Alexander <[email protected]>; Koenig, Ch= ristian <[email protected]>; Chen, Michael <Michael.Chen@am= d.com>; Liang, Prike <[email protected]><br> <b>Subject:</b> [PATCH] drm/amdgpu/mes: tear down the rs64mem bitmap in sw_= fini</font> <div> </div> </div> <div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;= "> <div class=3D"PlainText">It's more reasonable to tear down the rs64mem bitm= ap software<br> context at MES sw_fini phase. This also can avoid the following<br> lock order issue.<br> <br> [ 2529.150983] kworker/u64:1/3134 is trying to acquire lock:<br> [ 2529.151206] ffff8aebd9a3fa10 (reservation_ww_class_mutex){+.+.}-{4:4}, a= t: amdgpu_bo_free_kernel+0x4e/0x130 [amdgpu]<br> [ 2529.151989]<br> &nb= sp; but task is already holding lock:<br> [ 2529.152227] ffff8aebd2411648 (&reset_domain->sem){++++}-{4:4}, at= : amdgpu_device_lock_reset_domain+0x20/0x30 [amdgpu]<br> [ 2529.153008]<br> &nb= sp; which lock already depends on the new lock.<br> <br> [ 2529.153342]<br> &nb= sp; the existing dependency chain (in reverse order) is:<br> [ 2529.153645]<br> &nb= sp; -> #2 (&reset_domain->sem){++++}-{4:4}:<br> [ 2529.153913] down_read+0x4a/0x2= 40<br> [ 2529.154081] amdgpu_userq_destr= oy+0xd8/0x3a0 [amdgpu]<br> [ 2529.154635] amdgpu_userq_ioctl= +0x3e2/0xe30 [amdgpu]<br> [ 2529.155157] drm_ioctl_kernel+0= xaf/0x110 [drm]<br> [ 2529.155401] drm_ioctl+0x290/0x= 510 [drm]<br> [ 2529.155612] amdgpu_drm_ioctl+0= x52/0x90 [amdgpu]<br> [ 2529.156068] __x64_sys_ioctl+0x= a0/0xf0<br> [ 2529.156249] x64_sys_call+0x127= 8/0x21c0<br> [ 2529.156434] do_syscall_64+0xbe= /0x5f0<br> [ 2529.156611] entry_SYSCALL_64_a= fter_hwframe+0x76/0x7e<br> [ 2529.156842]<br> &nb= sp; -> #1 (&userq_mgr->userq_mutex){+.+.}-{4:4}:<br> [ 2529.157127] __mutex_lock+0xb2/= 0x11a0<br> [ 2529.157304] mutex_lock_nested+= 0x1f/0x30<br> [ 2529.157492] amdgpu_userq_vm_va= lidate_and_restore_queue+0x710/0x9b0 [amdgpu]<br> [ 2529.158098] amdgpu_userq_resto= re_worker+0x39/0x290 [amdgpu]<br> [ 2529.158646] process_one_work+0= x23e/0x6f<br> &nb= sp; -> #0 (reservation_ww_class_mutex){+.+.}-{4:4}:<br> [ 2529.159832] __lock_acquire+0x1= 4c4/0x2210<br> [ 2529.160021] lock_acquire+0xc6/= 0x310<br> [ 2529.160193] __ww_mutex_lock.co= nstprop.0+0xd9/0x1a30<br> [ 2529.160419] ww_mutex_lock+0x40= /0xb0<br> [ 2529.160590] amdgpu_bo_free_ker= nel+0x4e/0x130 [amdgpu]<br> [ 2529.161077] amdgpu_mes_rs64mem= _fini+0x35/0x60 [amdgpu]<br> [ 2529.161601] mes_v11_0_suspend+= 0x2d/0x40 [amdgpu]<br> [ 2529.162102] amdgpu_ip_block_su= spend+0x2b/0x70 [amdgpu]<br> [ 2529.162609] amdgpu_device_ip_s= uspend_phase2+0xa4/0x270 [amdgpu]<br> [ 2529.163116] amdgpu_device_pre_= asic_reset+0x130/0x1f0 [amdgpu]<br> [ 2529.163617] amdgpu_device_asic= _reset+0x55/0x530 [amdgpu]<br> [ 2529.164101] amdgpu_device_gpu_= recover+0x1e5/0x410 [amdgpu]<br> [ 2529.164592] amdgpu_userq_mgr_r= eset_work+0x80/0xa0 [amdgpu]<br> [ 2529.165140] amdgpu_userq_hang_= detect_work+0xbb/0x1c0 [amdgpu]<br> [ 2529.165693] process_one_work+0= x23e/0x6f0<br> [ 2529.165884] worker_thread+0x1c= 4/0x380<br> [ 2529.166065] kthread+0x10c/0x15= 0<br> [ 2529.166226] ret_from_fork+0x31= 4/0x390<br> [ 2529.166408] ret_from_fork_asm+= 0x1a/0x30<br> [ 2529.166595]<br> &nb= sp; other info that might help us debug this:<br> <br> [ 2529.166917] Chain exists of:<br> &nb= sp; reservation_ww_class_mutex --> &userq_mgr->= userq_mutex --> &reset_domain->sem<br> <br> Signed-off-by: Prike Liang <[email protected]><br> ---<br> drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 4 ++++<br> drivers/gpu/drm/amd/amdgpu/mes_v11_0.c | 4 ----<br> 2 files changed, 4 insertions(+), 4 deletions(-)<br> <br> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c b/drivers/gpu/drm/amd/= amdgpu/amdgpu_mes.c<br> index 5998e05ecd79..b96f94e5169f 100644<br> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c<br> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c<br> @@ -308,8 +308,12 @@ void amdgpu_mes_fini(struct amdgpu_device *adev)<br> <br> amdgpu_mes_doorbell_free(a= dev);<br> <br> + if (adev->mes.use_rs64mem)<br> + &n= bsp; amdgpu_mes_rs64mem_fini(&adev->mes);<br> +<br> ida_destroy(&adev->= mes.doorbell_ida);<br> mutex_destroy(&adev-&g= t;mes.mutex_hidden);<br> +<br> }<br> <br> int amdgpu_mes_suspend(struct amdgpu_device *adev, u32 xcc_id)<br> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/a= mdgpu/mes_v11_0.c<br> index 37985c37be18..31722bb10038 100644<br> --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c<br> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c<br> @@ -2038,10 +2038,6 @@ static int mes_v11_0_hw_init(struct amdgpu_ip_block = *ip_block)<br> <br> static int mes_v11_0_hw_fini(struct amdgpu_ip_block *ip_block)<br> {<br> - struct amdgpu_device *adev =3D ip_blo= ck->adev;<br> -<br> - if (adev->mes.use_rs64mem)<br> - &n= bsp; amdgpu_mes_rs64mem_fini(&adev->mes);<br> return 0;<br> }<br> <br> -- <br> 2.34.1<br> <br> </div> </span></font></div> </body> </html> --_000_DM6PR12MB361149752AE3CA5C022B41C194D42DM6PR12MB3611namp_--