RE: [RFC PATCH 2/3] drm/amdgpu: Add VM ioctl handlers for second-level trap handler

AMD General

> -----Original Message-----
> From: Natalie Vock <[email protected]>
> Sent: Thursday, August 20, 2026 4:08 PM
> To: SHANMUGAM, SRINIVASAN <[email protected]>;
> Koenig, Christian <[email protected]>; Deucher, Alexander
> <[email protected]>
> Cc: [email protected]; Kuehling, Felix <[email protected]>;
> Zhu, James <[email protected]>; Lazar, Lijo <[email protected]>; Six,
> Lancelot <[email protected]>; Pelloux-Prayer, Pierre-Eric <Pierre-
> [email protected]>; Timur Kristóf <[email protected]>; Samuel
> Pitoiset <[email protected]>; Natalie Vock <[email protected]>
> Subject: Re: [RFC PATCH 2/3] drm/amdgpu: Add VM ioctl handlers for second-level
> trap handler
>
> On 8/20/26 09:01, Srinivasan Shanmugam wrote:
> > Render-node user queues had no mechanism to install a userspace
> > second-level trap handler equivalent to what KFD provides for compute.
> > Add it via the VM ioctl and fix several correctness issues in the CWSR
> > TMA access path.
> >
> > On dGPUs the CWSR TMA buffer is placed in VRAM and accessed via MMIO
> > (PCI BAR). Raw CPU pointer dereferences are unsafe on platforms that
> > lack full load/store support for MMIO. Replace void *tma_cpu_addr with
> > struct iosys_map tma_map and use iosys_map_wr() for all TMA writes.
> >
> > Writing TBA and TMA as two separate stores creates a race where a
> > trapping wave can observe a mismatched pair. Evict all user queues and
> > flush the GPU TLB before writing, then restore queues after. Use
> > amdgpu_vm_wait_idle() on the CLEAR_L2_TRAP path to also drain kernel
> > queue VM operations before zeroing TBA/TMA.
>
> Once you properly track the TBA/TMA buffers as used like I suggested in my
> previous email, this dance becomes unnecessary. You should then be able to reuse
> how VM unmap operations sync to everything that may use the BO.

Hi Natalie,

Thanks — noted. Once the BO handles are properly tracked, the evict/restore
dance in SET_L2_TRAP and CLEAR_L2_TRAP will also be removed
and replaced with proper BO fence tracking as you suggested
in your previous email.

Thanks,
Srini

>
> This also takes care of the concern Timur brought up about handling kernel queues
> properly.
>
> Thanks,
> Natalie