Re: [PATCH 05/14] drm/amdgpu: add an buffer funcs callback for TLB invalidation

On 7/15/26 15:44, Alex Deucher wrote:
> On Wed, Jul 15, 2026 at 5:00 AM Christian König
> <[email protected]> wrote:
>>
>> On 7/14/26 17:19, Alex Deucher wrote:
>>>> Why are we using the MES for per VMID invalidation?
>>>
>>> gfxoff.  We can't access the registers directly unless we disallow gfxoff.
>>
>> Well there is a remark that VM_INVALIDATE_ENG*_ACK is always save to read, even when gfxoff is active.
>>
>> And we explicitly acquire the semaphore to avoid that GFXOFF kicks in.
> 
> I'm pretty sure the semaphore just keeps it active if it's already
> active, but the registers are in the gfx tile so they are gfxoff
> controlled.  I think the semaphore is just meant to be used as a lock
> to keep gfxoff from kicking in in the middle of an invalidation.

What exactly is the failure you are seeing?

What could potentially happen is that GFXOFF suspended the block and we are not able to acquire the semaphore, but in that care we could also simply ignore the invalidation request.

>>
>> I also don't see how we use the MES here?
>>
>> What we do is to use the KIQ on gfx9, but that is actually only as a workaround for some SRIOV problems.
>>
> 
> We do it for everything gfx9 and newer.  For example gmc_v11_0_flush_gpu_tlb():
> 
>         /* This is necessary for SRIOV as well as for GFXOFF to
> function
>          * properly under bare metal
>          */
>         if ((adev->gfx.kiq[0].ring.sched.ready ||
> adev->mes.ring[0].sched.ready) &&
>             (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev))) {
>         amdgpu_gmc_fw_reg_write_reg_wait(adev, req, ack, inv_req,
>                                                  1 << vmid, GET_INST(GC, 0));
>                 return;
>         }
> 
> The only time we use the MMIO path is if KIQ or MES is not ready yet.
> The goal of this patch set was to switch to using SDMA rather than MES
> and to clean up the gmc code since most of the logic is the same for
> gfx9 and above.

We should probably stop doing this. As far as I can see that should be completely unnecessary.

If we really want to move this to the SDMA we should do it a level higher in amdgpu_gmc_flush_gpu_tlb().

Regards,
Christian.

> 
> Alex
> 
> 
>> Christian.
>>
>>>
>>> Alex