Re: [PATCH v8 12/23] dma: swiotlb: pass mapping attributes by reference

On 07/08/2026 6:01 pm, Jason Gunthorpe wrote:
> On Fri, Aug 07, 2026 at 04:54:35PM +0100, Robin Murphy wrote:
> 
>>> We will need to teach GIC to understand if/when the device uses a T=0
>>> translation for MSI and then use a shared physical address for the ITS
>>> IOVA instead of getting an IOVA translation from the iommu. The
>>> hypervisor will setup the S2 for the T=0 SMMU translation to be
>>> identity with all shared memory and the ITS page will be shared
>>> memory.
>>
>> IIRC, for MSI or unlocked MSI-X it should look and work pretty much exactly
>> like regular VFIO, as everything can be mediated by the VMM and host kernel.
> 
> Right
> 
>>> Presumably in future we will have HW to handle a T=1 ITS page access
>>> and some way to negotiate with devices if they should use a T=1 path
>>> for MSI or not.
>>
>> Locked MSI-X would be a pain right now as we cannot intercept the Realm
>> programming the MSI-X cap with a doorbell address and EventID value decided
>> by the ITS driver in the Realm guest, so we'd somehow have to sniff those
>> values out of the VMM's vITS emulation then try to configure an equivalent
>> NS LPI to match, or have a hook in the ITS driver that knows when it's in a
>> Realm and do some RSI handshake to proxy-allocate NS MSI vectors on the host
>> and pass the real values back into the Realm.
> 
> Yes, we'd need to move to a model where the guest programs MSI
> directly and we learn the configuration required to emulate through
> the vGIC, not through MSI trapping. This broadly is the "righter" way
> to do interrupt routing but it will be hard to get there, if ever.
> 
>> I'm still hoping we (both CCA and possibly Linux in general) can get away
>> with just refusing to support Locked MSI-X without GICv5, as once we have
>> proper Realm MSIs with direct injection then all the problems go away (or at
>> least become much smaller RMM problems that remain invisible to the host -
>> the one "big" problem being that the RMM has to begrudgingly implement an
>> entire GIC driver since it now has its own whole GIC to look after).
> 
> Yeah, I'm fine with this, if GICv5 can allow direct MSI-X programming
> then great.
> 
> Devices are going to need some way to negotiate if MSI-X is locked, and
> if MSI is T=1 or not, I don't know if PCI has something for that
> already..
> 
>>> There won't be a vSMMU attached to the T=0 instance at all, it is just
>>> wired to be bypass.
>>
>> If that's all anyone will ever want then it does allow some degree of
>> hackery like mirroring the whole of Realm S2 in an equivalent NS IOMMUFD
>> domain, then forcing IOMMU_DOMAIN_IDENTITY for unaccepted devices within the
>> guest. However I was under the impression that folks want to make meaningful
>> use of devices while still in their untrusted pre-acceptance state (but
>> maybe still switch them later), so it seems almost inevitable that
>> eventually someone says "actually, we would like S1 vSMMU for untrusted
>> scatter-gather as well..."
> 
> So far almost all VMs today don't use vIOMMU at all, the ones that do
> are using it for things like PASID (or interrupt remapping on x86),
> not for translation. The cases where a vIOMMU is deliberately needed
> for translation seem to be mostly around SVA and PASID which isn't
> going to meaningfully work out of the box on a T=0 device.
> 
> My prediction is this is fine.
> 
> At least it is sufficiently hard to make two parallel vSMMU's
> controlling the same PCI device, and to make the ITS routing also
> somehow work right, that it isn't worth doing at this point when there
> is so much other more basic stuff to get done.

Cool. So in fact that puts us in an interesting position for now where 
non-CoCo "untrusted" (i.e. external) devices should have IOMMU 
translation forced on by default, while CoCo "unaccepted" devices (i.e. 
those which do have a mechanism to transition into a T=1 or equivalent 
state) should *not* try to use an associated IOMMU, on the assumption 
that it may only work for T=1 traffic. All the more reason to sort these 
abstractions out so we can make the right distinctions clearly :)

(And while untrusted vIOMMUs for purely-untrusted devices in CoCo 
environments would be pretty straightforward as well, I guess we might 
need some sort of acceptance status for trusted vIOMMU devices 
themselves? Hmm...)

Cheers,
Robin.

> I imagine the ACPI might someday gain a description of the T=0 vSMMU
> in a way that is invisible to todays Linux and a future Linux could
> understand how to juggle the two iommu drivers for the same struct
> device, somehow.
> 
> Regards,
> Jason