Re: [PATCH v4 02/21] iommufd: Add iommufd_sw_map_msi()

On Mon, Aug 24, 2026 at 09:52:04AM -0300, Jason Gunthorpe wrote:
> On Mon, Aug 24, 2026 at 10:50:07AM +0200, Andrew Jones wrote:
> > On Sat, Aug 22, 2026 at 11:06:25AM -0300, Jason Gunthorpe wrote:
> > > On Sat, Aug 22, 2026 at 03:50:05PM +0200, Andrew Jones wrote:
> > > > With the RISC-V IOMMU MSI table and irqbypass support in KVM, we can
> > > > write the guest's MSI messages directly to the device without
> > > > interpreting the IOVAs. 
> > > 
> > > Every architecture can work this way. Nobody has impleented Linux
> > > support for it, and if we do, it must be arch generic.
> > > 
> > > Not being able to discover the vCPU from the MSI descriptor means you
> > > can't use any of the existing less-optimal Linux flows and are forced
> > > to implement this hard thing..
> > 
> > The RISC-V irqbypass work does not require a new framework, or even
> > callback. It uses the existing irqbypass framework and implements
> > irq_set_vcpu_affinity(). Since that callback programs the IOMMU MSI table,
> > it belongs in the IOMMU irqdomain.
> 
> My point is you can't use the normal flow which relies on Linux fully
> programming the MSI-X on the host side and only uses the guest
> descriptor as input to that flow.
> 
> The iommu msi table doesn't actually help unless you can also directly
> use the guest's descriptor. Right?

I see a gap in VFIO's vector management for two-stage device assignment,
but I don't think it's as broad for RISC-V as it is for ARM. More below.

> 
> > > ARM is basically the same, the ITS page goes through the S1 and S2, so
> > > the goal is to get a valid ITS page into the S2, tell the guest to use
> > > it and create a S1 pointing at it then feed the MSI descriptor from
> > > the guest unmodified to the HW.
> > 
> > On ARM, the SMMU translates the MSI address, but the ITS performs the
> > interrupt remapping. KVM must therefore know which vITS the guest IOVA
> > selects so it can resolve that vITS's DeviceID/EventID mapping and program
> > the physical ITS with the corresponding vCPU-specific state.
> 
> Not quite..
> 
> IIRC the MSI entry itself tells which pITS page has to be used since
> they are 1:1. That tells KVM which vGIC has the vITS table so it can
> read that and get the vCPU. Now the IRQ layer has the pGIC, IRQ #, MSI
> # and vCPU. It then creates an entirely new MSI descriptor and
> programs the pITS and pMSI entry. The guest MSI descriptor is thrown
> away.
> 
> It is like this because this is how x86 does things so it is easy to
> implement.
> 
> However in idyllic ARM land it could work like RISC-V. The GIC could
> mirror the vITS table into th pITS table and apply the vCPU to pCPU
> mapping through an irq domain. Then the guest MSI descriptor would be
> used unmodified. These days x86 can be similar, if the guest has the
> iommu remapping HW it could be similarly mirrored into the physical
> remapping HW instead of reconstructing the MSI descriptors.
> 
> This has never been implemented because there are alot of missing
> generic pieces to use an unmodified guest MSI descriptor.
> 
> Now, risc-v doesn't need to mirrow TIS/remapping tables from guest to
> host, because it doesn't have such tables, but it still will need the
> rest of the generic machinery to use unmodified guest MSI descriptors.

With S1 Bare, irqbypass can fall back by restoring the host MSI message.
With guest-controlled S1, VFIO cannot silently switch back to host
delivery. We need plumbing to establish a guest-owned MSI mode, preserve
and mask the guest descriptor across the VFIO vector lifecycle, and stop
or reject the configuration if that mode cannot be maintained.

The guest descriptor itself is already available in
kvm_arch_update_irqfd_routing(), so RISC-V still does not need to
interpret or track the guest IOVA. The IOVA can be written directly to the
device (since the IOMMU MSI table is pre-populated with all vIMSIC GPAs).

> 
> > This ownership of remapping (for ARM in the ITS and for RISC-V in the
> > IOMMU) also justifies why ARM does not need an irqdomain in the IOMMU and
> > RISC-V does.
> 
> Yes, that's right, as I said earlier the irqdomain for the msi
> remapping table seems appropriate.
> 
> > The irqbypass support is already separate from this series. This series
> > only introduces the IOMMU irqdomain, which owns the IOMMU-specific MSI
> > composition now and will provide the irq_set_vcpu_affinity() hook needed
> > later. Putting the compose code in the IMSIC irqdomain now, only to move
> > it into the IOMMU irqdomain with irqbypass later, would add unnecessary
> > churn.
> 
> I'm sure you need more than irq_set_vcpu_affinity() - it still has to
> use the guest descriptor unchanged, and it still has to get the
> correct pages into the S2 at match them to the ACPI/DT description. We
> don't have any common APIs to do that, and every arch can use this
> stuff.

Indeed, irq_set_vcpu_affinity() alone will be insufficient for two-stage
guests due to the VFIO vector lifecycle concerns with guest-owned MSI
descriptors.

RISC-V does not need S2 page-table mappings for MSIs, though. The VM's
IMSIC topology identifies the vIMSIC GPAs used to populate the MSI table,
and an MSI-table match bypasses the normal S2 page table.

> 
> Without seeing the whole thing it is hard to be sure exactly what
> arrangement people are going to want to see. I'm also not sure how to
> combine the iommu MSI handling, unmodified guest MSI descriptor and
> this msiremapping all together. A guest centric flow and an normal
> host flow are very differnt in this model. This is all new.

I agree there is some new terrain to travel here. It'd be great to discuss
this at Plumber's.

Thanks,
drew

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv