Re: [RFC] KVM: Proposed uAPI for querying GSI and irqbypass status

On Tue, Jul 28, 2026 at 2:45 AM Mostafa Saleh <[email protected]> wrote:
>
> Hi Jing,
>
> On Fri, Jul 17, 2026 at 09:26:57AM -0700, Jing Zhang wrote:
> > This RFC proposes a new KVM uAPI to allow a VMM to programmatically
> > query the status of a GSI, with a particular focus on determining
> > the success or failure of Irqbypass.
> >
> >
> > 1. Motivation
> >
> > The primary motivation for this uAPI is to bridge the operational gap
> > between the VMM and KVM at the kernel level regarding interrupt
> > virtualization status:
> >
> > * KVM (Kernel) holds the precise, live runtime status of each GSI (such
> >   as whether Irqbypass is active, pending, or has encountered a setup
> >   error). However, it completely lacks the high-level semantic details
> >   of those GSIs—it does not understand which logical function or
> >   physical hardware device they represent.
> >
> > * The VMM (Userspace) possesses the complete context of the guest's
> >   interrupt topology. It knows exactly which GSI belongs to which
> >   passthrough or emulated device because it orchestrated the setup.
> >   Yet, the VMM has no stable, programmatic way to query the actual
> >   runtime execution status of these GSIs once passed to the kernel.
> >
> > Currently, diagnosing when a high-performance interrupt path falls
> > back to software mediation is highly problematic:
> >
> > * While some architectures (like ARM64) expose virtual ITS states via
> >   debugfs, this implementation is highly architecture-specific and
> >   fragile.
> >
> > * debugfs is not a stable API, is prone to breaking changes across
> >   kernel versions, and is entirely unsuitable for programmatic
> >   production monitoring and fleet-wide observability.
> >
> > This is not an ARM-specific problem; it is a general, cross-architecture
> > limitation within KVM's Irqbypass and interrupt routing subsystem.
> > By introducing an architecture-neutral uAPI, we allow the VMM to
> > marry its topological knowledge with KVM's live execution state.
> >
>
> I am trying to understand the benefit of introducing an new uAPI for
> this. I have been using irqbypass with arm64, and I would debug
> performance problems solely based on /proc/interrutps.
>
> On the host side you see a vector per interrupt and a new vector per
> vcpu for the doorbell interrupt in case the vcpu was not scheduled.
>
> Based on that I can tell if irqbypass was active or not, and in case
> I have /proc/interrupts from the guest, I can get a percentage of
> the fallback software path per-VM.
>
> A uAPI will not give much extra info, see my comment below.
>
> [...]
>
> >
> >
> > /* To be included in <linux/kvm.h> */
> >
> > /*
> >  * KVM_GET_GSI_STATE: Get the status of a specific GSI.
> >  *
> >  * The user passes a pointer to struct kvm_gsi_state. The kernel fills in
> >  * the status flags, performance counters, and the architecture-specific
> >  * union members directly.
> >  */
> >
> > /* Generic flags indicating high-level status */
> >
> > /*
> >  * The GSI's route is theoretically compatible with hardware bypass
> >  * on this host (e.g. is an MSI route and GICv4/VT-d is present).
> >  */
> > #define KVM_GSI_STATE_FLAG_BYPASSABLE      (1 << 0)
> >
> > /* An IRQFD is currently active and bound to this GSI */
> > #define KVM_GSI_STATE_FLAG_HAS_IRQFD       (1 << 1)
> >
> > /* The irqbypass path is fully enabled and active in hardware */
> > #define KVM_GSI_STATE_FLAG_BYPASS_ACTIVE   (1 << 2)
> >
> > /* Example architecture-specific failure reasons for ARM64 */
> > #define KVM_ARM_GSI_FAILURE_REASON_NONE             0
> > #define KVM_ARM_GSI_FAILURE_REASON_NO_ITS           1 /* No ITS/GICv4 support */
> > #define KVM_ARM_GSI_FAILURE_REASON_NO_MSI_ADDR      2 /* MSI address unset */
> > #define KVM_ARM_GSI_FAILURE_REASON_GIC_HW_REJECT    3 /* GIC rejected map */
> > #define KVM_ARM_GSI_FAILURE_REASON_INVALID_STATE    4 /* Guest state block */
> >
> > struct kvm_gsi_state {
> >     __u32 gsi;              /* IN: The GSI to query */
> >     __u32 flags;            /* OUT: High-level status flags */
> >     __u64 counter_success;  /* OUT: Generic counter for successful bypass */
>
> I do not think that is possible on arm64, the host does not know
> about the interrupts directly injected to the guest, so it can not
> maitain such a counter.
>
> Thanks,
> Mostafa
>

Hi Mostafa,

Thank you for taking the time to review this RFC and for your valuable feedback.

To clarify the primary motivation here: the goal of this new uAPI is
to enable programmatic, stable, and fine-grained monitoring of
interrupt bypass status in production environments, where existing
mechanisms like debugfs are simply not viable.

Here are a few key reasons why we believe a dedicated uAPI/ioctl is
necessary rather than relying on debugfs:
1. Production Hardening and Access Restrictions: In many hardened
production or cloud environments, debugfs is completely disabled
(CONFIG_DEBUG_FS=n), unmounted, or heavily restricted due to security
and performance overhead concerns. Relying on it means the Virtual
Machine Monitor (VMM) loses all visibility into interrupt performance
regressions in the very environment where monitoring matters most.
2. Lack of ABI Stability: As you know, debugfs makes no ABI stability
guarantees. Its output format can change between kernel versions
without notice. Basing production monitoring and telemetry pipelines
on scraping debugfs is fragile and prone to breaking during kernel
upgrades.
3. Programmatic Efficiency: Scraping and parsing text files from
userspace is inefficient for high-frequency or fleet-wide monitoring.
A binary ioctl interface (KVM_GET_GSI_STATE) provides a lightweight,
deterministic, and highly scalable way for the VMM to query state on
demand.
4. Marrying Topology with Live State: KVM holds the runtime status of
the bypass, but only userspace (the VMM) understands the full topology
(i.e., which GSI belongs to which passthrough device). This uAPI
allows the VMM to programmatically marry its topological knowledge
with KVM's live execution state without relying on side-band manual
debugging tools.

While interfaces in debugfs are fantastic for manual, retroactive
debugging on a developer workstation, they are not Dependable for
fleet-wide observability. This RFC aims to provide a proactive
monitoring solution that aligns KVM with production operational
requirements across architectures (both ARM and x86).

Would love to hear your thoughts on this perspective.

Thanks,
Jing