[RFC] PCI_IRQ_AFFINITY limits MSI-X allocation on 384 CPU / 1024 devices NVMe system

karunakar rao <[email protected]> Thu, 30 Jul 2026 16:11:13 +0530
Newsgroups org.infradead.lists.linux-nvme
Message-ID <CAGB8rXbGnJTR8GdHM2jSpa4XMi0FwWKKhYKr=C1Co3-H-+Qqsg@mail.gmail.com>
Hello
I am trying to create 1024 nvme devices , each device requesting 2 msix vectors.
Total we need 1024*2=2048 msix vectors
Total number of available device vectors: ~ (200 * 384) = 76800
But seeing following messages:
  [ 9166.370948] Interrupt reservation exceeds available resources
  [ 9167.455029] Interrupt reservation exceeds available resources
  [ 9167.455572] Interrupt reservation exceeds available resources
  [ 9167.455888] Interrupt reservation exceeds available resources


In nvme_setup_irqs, pci_alloc_irq_vectors_affinity disabled
PCI_IRQ_AFFINITY flag with that
All msix vectors allocated fine.

In Linux 6.13, PCI_IRQ_AFFINITY causes MSI-X vectors to receive
affinity masks generated through irq_create_affinity_masks() and
group_cpus_evenly(). On very large systems with hundreds of CPUs and
thousands of devices, these masks can       become sufficiently
restrictive that vector allocation is limited to a relatively small
subset of CPUs. When those CPUs become vector-constrained, some NVMe
devices receive fewer MSI-X vectors or fall back to shared interrupt
usage. Disabling  PCI_IRQ_AFFINITY removes these affinity constraints
and allows the allocator to place vectors more flexibly.

thanks