[RFC] PCI_IRQ_AFFINITY limits MSI-X allocation on 384 CPU / 1024 devices NVMe system
karunakar rao <[email protected]> Thu, 30 Jul 2026 16:11:13 +0530
| Newsgroups | org.infradead.lists.linux-nvme |
|---|---|
| Message-ID | <CAGB8rXbGnJTR8GdHM2jSpa4XMi0FwWKKhYKr=C1Co3-H-+Qqsg@mail.gmail.com> |
Hello I am trying to create 1024 nvme devices , each device requesting 2 msix vectors. Total we need 1024*2=2048 msix vectors Total number of available device vectors: ~ (200 * 384) = 76800 But seeing following messages: [ 9166.370948] Interrupt reservation exceeds available resources [ 9167.455029] Interrupt reservation exceeds available resources [ 9167.455572] Interrupt reservation exceeds available resources [ 9167.455888] Interrupt reservation exceeds available resources In nvme_setup_irqs, pci_alloc_irq_vectors_affinity disabled PCI_IRQ_AFFINITY flag with that All msix vectors allocated fine. In Linux 6.13, PCI_IRQ_AFFINITY causes MSI-X vectors to receive affinity masks generated through irq_create_affinity_masks() and group_cpus_evenly(). On very large systems with hundreds of CPUs and thousands of devices, these masks can become sufficiently restrictive that vector allocation is limited to a relatively small subset of CPUs. When those CPUs become vector-constrained, some NVMe devices receive fewer MSI-X vectors or fall back to shared interrupt usage. Disabling PCI_IRQ_AFFINITY removes these affinity constraints and allows the allocator to place vectors more flexibly. thanks