Re: [REGRESSION] iommu/amd: "Fix premature break in init_iommu_one()" causes early boot hang
Vasant Hegde <[email protected]>
| Newsgroups | dev.linux.lists.regressions,dev.linux.lists.iommu,org.kernel.vger.linux-kernel |
|---|---|
| Message-ID | <[email protected]> |
Hl Andreas, On 8/22/2026 7:02 PM, Andreas Juch wrote: > [Some people who received this message don't often get email from > [email protected]. Learn why this is important at https://aka.ms/ > LearnAboutSenderIdentification ] > > Hello Vasant, > > On 8/22/26 12:17, Vasant Hegde wrote: >> Andreas, >> >> On 8/22/2026 11:19 AM, Joerg Roedel wrote: >>> Adding Vasant. >>> >>> On Fri, Aug 21, 2026 at 11:11:07PM +0200, Andreas Juch wrote: >>>> Hello, >>>> >>>> I noticed that one of my NixOS sytems failed to boot with kernel 6.18.40. >>>> The >>>> older 6.18.39 booted fine. A full git bisect of v6.18.39..v6.18.40 >>>> identified >>>> the following commit as the first bad commit: >>>> >>>> dc266f6c4e2628576ab33042b6b2a12480220026 >>>> iommu/amd: Fix premature break in init_iommu_one() >>>> >>>> The visible symptom is a black screen immediately after systemd-boot. The >>>> machine also remains unreachable over SSH and leaves no persistent journal >>>> entry >>>> for the failed boot. A reset or power cycle is required. >> Interesting. I never imagined above fix will break some system! >> >> I assumed below dmesg is from good boot (without above patch) and did the >> analysis (code walk with below IVRS data): > > Yes, that's correct, it's from the .40 kernel without commit dc266f6c. > >> - EFR advertised by IVRS is different from actual EFR (read by IOMMU MMIO >> register). Its worse because IVRS advertised few features like GT/PPR which is >> not supported by IOMMU. >> - Without this patch we would have skipped early_iommu_features_init() so >> global EFR is zero and then init_iommu_one_late() set the EFR properly based on >> actual IOMMU capability. >> >> Only with this log, its hard to tell where exactly system failed. My guess is >> driver enabled features which are not supported (like GT/PPR) or we setup 32bit >> interrupt remapping table. >> >> I believe above patch exposed an existing bug with some other commit. >> >> >> Ideally we should properly fix late_iommu_features_init() and use actual EFRs. >> But for now lets fix the regression. Can you please try below fix? >> >> -Vasant > > Thank you for investigating! I created a new branch from the v6.18.40 > tag and > applied your patch. The resulting kernel booted without any issues! Thanks a lot. This is like another band-aid fix, but I think it's good enough for a stable backport. In my view, we should prioritize the IOMMU MMIO-advertised EFR over the one reported via IVRS. However, that would require much more extensive changes and isn't a good candidate for stable. > > If you need any further information or testing (for a proper fix later > on), I'd > be happy to test patches again! Can you help me to test below patch (just apply this patch and remove previously provided fix patch). I want to make sure that prioritizing MMIO EFR doesn't break things. -Vasant ---<--- commit aa5f50f1ee66ebf7a219906a4df6aa2b7cae9717 Author: Vasant Hegde <[email protected]> Date: Mon Aug 24 11:12:27 2026 +0530 iommu/amd: Prioritize IOMMU MMIO advertised EFR over IVRS Signed-off-by: Vasant Hegde <[email protected]> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 69170146d442..0172db87b1df 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -2153,9 +2153,11 @@ static void __init late_iommu_features_init(struct amd_iommu *iommu) if (features != amd_iommu_efr || features2 != amd_iommu_efr2) { pr_warn(FW_WARN - "EFR mismatch. Use IVHD EFR (%#llx : %#llx), EFR2 (%#llx : %#llx).\n", + "EFR mismatch. Use MMIO EFR (%#llx : %#llx), EFR2 (%#llx : %#llx).\n", features, amd_iommu_efr, features2, amd_iommu_efr2); + amd_iommu_efr = features; + amd_iommu_efr2 = features2; } } > > Thanks again! > > Andreas > >> >> ---<--- >> >> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c >> index 69170146d442..7a5b78beb352 100644 >> --- a/drivers/iommu/amd/init.c >> +++ b/drivers/iommu/amd/init.c >> @@ -1930,19 +1930,20 @@ static int __init init_iommu_one(struct amd_iommu >> *iommu, struct ivhd_header *h, >> else >> iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET; >> >> + if (h->efr_attr & BIT(IOMMU_IVHD_ATTR_HATDIS_SHIFT)) { >> + pr_warn_once("Host Address Translation is not supported. >> \n"); >> + amd_iommu_hatdis = true; >> + } >> + >> /* XT and GAM require GA mode. */ >> if ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) == 0) { >> amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_LEGACY; >> + break; >> } else { >> if (h->efr_reg & BIT(IOMMU_EFR_XTSUP_SHIFT)) >> amd_iommu_xt_mode = IRQ_REMAP_X2APIC_MODE; >> } >> >> - if (h->efr_attr & BIT(IOMMU_IVHD_ATTR_HATDIS_SHIFT)) { >> - pr_warn_once("Host Address Translation is not supported. >> \n"); >> - amd_iommu_hatdis = true; >> - } >> - >> early_iommu_features_init(iommu, h); >> >> break; >> >> >> >>>> I reverted that single commit, built the kernel again and the machine >>>> booted, so >>>> I'm quite sure that I found the right commit. >>>> >>>> System information: >>>> >>>> ASRockRack B550D4-4L >>>> BIOS P1.10 (06/28/2021) >>>> >>>> I'm no expert at all, but my research found that the following information >>>> might >>>> be relevant: >>>> >>>> journalctl -W -b -k --no-pager | rg -i 'AMD-Vi|AMD IOMMU|IOMMU|IVRS|IVHD' >>>> Aug 21 22:24:17 kernel: ACPI: IVRS 0x00000000BB062000 0000D0 (v02 AMD >>>> AmdTable 00000001 AMD 00000001) >>>> Aug 21 22:24:17 kernel: ACPI: Reserving IVRS table memory at [mem >>>> 0xbb062000-0xbb0620cf] >>>> Aug 21 22:24:17 kernel: AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0 >>>> Aug 21 22:24:17 kernel: iommu: Default domain type: Translated >>>> Aug 21 22:24:17 kernel: iommu: DMA domain TLB invalidation policy: lazy >>>> mode >>>> Aug 21 22:24:17 kernel: pci 0000:00:01.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:00:01.2: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:00:02.0: Adding to iommu group 1 >>>> Aug 21 22:24:17 kernel: pci 0000:00:03.0: Adding to iommu group 2 >>>> Aug 21 22:24:17 kernel: pci 0000:00:04.0: Adding to iommu group 3 >>>> Aug 21 22:24:17 kernel: pci 0000:00:05.0: Adding to iommu group 4 >>>> Aug 21 22:24:17 kernel: pci 0000:00:07.0: Adding to iommu group 5 >>>> Aug 21 22:24:17 kernel: pci 0000:00:07.1: Adding to iommu group 6 >>>> Aug 21 22:24:17 kernel: pci 0000:00:08.0: Adding to iommu group 7 >>>> Aug 21 22:24:17 kernel: pci 0000:00:08.1: Adding to iommu group 8 >>>> Aug 21 22:24:17 kernel: pci 0000:00:14.0: Adding to iommu group 9 >>>> Aug 21 22:24:17 kernel: pci 0000:00:14.3: Adding to iommu group 9 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.0: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.1: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.2: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.3: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.4: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.5: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.6: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:00:18.7: Adding to iommu group 10 >>>> Aug 21 22:24:17 kernel: pci 0000:02:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:02:00.1: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:02:00.2: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:01.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:02.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:03.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:04.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:08.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:20:09.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:21:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:22:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:23:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:24:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:25:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:29:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:2a:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:2b:00.0: Adding to iommu group 0 >>>> Aug 21 22:24:17 kernel: pci 0000:2c:00.0: Adding to iommu group 11 >>>> Aug 21 22:24:17 kernel: pci 0000:2d:00.0: Adding to iommu group 12 >>>> Aug 21 22:24:17 kernel: pci 0000:2d:00.1: Adding to iommu group 13 >>>> Aug 21 22:24:17 kernel: pci 0000:2d:00.3: Adding to iommu group 14 >>>> Aug 21 22:24:17 kernel: pci 0000:2d:00.4: Adding to iommu group 15 >>>> Aug 21 22:24:17 kernel: AMD-Vi: Extended features (0x58f77e022294881, >>>> 0x0): PreF GA GA_vAPIC >>>> Aug 21 22:24:17 kernel: AMD-Vi: Interrupt remapping enabled >>>> >>>> And the output of the decoded IVRS table, captured by `acpidump -n IVRS -b`: >>>> >>>> 00000000 49 56 52 53 d0 00 00 00 02 e2 41 4d 44 20 20 00 |IVRS......AMD >>>> .| >>>> 00000010 41 6d 64 54 61 62 6c 65 01 00 00 00 41 4d 44 20 >>>> |AmdTable....AMD | >>>> 00000020 01 00 00 00 41 30 20 00 00 00 00 00 00 00 00 00 |....A0 >>>> .........| >>>> 00000030 10 b0 48 00 02 00 40 00 00 00 50 fd 00 00 00 00 >>>> |[email protected].....| >>>> 00000040 00 00 00 00 2e 8f 04 80 03 08 00 00 04 fe ff 00 >>>> |................| >>>> 00000050 43 00 ff 00 00 a5 00 00 04 ff ff 00 00 00 00 00 >>>> |C...............| >>>> 00000060 48 00 00 00 00 a0 00 02 48 00 00 d7 0d a0 00 01 >>>> |H.......H.......| >>>> 00000070 48 00 00 00 0e 01 00 01 11 b0 58 00 02 00 40 00 >>>> |H.........X...@.| >>>> 00000080 00 00 50 fd 00 00 00 00 00 00 00 00 00 02 04 00 >>>> |..P.............| >>>> 00000090 5a 4a 29 22 ef 77 8f 05 00 00 00 00 00 00 00 00 >>>> |ZJ)".w..........| >>>> 000000a0 03 08 00 00 04 fe ff 00 43 00 ff 00 00 a5 00 00 >>>> |........C.......| >>>> 000000b0 04 ff ff 00 00 00 00 00 48 00 00 00 00 a0 00 02 >>>> |........H.......| >>>> 000000c0 48 00 00 d7 0d a0 00 01 48 00 00 00 0e 01 00 01 >>>> |H.......H.......| >>>> 000000d0 >>>> >>>> The relevant part (after decoding with `iasl -d ivrs.dat`): >>>> >>>> [090h 0144 008h] EFR Image : 058F77EF22294A5A >>>> >>>> I'll provide any required details and test patches! >>>> >>>> Thanks for your work, it's very appreciated! >>>> >>>> Andreas >>>> >