Re: [REGRESSION] mainline/master: Apalis iMX6 no longer boots

On Wed, Aug 12, 2026 at 08:54:13AM +0200, Francesco Dolcini wrote:
> On Wed, Aug 12, 2026 at 06:22:57AM +0000, Hongxing Zhu wrote:
> > 
> > NXP Confidential
> > > -----Original Message-----
> > > From: Francesco Dolcini <[email protected]>
> > > Sent: Tuesday, August 11, 2026 2:38 PM
> > > To: Hongxing Zhu (OSS) <[email protected]>
> > > Cc: Leonardo Costa <[email protected]>; Bjorn Helgaas
> > > <[email protected]>; [email protected]; [email protected];
> > > [email protected]; Frank Li <[email protected]>; [email protected];
> > > [email protected]; [email protected]; [email protected];
> > > [email protected]; [email protected]; linux-arm-
> > > [email protected]; [email protected];
> > > [email protected]; Sherry Sun <[email protected]>; Manivannan
> > > Sadhasivam <[email protected]>; Hongxing Zhu <[email protected]>;
> > > [email protected]
> > > Subject: Re: [REGRESSION] mainline/master: Apalis iMX6 no longer boots
> > >
> > > On Tue, Aug 11, 2026 at 12:47:18AM +0000, Hongxing Zhu (OSS) wrote:
> > > > > -----Original Message-----
> > > > > From: Leonardo Costa <[email protected]>
> > > > > Sent: Tuesday, August 11, 2026 2:26 AM
> > > > > To: Hongxing Zhu (OSS) <[email protected]>
> > > > > Cc: Bjorn Helgaas <[email protected]>; [email protected];
> > > > > [email protected]; [email protected]; Frank Li
> > > > > krzk+<[email protected]>;
> > > > > [email protected]; [email protected]; [email protected];
> > > > > [email protected]; [email protected];
> > > > > [email protected]; [email protected]; linux-
> > > > > [email protected]; [email protected]; Sherry Sun
> > > > > <[email protected]>; Manivannan Sadhasivam <[email protected]>;
> > > > > Hongxing Zhu <[email protected]>; [email protected]
> > > > > Subject: Re: [REGRESSION] mainline/master: Apalis iMX6 no longer
> > > > > boots
> > > > >
> > > > > On Thu, Jul 16, 2026 at 02:09:19PM -0300, Leonardo Costa wrote:
> > > > > > On Thu, Jul 16, 2026 at 02:15:42AM +0000, Hongxing Zhu (OSS) wrote:
> > > > > > > Hi Leonardo,
> > > > > > >
> > > > > > > Would you be able to test this patch on your affected boards?
> > > > > > > Your testing would help expedite the merge process and ensure
> > > > > > > the fix resolves the boot issue you're experiencing.
> > > > > > >
> > > > > > > Best Regards
> > > > > > > Richard Zhu
> > > > > >
> > > > > > Hi Richard,
> > > > > >
> > > > > > Sorry for taking long to respond, I was caught up with something.
> > > > > > I tested the patch and it solved the issue. I sent a Tested-by tag there.
> > > > > > Thanks for looking into this!
> > > > > >
> > > > > > Best Regards
> > > > > > Leonardo
> > > > >
> > > > > Hi Richard,
> > > > >
> > > > > That patch might have introduced/exposed a second regression where
> > > > > PCI devices are not linked properly during the initialization. Sorry
> > > > > for not catching this before, as I didn't have any device connected during
> > > the previous test.
> > > > Hi Leonardo:
> > > > Sorry to hear about that.
> > > > I did test commit 7678e81498e2 with one Intel NIC (82574L) connected,
> > > > and I've double-checked today based on v7.2-rc7. The PCIe link can be
> > > > set up properly on the i.MX6Q Sabresd board in my testing.
> > > >
> > > > Here are the logs:
> > > > root@imx6qpdlsolox:~# lspci
> > > > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> > > > 01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > > > Connection root@imx6qpdlsolox:~# uname -a Linux imx6qpdlsolox
> > > > 7.2.0-rc7 #30 SMP Tue Aug 11 00:21:26 UTC 2026 armv7l GNU/Linux
> > > > root@imx6qpdlsolox:~# dmesg | grep pci
> > > > [    0.256138] imx6q-pcie 1ffc000.pcie: host bridge /soc/pcie@1ffc000 ranges:
> > > > [    0.256307] imx6q-pcie 1ffc000.pcie:       IO 0x0001f80000..0x0001f8ffff ->
> > > 0x0000000000
> > > > [    0.256339] imx6q-pcie 1ffc000.pcie:      MEM 0x0001000000..0x0001efffff -
> > > > 0x0001000000
> > > > [    0.256434] imx6q-pcie 1ffc000.pcie: config reg[1] 0x01f00000 == cpu
> > > 0x01f00000
> > > > [    0.458388] imx6q-pcie 1ffc000.pcie: iATU: unroll F, 4 ob, 4 ib, align 64K,
> > > limit 4G
> > > > [    0.458492] imx6q-pcie 1ffc000.pcie: Link: Only Gen1 is enabled
> > > > [    0.555294] imx6q-pcie 1ffc000.pcie: PCIe Gen.1 x1 link up
> > > > [    0.555783] imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00
> > > > ...
> > > >
> > > > The following endpoint devices also yielded the same results.
> > > > root@imx6qpdlsolox:~# lspci
> > > > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> > > > 01:00.0 Network controller: Qualcomm Atheros AR9287 Wireless Network
> > > > Adapter (PCI-Express) (rev 01) root@imx6qpdlsolox:~# uname -a Linux
> > > > imx6qpdlsolox 7.2.0-rc7 #30 SMP Tue Aug 11 00:21:26 UTC 2026 armv7l
> > > > GNU/Linux ...
> > > >
> > > > root@imx6qpdlsolox:~# lspci
> > > > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> > > > 01:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host
> > > > Controller (rev 04) root@imx6qpdlsolox:~# uname -a Linux imx6qpdlsolox
> > > > 7.2.0-rc7 #30 SMP Tue Aug 11 00:21:26 UTC 2026 armv7l GNU/Linux
> > > >
> > > > Could you provide more details of your hardware setup (board,
> > > > connected devices, PERST# present or not)?
> > > > This may help us identify if there's a specific scenario that triggers
> > > > the regression you're experiencing.
> > >
> > > It's an apalis imx6, as reported already in this email thread.
> > > The DT is in Linux, you can check
> > > arch/arm/boot/dts/nxp/imx/imx6q-apalis-ixora-v1.2.dts every details about
> > > PERST# and more.
> > >
> > > This is the device connected:
> > >
> > > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> > > 01:00.0 SATA controller: ASMedia Technology Inc. ASM1061/ASM1062 Serial
> > > ATA Controller (rev 02)
> > >
> > > 01:00.0 SATA controller: ASMedia Technology Inc. ASM1061/ASM1062 Serial
> > > ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
> > >       Subsystem: ASMedia Technology Inc. ASM1061/ASM1062 Serial ATA
> > > Controller
> > >       Flags: bus master, fast devsel, latency 0, IRQ 308
> > >       I/O ports at 1020 [size=8]
> > >       I/O ports at 1030 [size=4]
> > >       I/O ports at 1028 [size=8]
> > >       I/O ports at 1034 [size=4]
> > >       I/O ports at 1000 [size=32]
> > >       Memory at 01100000 (32-bit, non-prefetchable) [size=512]
> > >       Expansion ROM at 01200000 [virtual] [disabled] [size=64K]
> > >       Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
> > >       Capabilities: [78] Power Management version 3
> > >       Capabilities: [80] Express Legacy Endpoint, IntMsgNum 0
> > >       Capabilities: [100] Virtual Channel
> > >       Kernel driver in use: ahci
> > >       Kernel modules: ahci
> > >
> 
> I just did one additional test, trying to help out Leonardo.
> 
> Unfortunately I do not have the same PCIe card available, so I tested with a
> 
> 01:00.0 Network controller: Intel Corporation Wireless 7260 (rev 73)
> 	Subsystem: Intel Corporation Dual Band Wireless-AC 7260 [Wilkins Peak 2]
> 	Flags: fast devsel
> 	Memory at 01100000 (64-bit, non-prefetchable) [disabled] [size=8K]
> 	Capabilities: [c8] Power Management version 3
> 	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
> 	Capabilities: [40] Express Endpoint, IntMsgNum 0
> 	Capabilities: [100] Advanced Error Reporting
> 	Capabilities: [140] Device Serial Number ac-7b-a1-ff-ff-4e-a1-d8
> 	Capabilities: [14c] Latency Tolerance Reporting
> 	Capabilities: [154] Vendor Specific Information: ID=cafe Rev=1 Len=014 <?>
> 
> no wi-fi driver is loaded (but this seems unrelevant for the issue here), but
> the card is detected with the current mainline Linux 7.2.0-rc7+. I tested the
> apalis SoM on both the eval carrier board and on ixora v1.2.
> 
> Same kernel config as the one failing.
> 
> So, unless I did some mistake in my manual test, the issue is related to the specific
> PCIe card or some other subtle difference I cannot see yet.
> 
> For the record, the failure was detected by our CI/CD, and it is systematically
> reproduced on multiple hardware boards.
> 
> Leonardo: any chance you can test with the same PCIe board that is failing?

Hi all,

I reproduced the issue on my setup (eval board + same PCIe card as in our CI/CD).

root@apalis-imx6-11486042:~# lspci
root@apalis-imx6-11486042:~# dmesg | grep pcie
[    0.162860] imx6q-pcie 1ffc000.pcie: host bridge /soc/pcie@1ffc000 ranges:
[    0.162974] imx6q-pcie 1ffc000.pcie:       IO 0x0001f80000..0x0001f8ffff -> 0x0000000000
[    0.163043] imx6q-pcie 1ffc000.pcie:      MEM 0x0001000000..0x0001efffff -> 0x0001000000
[    0.163193] imx6q-pcie 1ffc000.pcie: config reg[1] 0x01f00000 == cpu 0x01f00000
[    0.505056] imx6q-pcie 1ffc000.pcie: iATU: unroll F, 4 ob, 4 ib, align 64K, limit 4G
[    0.505230] imx6q-pcie 1ffc000.pcie: Link: Only Gen1 is enabled
[    1.508173] imx6q-pcie 1ffc000.pcie: Link failed to come up. LTSSM: POLL_CONFIG
[    1.508314] imx6q-pcie 1ffc000.pcie: probe with driver imx6q-pcie failed with error -110

After applying the new sequence proposed by Zhu, the PCIe card comes up.

root@apalis-imx6-11486042:~# lspci
00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
01:00.0 SATA controller: ASMedia Technology Inc. ASM1061/ASM1062 Serial ATA Controller (rev 02)
root@apalis-imx6-11486042:~# dmesg | grep pcie
[    0.162515] imx6q-pcie 1ffc000.pcie: host bridge /soc/pcie@1ffc000 ranges:
[    0.162636] imx6q-pcie 1ffc000.pcie:       IO 0x0001f80000..0x0001f8ffff -> 0x0000000000
[    0.162702] imx6q-pcie 1ffc000.pcie:      MEM 0x0001000000..0x0001efffff -> 0x0001000000
[    0.162828] imx6q-pcie 1ffc000.pcie: config reg[1] 0x01f00000 == cpu 0x01f00000
[    0.488893] imx6q-pcie 1ffc000.pcie: iATU: unroll F, 4 ob, 4 ib, align 64K, limit 4G
[    0.489033] imx6q-pcie 1ffc000.pcie: Link: Only Gen1 is enabled
[    0.591333] imx6q-pcie 1ffc000.pcie: PCIe Gen.1 x1 link up
[    0.591967] imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00
[    0.602947] pcieport 0000:00:00.0: PME: Signaling with IRQ 291
[    0.603534] pcieport 0000:00:00.0: AER: enabled with IRQ 291

I'm completely out of the loop on this whole issue and I only helped out
to test the patch. However, I wanted to add that before applying Zhu's 
proposed sequence, I could only reproduce the issue by power cycling the
module, as rebooting alone did not trigger the issue.

kind regards

Franz