Re: i2c designware change broke touchpad of a thinkpad
Hardik Prakash <[email protected]>
| Newsgroups | dev.linux.lists.regressions,org.kernel.vger.linux-gpio,org.kernel.vger.linux-i2c |
|---|---|
| Message-ID | <CANTFpSV9jguMzmfN7STb7XANYdn8i34bVmpT=0NdJ_o=3_Y1CQ@mail.gmail.com> |
On Wed, 13 Aug 2026, Thorsten Leemhuis wrote: > Hardik, could you submit a revert to the lists? And then I'll provide my > Tested-by and ask Linus to directly pick this up? Sent: https://lore.kernel.org/all/[email protected]/ Verified on my end that it builds clean, boots clean, and correctly restores the original Yoga 7 race (expected tradeoff) without introducing anything else. Thanks, Hardik On Thu, 13 Aug 2026 at 16:26, Thorsten Leemhuis <[email protected]> wrote: > > On 8/12/26 14:34, Mario Limonciello wrote: > > On 8/12/26 00:45, Hardik Prakash wrote: > >> On Wed, 12 Aug 2026, Thorsten Leemhuis wrote: > >>> While we wait for Mario a quick question: Should we revert > >>> the culprit for 7.2 and reapply it later once we sorted this out? Of > >>> course I'll continue to help debugging this. > >>> > >>> If it was just me I'd say "don't bother", but these kind of Thinkpads > >>> are what some Linux users frequently buy, so I fear that other people > >>> will run into this if the culprit makes it into 7.2. > >> > >> I agreed, I'd rather have this reverted than ship a known regression. > >> > >> Andi, WSA -- given 7.2 is close, could this go out as a revert now? I'll > >> keep working the AMD bus-timing angle with Thorsten and Mario, and > >> we can resubmit a corrected fix for a later release once it's actually > >> understood. > > > > Let's please revert for 7.2 final and keep working at this. > > Can't see any messages from Andi on lore in the past week and just one > from WSA. Guess well deserved holiday season or something like that. > Happens, no worries, but I guess we thus should try to handle this on > our own. > > Hardik, could you submit a revert to the lists? And then I'll provide my > Tested-by and ask Linus to directly pick this up? In a case like this I > guess that is one of the viable options. Or does anyone see a better > path forward here? > > Ciao, Thorsten > > I've got a /hunch/ what's going on. > > > > For both Dell and Lenovo I know a lot of designs have these I2C buses > > that have both the EC and the APU connected to them. "Both" the EC and > > APU "can" operate as master. > > > > The reason that this exists is so that when the system is in pre-boot > > the EC acts as master. This lets you use the touchpad in the BIOS menus > > for example. When the OS boots up there is some ASL called which sends > > an EC command that tells the EC to stand down. At this point the APU > > takes over and acts as master. > > > > So my theory here is that by changing this ordering the _DSM is not > > called at the right timing anymore. > > > > This is a fairly similar issue as was seen that led to this commit: > > > > https://git.kernel.org/torvalds/c/7d62beb102d6f > >> > >> Thanks, > >> Hardik > >> > >> On Wed, 12 Aug 2026 at 10:51, Thorsten Leemhuis > >> <[email protected]> wrote: > >>> > >>> On 8/12/26 06:59, Hardik Prakash wrote: > >>>> On Mon, 10 Aug 2026, Thorsten Leemhuis wrote: > >>>>> Did a fresh boot and uploaded the dmesg again as well just in case: > >>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg2 > >>>>> https://www.leemhuis.info/files/misc/lsmod > >>>>> https://www.leemhuis.info/files/misc/interrupts > >>>> > >>>> Thanks. Second boot reproduces the identical pattern: device_is_bound() > >>>> =true at 1.132s, lost arbitration at 2.139s -- same ~1s gap as the > >>>> first > >>>> capture, so this is consistent and reproducible, not a one-off. > >>>> > >>>> interrupts rules out simple IRQ-line sharing: AMDI0010:01 has its own > >>>> dedicated line (IRQ 11), separate from AMDI0010:00 (IRQ 10) and every > >>>> amd_gpio ACPI:Event line. psp-1 exists (IR-PCI-MSIX-0000:c3:00.2) but > >>>> shows zero recorded activity on any CPU. > >>>> > >>>> Since lost arbitration means an actual SDA/SCL-level collision (not an > >>>> IRQ conflict), and there's no shared IRQ or visible PSP activity, if > >>>> something else is touching this bus it's happening below what Linux's > >>>> interrupt accounting can see -- which is why I wanted your take, Mario. > >>>> Does this match a known pattern of firmware (PSP/EC) briefly taking > >>>> this bus during boot on AMD platforms? > >>> > >>> Thx again. While we wait for Mario a quick question: Should we revert > >>> the culprit for 7.2 and reapply it later once we sorted this out? Of > >>> course I'll continue to help debugging this. > >>> > >>> If it was just me I'd say "don't bother", but these kind of Thinkpads > >>> are what some Linux users frequently buy, so I fear that other people > >>> will run into this if the culprit makes it into 7.2. > >>> > >>> Ciao, Thorsten > >>> > >>>> On Mon, 10 Aug 2026 at 12:18, Thorsten Leemhuis > >>>> <[email protected]> wrote: > >>>>> > >>>>> On 8/10/26 07:54, Hardik Prakash wrote: > >>>>>> On Sun, 9 Aug 2026, Thorsten Leemhuis wrote: > >>>>>>> Sorry, picked the wrong file while preparing the mail, it's here: > >>>>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg > >>>>>> > >>>>>> Thanks, that's the one. Full timeline from that boot: > >>>>>> > >>>>>> 0.619s AMDI0010:00 and AMDI0010:01 defer (AMDI0030:00 not > >>>>>> registered) > >>>>>> 1.136s Retry succeeds: device_is_bound()=true for AMDI0030:00, > >>>>>> confirmed for all three children (XXXX0000:00, > >>>>>> SYNA8018:00, > >>>>>> NXP1001:00) > >>>>>> 2.222s lost arbitration on AMDI0010:01 (x3) > >>>>>> 3.260s controller timed out, HID descriptor fetch fails > >>>>>> > >>>>>> The dependency check itself is behaving correctly here -- it defers, > >>>>>> then correctly confirms AMDI0030:00 is bound before letting probe > >>>>>> continue. The arbitration failure happens over a second *after* that, > >>>>>> when the touchpad attempts its actual first transaction. > >>>>>> > >>>>>> So this doesn't look like a logic bug in the check itself. My > >>>>>> read: by > >>>>>> delaying dw_i2c_plat_probe() by ~500ms (correctly, for GPIO's sake), > >>>>>> something else ends up contending for the AMDI0010:01 bus by the time > >>>>>> the touchpad's first transaction actually happens -- a window that > >>>>>> the original (undeferred, earlier) probe attempt apparently avoided. > >>>>>> > >>>>>> Mario -- does this look like a known AMD I2C/PSP semaphore contention > >>>>>> pattern to you? Wondering if there's a shared bus/firmware consumer > >>>>>> that could explain a ~1s-later collision like this, independent of > >>>>>> the GPIO controller itself being ready. > >>>>>> > >>>>>> Thorsten, if you have it handy: /proc/interrupts and lsmod from this > >>>>>> boot would help narrow down what else might be touching that bus > >>>>>> around the 2.2s mark. > >>>>> > >>>>> Did a fresh boot and uploaded the dmesg again as well just in case: > >>>>> > >>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg2 > >>>>> https://www.leemhuis.info/files/misc/lsmod > >>>>> https://www.leemhuis.info/files/misc/interrupts > >>>>> > >>>>> Ciao, Thorsten > >>>>>> On Mon, 10 Aug 2026 at 11:06, Thorsten Leemhuis > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> On 8/10/26 07:23, Hardik Prakash wrote: > >>>>>>>> On Sat, 8 Aug 2026, Thorsten Leemhuis wrote: > >>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert > >>>>>>>>> > >>>>>>>>> [ 0.619018] i2c-dw-gpio-dbg: > >>>>>>>>> i2c_dw_check_gpio_dependencies(AMDI0010:00) > >>>>>>>>> [...] > >>>>>>>>> [ 1.136010] i2c-dw-gpio-dbg: device_is_bound()=true for > >>>>>>>>> AMDI0030:00 > >>>>>>>> > >>>>>>>> Thanks, this trace looks correct on its face (defer at 0.619s, > >>>>>>>> retry > >>>>>>>> succeeds at 1.122-1.136s), but the link above points to the same > >>>>>>>> file > >>>>>>>> you sent last round (the master+revert baseline, no debug patch, no > >>>>>>>> i2c-dw-gpio-dbg lines in it at all) rather than this new capture. > >>>>>>>>> Could you share the full, unfiltered dmesg from this exact boot > >>>>>>>>> (the > >>>>>>>> one with the debug patch applied)? I need to see where > >>>>>>>> "lost arbitration" falls relative to the trace above. > >>>>>>> > >>>>>>> Sorry, picked the wrong file while preparing the mail, it's here: > >>>>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg > >>>>>>> > >>>>>>> Ciao, Thorsten > >>>>>>> > >>>>>>>> Separately, from your DSDT: NFC1 (NXP1001), which shows up as a > >>>>>>>> child > >>>>>>>> of AMDI0010:01 in your trace, declares its _CRS as a raw resource > >>>>>>>> buffer containing three GPIO resources referencing \_SB.GPIO -- one > >>>>>>>> GpioIo and two separate GpioInt entries. Not sure yet if that's > >>>>>>>> relevant, but it's a pattern I haven't seen on my hardware > >>>>>>>> which I've tested against. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Hardik > >>>>>>>> > >>>>>>>> On Sun, 9 Aug 2026 at 21:49, Thorsten Leemhuis > >>>>>>>> <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>> On 8/8/26 22:25, Andy Shevchenko wrote: > >>>>>>>>>> On Sat, Aug 08, 2026 at 10:31:28PM +0530, Hardik Prakash wrote: > >>>>>>>>>>> On Fri, 7 Aug 2026, Thorsten Leemhuis wrote: > >>>>>>>>>>>> This became 0a4bb2abc3e56d ("i2c: designware: defer probe if > >>>>>>>>>>>> child > >>>>>>>>>>>> GpioInt controllers are not bound") and causes a regression > >>>>>>>>>>>> for me: the > >>>>>>>>>>>> Touchpad in my ThinkPad T14s Gen 4 (AMD) stopped working. > >>>>>>>>>>>> From the > >>>>>>>>>>>> kernel log: > >>>>>>>>>>>> > >>>>>>>>>>>> i2c_designware AMDI0010:01: i2c_dw_handle_tx_abort: lost > >>>>>>>>>>>> arbitration > >>>>>>>>>>>> i2c_designware AMDI0010:01: controller timed out > >>>>>>>>>>>> i2c_hid_acpi i2c-SYNA8018:00: failed to fetch HID > >>>>>>>>>>>> descriptor: -110 > >>>>>>>>>>>> i2c_hid_acpi i2c-SYNA8018:00: Failed to fetch the HID > >>>>>>>>>>>> Descriptor > >>>>>>>>>>>> > >>>>>>>>>>>> Reverting this change in current mainline/master > >>>>>>>>>>>> (f9a2394a23482) fixed > >>>>>>>>>>>> things for me. Full dmesg of current master and > >>>>>>>>>>>> master+revert can be > >>>>>>>>>>>> found here: > >>>>>>>>>>>> > >>>>>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master > >>>>>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert > >>>>>>>>>>> [...] > >>>>>>>>>>> Two things that would help narrow this down, whenever you > >>>>>>>>>>> have a moment: > >>>>>>>>> > >>>>>>>>> Thx for looking into this. Here we go: > >>>>>>>>> > >>>>>>>>>>> 1. A DSDT dump > >>>>>>>>> > >>>>>>>>> https://www.leemhuis.info/files/misc/dsdt.dsl > >>>>>>>>> > >>>>>>>>>>> 2. A dmesg with dynamic debug from a kernel built with the > >>>>>>>>>>> attached > >>>>>>>>>>> patch on top of current mainline/master > >>>>>>>>> > >>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert > >>>>>>>>> > >>>>>>>>> [ 0.619018] i2c-dw-gpio-dbg: > >>>>>>>>> i2c_dw_check_gpio_dependencies(AMDI0010:00) > >>>>>>>>> [ 0.619018] i2c-dw-gpio-dbg: > >>>>>>>>> check_child_gpioint(adev=XXXX0000:00) > >>>>>>>>> [ 0.619076] i2c-dw-gpio-dbg: GpioInt > >>>>>>>>> resource_source=\_SB.GPIO (len=10) > >>>>>>>>> [ 0.619079] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded, > >>>>>>>>> device=AMDI0030:00 > >>>>>>>>> [ 0.619080] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() > >>>>>>>>> returned > >>>>>>>>> NULL, deferring > >>>>>>>>> [ 0.619082] i2c-dw-gpio-dbg: check_child_gpioint ret=-517 > >>>>>>>>> (defer) > >>>>>>>>> [ 0.619102] i2c-dw-gpio-dbg: > >>>>>>>>> i2c_dw_check_gpio_dependencies(AMDI0010:01) > >>>>>>>>> [ 0.619104] i2c-dw-gpio-dbg: > >>>>>>>>> check_child_gpioint(adev=SYNA8018:00) > >>>>>>>>> [ 0.619159] i2c-dw-gpio-dbg: GpioInt > >>>>>>>>> resource_source=\_SB.GPIO (len=10) > >>>>>>>>> [ 0.619162] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded, > >>>>>>>>> device=AMDI0030:00 > >>>>>>>>> [ 0.619162] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() > >>>>>>>>> returned > >>>>>>>>> NULL, deferring > >>>>>>>>> [ 0.619164] i2c-dw-gpio-dbg: check_child_gpioint ret=-517 > >>>>>>>>> (defer) > >>>>>>>>> [ 1.122517] i2c-dw-gpio-dbg: > >>>>>>>>> i2c_dw_check_gpio_dependencies(AMDI0010:00) > >>>>>>>>> [ 1.122523] i2c-dw-gpio-dbg: > >>>>>>>>> check_child_gpioint(adev=XXXX0000:00) > >>>>>>>>> [ 1.122625] i2c-dw-gpio-dbg: GpioInt > >>>>>>>>> resource_source=\_SB.GPIO (len=10) > >>>>>>>>> [ 1.122636] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded, > >>>>>>>>> device=AMDI0030:00 > >>>>>>>>> [ 1.122639] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() > >>>>>>>>> succeeded > >>>>>>>>> [ 1.122642] i2c-dw-gpio-dbg: device_is_bound()=true for > >>>>>>>>> AMDI0030:00 > >>>>>>>>> [ 1.135894] i2c-dw-gpio-dbg: > >>>>>>>>> i2c_dw_check_gpio_dependencies(AMDI0010:01) > >>>>>>>>> [ 1.135898] i2c-dw-gpio-dbg: > >>>>>>>>> check_child_gpioint(adev=SYNA8018:00) > >>>>>>>>> [ 1.135976] i2c-dw-gpio-dbg: GpioInt > >>>>>>>>> resource_source=\_SB.GPIO (len=10) > >>>>>>>>> [ 1.135980] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded, > >>>>>>>>> device=AMDI0030:00 > >>>>>>>>> [ 1.135983] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() > >>>>>>>>> succeeded > >>>>>>>>> [ 1.135985] i2c-dw-gpio-dbg: device_is_bound()=true for > >>>>>>>>> AMDI0030:00 > >>>>>>>>> [ 1.135987] i2c-dw-gpio-dbg: > >>>>>>>>> check_child_gpioint(adev=NXP1001:00) > >>>>>>>>> [ 1.136005] i2c-dw-gpio-dbg: GpioInt > >>>>>>>>> resource_source=\_SB.GPIO (len=10) > >>>>>>>>> [ 1.136008] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded, > >>>>>>>>> device=AMDI0030:00 > >>>>>>>>> [ 1.136009] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() > >>>>>>>>> succeeded > >>>>>>>>> [ 1.136010] i2c-dw-gpio-dbg: device_is_bound()=true for > >>>>>>>>> AMDI0030:00 > >>>>>>>>> > >>>>>>>>> HTH, Ciao, Thorsten > >>>>>>>>> > >>>>>>>>>>> On Fri, 7 Aug 2026 at 19:37, Thorsten Leemhuis > >>>>>>>>>>> <[email protected]> wrote: > >>>>>>>>>>>> On 7/28/26 10:30, Andi Shyti wrote: > >>>>>>>>>>>>> On Sat, Jul 18, 2026 at 11:13:31AM +0530, Hardik Prakash > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> I2C controllers may have child devices with GpioInt > >>>>>>>>>>>>>> resources that > >>>>>>>>>>>>>> depend on GPIO controllers being fully initialized. If the > >>>>>>>>>>>>>> I2C > >>>>>>>>>>>>>> controller probes and enumerates children before the > >>>>>>>>>>>>>> referenced GPIO > >>>>>>>>>>>>>> controller has completed probe, GPIO interrupts may not be > >>>>>>>>>>>>>> properly > >>>>>>>>>>>>>> configured, leading to device failures. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Lenovo Yoga 7 14AGP11, the WACF2200 touchscreen (child of > >>>>>>>>>>>>>> AMDI0010:02) has a GpioInt resource pointing to GPIO 157 > >>>>>>>>>>>>>> on the > >>>>>>>>>>>>>> pinctrl-amd controller (AMDI0030:00). When i2c-designware > >>>>>>>>>>>>>> probes > >>>>>>>>>>>>>> AMDI0010:02 before pinctrl-amd finishes initializing, I2C > >>>>>>>>>>>>>> transactions > >>>>>>>>>>>>>> fail with lost arbitration errors: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 0.285952 amd_gpio_probe: registering gpiochip <- GPIO > >>>>>>>>>>>>>> chip visible > >>>>>>>>>>>>>> 0.287121 amd_gpio_probe: requesting parent IRQ <- > >>>>>>>>>>>>>> probe still running > >>>>>>>>>>>>>> 0.301454 AMDI0010:02 dw_i2c_plat_probe: start <- > >>>>>>>>>>>>>> races here > >>>>>>>>>>>>>> 2.348157 lost arbitration > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Add a dependency check that walks ACPI child devices and > >>>>>>>>>>>>>> defers probe > >>>>>>>>>>>>>> until any referenced GPIO controller is bound. > >>>>>>>>>> > >>>>>>>>>> [...] > >>>>>>>>>> > >>>>>>>>>>>> #regzbot introduced: 0a4bb2abc3e56d > >>>>>>>>>>>> #regzbot title i2c: designware: ThinkPad touchpad broke > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > > >