Re: [PATCH] PCI: Disable NoSnoop and Relaxed ordering for Intel wireless BE200

On Tue, 2026-07-28 at 14:11 -0500, Bjorn Helgaas wrote:
> On Tue, Jul 28, 2026 at 06:21:23PM +0000, Grumbach, Emmanuel wrote:
> > On Tue, 2026-07-28 at 13:03 -0500, Bjorn Helgaas wrote:
> > > On Thu, Jul 23, 2026 at 11:35:41AM +0000, Grumbach, Emmanuel
> > > wrote:
> > > > On Thu, 2026-07-23 at 15:19 +0530, Manikanta Maddireddy wrote:
> > > > > On 22/07/26 10:34 pm, Bjorn Helgaas wrote:
> > > > > > > The problem was that we didn't show up at all in the
> > > > > > > enumeration.
> > > > > > > We
> > > > > > > send a malformed TLP. I'm not quite an expert at this,
> > > > > > > but
> > > > > > > our
> > > > > > > PCIe
> > > > > > > experts run a PCI analyzer on the enumeration on that
> > > > > > > specific
> > > > > > > platform and they saw that platform's PCI controller sets
> > > > > > > the
> > > > > > > NoSnoop and Relaxed ordering bit in the TLP. According to
> > > > > > > the
> > > > > > > spec
> > > > > > > (which I ignore), the BE200 is supposed to return the TLP
> > > > > > > as
> > > > > > > received, but we reply with 0 Attributes and the
> > > > > > > enumeration
> > > > > > > doesn't
> > > > > > > complete successfully.
> > > > > > I'm curious about the details of this enumeration failure. 
> > > > > > Do
> > > > > > you
> > > > > > know which TLPs had No Snoop and Relaxed Ordering set?  Per
> > > > > > the
> > > > > > PCIe
> > > > > > spec, they shouldn't be set for the config requests used
> > > > > > for
> > > > > > PCI
> > > > > > core
> > > > > > enumeration.
> > > > > > 
> > > > > > At least*some* config reads to the BE200 must work;
> > > > > > otherwise,
> > > > > > we
> > > > > > wouldn't know the Vendor or Device ID, which we need to
> > > > > > apply
> > > > > > the
> > > > > > quirk.  So I think you should see something like this in
> > > > > > dmesg,
> > > > > > and
> > > > > > BE200 would probably appear in lspci output:
> > > > > > 
> > > > > >    pci 0000:04:00.0: [8086:272b] type 00 class ...
> > > > > > 
> > > > > > Is the failure that iwl_pci_probe() itself fails somehow?
> > > > > > 
> > > > > > You mentioned that this happens on Jetson Thor, but I'm not
> > > > > > sure
> > > > > > what PCIe controller that is.  My guess is it might be
> > > > > > Tegra264
> > > > > > [1], which doesn't look like it's merged yet.
> > > > > > 
> > > > > > Thierry, Manikanta, do you have any insight into this? 
> > > > > > Does
> > > > > > this PCIe controller set No Snoop and Relaxed Ordering for
> > > > > > some
> > > > > > reason?  I don't think endpoint drivers are expecting that.
> > > > > > 
> > > > > > [1]
> > > > > > https://lore.kernel.org/linux-pci/20260716-tegra264-pcie- 
> > > > > > [email protected]/
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > Jetson Thor(Tegra264) is not setting NoSnoop and RlxdOrd bits
> > > > > in
> > > > > config 
> > > > > read TLP.
> > > > > 
> > > > > I made sure that both these bits are set in RP's DevCtl
> > > > > 
> > > > >                  DevCtl: CorrErr+ NonFatalErr+ FatalErr+
> > > > > UnsupReq+
> > > > >                          RlxdOrd+ ExtTag+ PhantFunc- AuxPwr-
> > > > > NoSnoop+
> > > > > 
> > > > > and dumped TLP header for a config read towards BDF 0x100
> > > > > with
> > > > > offset
> > > > > 0x24.
> > > > >     0x04000001      0x0000000f      0x01000024     
> > > > > 0x00000000
> > > > > 
> > > > > I think dmesg and AER log with header information might help
> > > > > with
> > > > > this 
> > > > > particular issue.
> > > > 
> > > > Ok, so I checked again the logs and I was wrong.
> > > > We do see the device in the enumeration, it does show up in
> > > > lspci.
> > > > Problems start when we want to access our registers.
> > > > I attached the full dmesg output. In that log we try to load
> > > > iwlwifi
> > > > twice.
> > > > 
> > > > I'm adding here the snippet of the first load:
> > > > 
> > > > [   15.275887] iwlwifi 0001:01:00.0: Adding to iommu group 53
> > > > [   15.279737] iwlwifi 0001:01:00.0: enabling device (0100 ->
> > > > 0102)
> > > > [   15.280047] iwlwifi 0001:01:00.0: HW_REV=0xFFFFFFFF, PCI
> > > > issues?  \\ **
> > > 
> > > The "enabling device" message is from pci_enable_resources(),
> > > called
> > > in the pci_enable_device() path.  The 0100 is from a config read
> > > of
> > > PCI_COMMAND, and the 0102 is from adding PCI_COMMAND_MEMORY to
> > > enable
> > > memory BARs.
> > > 
> > > The "HW_REV=" is from iwl_pci_probe(), which looks like the very
> > > first
> > > MMIO read to a BE200 BAR.
> > 
> > Indeed
> > 
> > > So I guess the theory is that Tegra264 set NoSnoop and/or RlxdOrd
> > > in
> > > the MMIO read, BE200 didn't copy the attributes from the Request
> > > to
> > > the Completion as required by PCIe r7.0, sec 2.2.9.1, and
> > > Tegra264
> > > logged a Malformed TLP?
> > 
> > That's the assumption based on the AER log, yes.
> > 
> > > AFAICS we still don't know why Tegra264 would set NoSnoop and/or
> > > RlxdOrd in the MMIO read.
> > 
> > Me neither but... I can't comment on that. And de-facto, once it
> > does
> > that, the BE200 replies with the NoSnoop and RlxOrd clear in the
> > TLP
> > which is then considered as a malformed TLP.
> > 
> > > Unless iwlwifi asked for NoSnoop and/or RlxdOrd to be set, I
> > > think
> > > it's a potential problem for drivers if Tegra264 sets them.
> > 
> > I ... don't think we would do that. The driver would certainly not
> > do
> > that... Regarding the hardware itself, I can't comment, but I can
> > check
> > internally.
> 
> Right, I don't see any mention of PCI_EXP_DEVCTL_RELAX_EN or
> PCI_EXP_DEVCTL_NOSNOOP_EN or even PCI_EXP_DEVCTL there.  And it would
> have to be set in the Root Port for it to affect the MMIO read
> request, not in BE200.  That would affect everything under the RP, so
> I think these are probably most useful in the DMA direction where an
> individual device can control its own Requests.
> 
> But the analyzer saw them set in the request from the Root Port, so
> it's still a mystery why what happened.

Or...  Tegra264 simply sets them for an unclear reason?

Note that our previous device (AX210) doesn't have the bug, it sets the
bits in the TLP as expected and it works with Tegra264.
I have to admit that at this point, I don't really know what to do. The
quirk that I am suggesting fixed the bug for us. I just wonder what
else is needed to take the decision whether to take it or not.