Re: AW: [BUG] ice: E810-CAM1 custom board - topology/netlist resolution fails (EINVAL on all AQ topology commands), no I2C activity on GPIO expander bus

On 8/18/2026 11:47 PM, Hoffmann, Fabrice wrote:
> Hi all,
> 
> Follow-up to my earlier report (no replies yet, but we made concrete
> progress on our end that narrows this down significantly).

Hi Fabrice,

I'm working on finding the right person to assist you with these 
issues/questions.

Thanks,
Tony

> We built the ice driver from a 6.14-based kernel (our original test
> system was on 6.12, which lacks the devlink health reporters) and
> tested the same board on different host hardware. The original EINVAL/
> EIO behavior on every topology-dependent AQ command reproduces
> identically on this new setup - so that part is confirmed independent
> of host platform, CPU, and kernel version.
> 
> With devlink health now available, we finally got a direct,
> plain-text firmware diagnosis instead of raw AQ error codes:
> 
>    $ devlink health diagnose pci/0000:05:00.0 reporter fw
>    Syndrome: 274  Description: LOM topology netlist is corrupted.
>    Possible Solution: Update to the latest NVM image.
> 
>    $ devlink health diagnose pci/0000:05:00.0 reporter port
>    Syndrome: 275  Description: Unrecoverable netlist error.
>    Possible Solution: Update to the latest NVM image.
>    Port Number: 0
> 
>    $ devlink health diagnose pci/0000:05:00.1 reporter port
>    Syndrome: 0
>    (ports .4 and .5 likewise report Syndrome: 0 on the port reporter)
> 
> So: the "fw" reporter (global) reports netlist corruption on all 4
> PFs identically, but the "port" reporter only flags an actual
> unrecoverable error on port 0 - ports 1-3 come back clean. Combined
> with the dmesg output being identical on all 4 ports ("Get PHY
> capabilities failed status = -5" / "ice_init_nvm_phy_type failed: -5"),
> our working theory is that netlist/topology processing walks the
> node table in order, hits an unrecoverable condition at the first
> cage node (port 0), and aborts the entire walk - which would explain
> why ports 1-3 are simply never reached/checked (hence "healthy" by
> omission) while the overall topology init still fails for all 4 PFs.
> 
> We're fairly confident this isn't a NVM data-integrity problem in the
> usual sense:
> - The board is already on the latest available NVM/FW (5.01,
>    EETRACK 80021FBF; nvmupdate reports "Update Not Available")
> - We reverse-engineered the Link Topology Netlist (TLV 0x11B) from
>    our image and confirmed it's byte-for-byte identical to Intel's own
>    E810_XXVDA4 reference image at the node/IO table level (PCA9575 I2C
>    GPIO expander correctly present at Bus 4, RESET_N/INT_N on native
>    SDP5/SDP6, matching the datasheet's "4x SFP Native" reference
>    schematic, Section 17.2.2.1)
> - Re-verified this against the actual flashed chip content (ethtool -e
>    readback) after a fresh NVM update - identical result
> - NVM Update Checksum (AQ 0x0706) reports valid
> - GL_MNG_FWSM and GL_MNG_SHA_EXTEND_STATUS (read via UEFI Shell/BAR0,
>    pre-OS) show a completely clean boot: main FW authenticated (STAGE=
>    100b), no POR_AL_FAILURE/PCIR_AL_FAILURE, no EXT_ERR_IND, FW_MODES
>    bit1 (Recovery Mode) = 0
> 
> Given all of that checks out, "corrupted" seems to mean something
> more like a structural/semantic validation failure specific to the
> port-0 cage node (or something processed immediately before it),
> rather than actual bit-level data corruption - but we don't have
> visibility into what that validation actually checks.
> 
> Questions, now with concrete syndrome data attached:
> 
> - What does the firmware's netlist walk actually validate at/around
>    the first cage node that could produce "Unrecoverable netlist error"
>    (syndrome 275) specifically for port 0, while leaving ports 1-3
>    unflagged?
> - Is there a way to get more granular logging out of this
>    specifically (we tried fwlog modules i2c/link_topo/health/adminq via
>    debugfs, and can correlate some opaque message IDs with specific
>    AQ calls, but can't decode their actual content without an internal
>    format-string table)?
> - The suggested remediation ("update to latest NVM image") doesn't
>    apply since we're already on the latest - is there a separate
>    netlist/board-config bundle that could be out of sync independent
>    of the FW/NVM version number?
> 
> Happy to share the full netlist dump, oscilloscope captures, and raw
> AQ traces if that's useful to anyone looking into this.
> 
> Thanks,
> Fabrice