AW: AW: [BUG] ice: E810-CAM1 custom board - topology/netlist resolution fails (EINVAL on all AQ topology commands), no I2C activity on GPIO expander bus

Hi Tony,

Thank you for the promised assistance.
We have actually made some further progress since then. We now know that the netlist for the image `E810_SFP_10G_NRB_SEC_FW_1p8p0p2_NVM_5p01_NCSIwPLDMoMCTP_0.01_80021FBF` is designed for 8 ports—a detail that isn't apparent or mentioned in the datasheet. In any case, the error disappears when we flash the netlist from the image `E810_SFP_25G_NRB_SD_SEC_FW_1p8p0p2_NVM_5p01_NCSIwPLDMoMCTP_0.01_80021FA1`, and everything works fine. However, that configures the card for 25G, which isn't what we need. Consequently, we still require a suitable image—one that isn't available "off the shelf." We need a 4x10G SFP configuration without NCSI. How can we obtain such an image? Does anyone already have one? Or how can we access the tools needed to create a suitable image?

Thank sin advance

Best regards

Fabrice


________________________________
Von: Tony Nguyen <[email protected]>
Gesendet: Donnerstag, 20. August 2026 02:02
An: Hoffmann, Fabrice <[email protected]>; [email protected] <[email protected]>
Betreff: Re: AW: [BUG] ice: E810-CAM1 custom board - topology/netlist resolution fails (EINVAL on all AQ topology commands), no I2C activity on GPIO expander bus



On 8/18/2026 11:47 PM, Hoffmann, Fabrice wrote:
> Hi all,
>
> Follow-up to my earlier report (no replies yet, but we made concrete
> progress on our end that narrows this down significantly).

Hi Fabrice,

I'm working on finding the right person to assist you with these
issues/questions.

Thanks,
Tony

> We built the ice driver from a 6.14-based kernel (our original test
> system was on 6.12, which lacks the devlink health reporters) and
> tested the same board on different host hardware. The original EINVAL/
> EIO behavior on every topology-dependent AQ command reproduces
> identically on this new setup - so that part is confirmed independent
> of host platform, CPU, and kernel version.
>
> With devlink health now available, we finally got a direct,
> plain-text firmware diagnosis instead of raw AQ error codes:
>
>    $ devlink health diagnose pci/0000:05:00.0 reporter fw
>    Syndrome: 274  Description: LOM topology netlist is corrupted.
>    Possible Solution: Update to the latest NVM image.
>
>    $ devlink health diagnose pci/0000:05:00.0 reporter port
>    Syndrome: 275  Description: Unrecoverable netlist error.
>    Possible Solution: Update to the latest NVM image.
>    Port Number: 0
>
>    $ devlink health diagnose pci/0000:05:00.1 reporter port
>    Syndrome: 0
>    (ports .4 and .5 likewise report Syndrome: 0 on the port reporter)
>
> So: the "fw" reporter (global) reports netlist corruption on all 4
> PFs identically, but the "port" reporter only flags an actual
> unrecoverable error on port 0 - ports 1-3 come back clean. Combined
> with the dmesg output being identical on all 4 ports ("Get PHY
> capabilities failed status = -5" / "ice_init_nvm_phy_type failed: -5"),
> our working theory is that netlist/topology processing walks the
> node table in order, hits an unrecoverable condition at the first
> cage node (port 0), and aborts the entire walk - which would explain
> why ports 1-3 are simply never reached/checked (hence "healthy" by
> omission) while the overall topology init still fails for all 4 PFs.
>
> We're fairly confident this isn't a NVM data-integrity problem in the
> usual sense:
> - The board is already on the latest available NVM/FW (5.01,
>    EETRACK 80021FBF; nvmupdate reports "Update Not Available")
> - We reverse-engineered the Link Topology Netlist (TLV 0x11B) from
>    our image and confirmed it's byte-for-byte identical to Intel's own
>    E810_XXVDA4 reference image at the node/IO table level (PCA9575 I2C
>    GPIO expander correctly present at Bus 4, RESET_N/INT_N on native
>    SDP5/SDP6, matching the datasheet's "4x SFP Native" reference
>    schematic, Section 17.2.2.1)
> - Re-verified this against the actual flashed chip content (ethtool -e
>    readback) after a fresh NVM update - identical result
> - NVM Update Checksum (AQ 0x0706) reports valid
> - GL_MNG_FWSM and GL_MNG_SHA_EXTEND_STATUS (read via UEFI Shell/BAR0,
>    pre-OS) show a completely clean boot: main FW authenticated (STAGE=
>    100b), no POR_AL_FAILURE/PCIR_AL_FAILURE, no EXT_ERR_IND, FW_MODES
>    bit1 (Recovery Mode) = 0
>
> Given all of that checks out, "corrupted" seems to mean something
> more like a structural/semantic validation failure specific to the
> port-0 cage node (or something processed immediately before it),
> rather than actual bit-level data corruption - but we don't have
> visibility into what that validation actually checks.
>
> Questions, now with concrete syndrome data attached:
>
> - What does the firmware's netlist walk actually validate at/around
>    the first cage node that could produce "Unrecoverable netlist error"
>    (syndrome 275) specifically for port 0, while leaving ports 1-3
>    unflagged?
> - Is there a way to get more granular logging out of this
>    specifically (we tried fwlog modules i2c/link_topo/health/adminq via
>    debugfs, and can correlate some opaque message IDs with specific
>    AQ calls, but can't decode their actual content without an internal
>    format-string table)?
> - The suggested remediation ("update to latest NVM image") doesn't
>    apply since we're already on the latest - is there a separate
>    netlist/board-config bundle that could be out of sync independent
>    of the FW/NVM version number?
>
> Happy to share the full netlist dump, oscilloscope captures, and raw
> AQ traces if that's useful to anyone looking into this.
>
> Thanks,
> Fabrice



Impressum/Imprint: https://www.ipetronik.com/impressum