Re: Firmware Recovery Protocols
Arun <[email protected]>
| Newsgroups | org.ozlabs.lists.openbmc |
|---|---|
| Message-ID | <[email protected]> |
Hi all,
Following up on the proposal below, we now have a working implementation of the OCP Secure Firmware Recovery path that realizes this approach, and I'd like to put it up for review.
This covers Stage 1 of the two-stage flow described below: OCP recovery delivers the initial firmware over SMBus to a device parked in its recovery ROM; Stage 2 then proceeds over the
existing MCTP/PLDM T5 update stack.
Design summary:
- OCP recovery code updater: a Non-PLDM code updater built on the existing common/ SoftwareManager/Device framework in phosphor-bmc-code-mgmt. The operator-facing interface is
identical to a normal firmware update: a targeted Redfish multipart update against the device's FirmwareInventory entry, with the
recovery image(s) carried in a PLDM (DSP0267) package.
- Protocol library: a small transport-abstract C++ library implementing the OCP Secure Firmware Recovery 1.0 indirect flow over I2C/SMBus, unit-tested against a mock transport. A
standalone CLI (ocp-recovery-tool) drives the same library for bring-up and debug.
- Discovery: devices are described by a new entity-manager exposes type, OCPRecoveryFirmware, so recovery plugs directly into the standard Redfish targeted-update flow.
Testing: recovered eight NVIDIA RTX 6000 Pro (GB202) GPUs end-to-end over the Redfish update path, including forcing devices into recovery mode, skipping already-healthy devices, and
rejecting non-applicable packages (UpdateNotApplicable). The entity-manager configuration used for this is part of the series.
Changes for review (gerrit topic ocp-fw-recovery: https://gerrit.openbmc.org/q/topic:ocp-fw-recovery):
- phosphor-bmc-code-mgmt, split by area:
- libocp/ — protocol library: https://gerrit.openbmc.org/c/openbmc/phosphor-bmc-code-mgmt/+/92450
- libocp/tools/ — bring-up CLI: https://gerrit.openbmc.org/c/openbmc/phosphor-bmc-code-mgmt/+/92451
- common/ — multi-component package support: https://gerrit.openbmc.org/c/openbmc/phosphor-bmc-code-mgmt/+/92452 and a small D-Bus helper:
https://gerrit.openbmc.org/c/openbmc/phosphor-bmc-code-mgmt/+/93329
- ocp-recovery/ — the updater daemon (incl. README): https://gerrit.openbmc.org/c/openbmc/phosphor-bmc-code-mgmt/+/92453
- entity-manager: OCPRecoveryFirmware schema (https://gerrit.openbmc.org/c/openbmc/entity-manager/+/93330) and the RTX 6000 Pro configuration used for validation
(https://gerrit.openbmc.org/c/openbmc/entity-manager/+/93331)
- OCP Secure Firmware Recovery 1.0 spec: https://www.opencompute.org/documents/ocp-recovery-document-1p0-final-1-pdf
Feedback on the design is welcome, in particular the Non-PLDM-updater framing and the entity-manager configuration.
Thanks,
Arun
> On 15 Aug 2025, at 15:25, Tom Joseph <[email protected]> wrote:
>
> Hello all,
>
> Devices such as GPUs, CPUs, and NICs support firmware update using
> standard protocols like MCTP and PLDM T5. They also support recovery
> protocols, providing out-of-band mechanisms to detect when a device enters
> a fault state where firmware update is not possible and to securely restore
> it to a known good condition.
>
> The Open Compute Project (OCP) offers a secure firmware protocol [1] to
> determine if a device is in recovery mode and to facilitate recovery
> through the transfer of recovery images, typically over SMBus. Firmware
> recovery generally follows a two-stage process: OCP recovery is first used
> to deliver the Stage 1 (initial) firmware to the device, which then enables
> further updates through MCTP/PLDM T5. Stage 2 involves transferring
> complete device firmware using MCTP/PLDM T5, which is already supported by
> the OpenBMC stack. There are also similar custom protocols for device
> recovery, some of which support two stage updates while others recover with
> a single-stage update.
>
> To bring these capabilities into the OpenBMC ecosystem, the proposal is to
> leverage the standard Redfish API for code updates and utilize PLDM
> packaging for recovery images. Recovery protocols like OCP would be
> implemented as Non-PLDM code updaters, which handle the specific recovery
> protocol and parse PLDM packages to extract the appropriate recovery images
> for each device. This approach builds on the current efforts to update
> non-PLDM devices, such as VRs and CPLDs, with PLDM image packaging.
>
> I welcome feedback from the community on the approach and invite any
> alternative suggestions.
>
> [1]
> https://www.opencompute.org/documents/ocp-recovery-document-1p0-final-1-pdf
> [2] https://gerrit.openbmc.org/c/openbmc/docs/+/76645
> Regards,
> Tom
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20250815/ee747a17/attachment.htm>