[bmcweb] Proposal: RFC 9110 Range Request support for large file-body responses (BMC dump offload)
Abhilash Raju <[email protected]> Mon, 27 Jul 2026 05:29:57 +0000
| Newsgroups | org.ozlabs.lists.openbmc |
|---|---|
| Message-ID | <MW4PR15MB4732A5606EC4B75814BC9421E5CC2@MW4PR15MB4732.namprd15.prod.outlook.com> |
--_000_MW4PR15MB4732A5606EC4B75814BC9421E5CC2MW4PR15MB4732namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi all, I'd like to propose adding RFC 9110 =A714 (Range Requests) support to bmcweb's HTTP file-body layer, driven by a concrete operational problem with BMC diagnostic dump offload. THE PROBLEM ----------- Offloading a BMC diagnostic dump over Redfish today means issuing a single GET against the dump attachment URI: GET /redfish/v1/Managers/bmc/LogServices/Dump/Entries/<id>/attachment/ Dump files routinely fall in the 500 MB - 1 GB range. This creates two failure modes on constrained or out-of-band management networks: 1. Timeout mid-transfer. The connection's idle timer fires before the transfer completes, killing an otherwise healthy transfer. There is no way to resume -- the client must restart from byte zero. 2. All-or-nothing cost. Any network interruption forces a full restart. There is no mechanism to fetch parts in parallel or retry only the failed segment. PROPOSED SOLUTION: TWO COMPLEMENTARY PATCHES --------------------------------------------- Patch 1 -- Per-chunk deadline reset Install a chunkCallback in HttpBody::value_type that fires after each file chunk is acknowledged by the client. The connection uses it to cancel and restart the idle deadline timer, giving the client a fresh idle window per chunk rather than a single wall-clock deadline covering the entire transfer. A truly stalled connection still times out; a slow-but-making-progress one does not. This is a transparent fix -- no API or client change required. Patch 2 -- RFC 9110 Range Request support Add setFdWithRange() / openFdWithRange() to the HTTP body layer so a file descriptor can be streamed from an arbitrary offset for a given byte count. Wire this into the existing BMC dump attachment URI to honour the standard Range request header: GET /redfish/v1/Managers/bmc/LogServices/Dump/Entries/<id>/attachment/ Range: bytes=3D0-524287 The server responds with: 206 Partial Content Content-Range: bytes 0-524287/1073741824 Accept-Ranges: bytes Clients can split the file into N byte ranges, fetch them concurrently across independent connections, and reassemble the parts locally. A failed segment is retried in isolation -- not the whole file. This is fully interoperable with standard HTTP clients (curl, wget, Java's HttpClient) without any custom tooling. Implementation details: - clampLengthToFileContent() uses fstat(2) on the returned D-Bus unix_fd to resolve the total file size and clamp any over-estimated length, guarding against stale AdditionalDataSizeBytes values. - Ranges are validated per RFC 9110 =A714.1; an out-of-range request returns 416 Range Not Satisfiable with a Content-Range: bytes */ body as required by =A714.4. - The existing full-download path is unchanged. Clients that do not send a Range header receive the same 200 OK full response as today, now with Accept-Ranges: bytes to advertise the capability. - The 20 MB checkSizeLimit guard in downloadEntryCallback is intentionally bypassed on the ranged path -- the client is responsible for splitting the file into suitably sized requests. WHAT THIS DOES NOT DO --------------------- This proposal does not implement HTTP multi-range (Range: bytes=3D0-99,200-299) -- only single contiguous ranges, which is all that is needed for parallel chunked offload. Multi-range can be added later if there is demand. This also does not change the Redfish schema or add any new URI surface. The capability is expressed entirely through standard HTTP headers on an existing URI. OPEN QUESTIONS FOR THE COMMUNITY --------------------------------- 1. Is the per-chunk deadline reset granularity in Patch 1 acceptable, or would a different idle-timeout strategy be preferred for large responses? 2. The checkSizeLimit 20 MB cap was originally added to "accommodate BMC dumps" but in practice blocks any dump larger than 20 MB from being downloaded at all via the full-download path. Should this limit be revisited independently of this proposal? 3. Are there concerns about advertising Accept-Ranges: bytes on the BMC dump attachment endpoint that I have not considered? PATCHES ------- The two patches are posted on Gerrit for review: - Patch 1: http: add per-chunk deadline reset for file-body responses https://gerrit.openbmc.org/c/openbmc/bmcweb/+/92612 - Patch 2: http: add ranged file-body support with send-limit enforcement https://gerrit.openbmc.org/c/openbmc/bmcweb/+/92729 Both are independent and can be reviewed and merged separately. Any feedback, concerns, or alternative approaches are very welcome. Thanks, Abhilash Raju --_000_MW4PR15MB4732A5606EC4B75814BC9421E5CC2MW4PR15MB4732namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> </head> <body> <pre style=3D"margin: 0px;"><div style=3D"text-align: left; text-indent: 0p= x; line-height: 1.65; white-space: pre-wrap; text-transform: none; font-siz= e: 13px;"><span style=3D"color: rgb(31, 35, 40);">Hi all,=0A= =0A= I'd like to propose adding RFC 9110 =A714 (Range Requests) support to=0A= bmcweb's HTTP file-body layer, driven by a concrete operational problem=0A= with BMC diagnostic dump offload.=0A= =0A= =0A= THE PROBLEM=0A= -----------=0A= =0A= Offloading a BMC diagnostic dump over Redfish today means issuing a=0A= single GET against the dump attachment URI:=0A= =0A= GET /redfish/v1/Managers/bmc/LogServices/Dump/Entries/<id>/att= achment/=0A= =0A= Dump files routinely fall in the 500 MB - 1 GB range. This creates=0A= two failure modes on constrained or out-of-band management networks:=0A= =0A= 1. Timeout mid-transfer. The connection's idle timer fires before the=0A= transfer completes, killing an otherwise healthy transfer. There is= =0A= no way to resume -- the client must restart from byte zero.=0A= =0A= 2. All-or-nothing cost. Any network interruption forces a full restart.=0A= There is no mechanism to fetch parts in parallel or retry only the= =0A= failed segment.=0A= =0A= =0A= PROPOSED SOLUTION: TWO COMPLEMENTARY PATCHES=0A= ---------------------------------------------=0A= =0A= Patch 1 -- Per-chunk deadline reset=0A= =0A= Install a chunkCallback in HttpBody::value_type that fires after each=0A= file chunk is acknowledged by the client. The connection uses it to=0A= cancel and restart the idle deadline timer, giving the client a fresh=0A= idle window per chunk rather than a single wall-clock deadline covering=0A= the entire transfer. A truly stalled connection still times out; a=0A= slow-but-making-progress one does not.=0A= =0A= This is a transparent fix -- no API or client change required.=0A= =0A= Patch 2 -- RFC 9110 Range Request support=0A= =0A= Add setFdWithRange() / openFdWithRange() to the HTTP body layer so a=0A= file descriptor can be streamed from an arbitrary offset for a given=0A= byte count. Wire this into the existing BMC dump attachment URI to=0A= honour the standard Range request header:=0A= =0A= GET /redfish/v1/Managers/bmc/LogServices/Dump/Entries/<id>/att= achment/=0A= Range: bytes=3D0-524287=0A= =0A= The server responds with:=0A= =0A= 206 Partial Content=0A= Content-Range: bytes 0-524287/1073741824=0A= Accept-Ranges: bytes=0A= =0A= Clients can split the file into N byte ranges, fetch them concurrently=0A= across independent connections, and reassemble the parts locally. A=0A= failed segment is retried in isolation -- not the whole file. This is=0A= fully interoperable with standard HTTP clients (curl, wget, Java's=0A= HttpClient) without any custom tooling.=0A= =0A= Implementation details:=0A= =0A= - clampLengthToFileContent() uses fstat(2) on the returned D-Bus=0A= unix_fd to resolve the total file size and clamp any over-estimated= =0A= length, guarding against stale AdditionalDataSizeBytes values.=0A= =0A= - Ranges are validated per RFC 9110 =A714.1; an out-of-range request=0A= returns 416 Range Not Satisfiable with a Content-Range: bytes */=0A= body as required by =A714.4.=0A= =0A= - The existing full-download path is unchanged. Clients that do not=0A= send a Range header receive the same 200 OK full response as today,= =0A= now with Accept-Ranges: bytes to advertise the capability.=0A= =0A= - The 20 MB checkSizeLimit guard in downloadEntryCallback is=0A= intentionally bypassed on the ranged path -- the client is responsib= le=0A= for splitting the file into suitably sized requests.=0A= =0A= =0A= WHAT THIS DOES NOT DO=0A= ---------------------=0A= =0A= This proposal does not implement HTTP multi-range=0A= (Range: bytes=3D0-99,200-299) -- only single contiguous ranges, which is=0A= all that is needed for parallel chunked offload. Multi-range can be=0A= added later if there is demand.=0A= =0A= This also does not change the Redfish schema or add any new URI=0A= surface. The capability is expressed entirely through standard HTTP=0A= headers on an existing URI.=0A= =0A= =0A= OPEN QUESTIONS FOR THE COMMUNITY=0A= ---------------------------------=0A= =0A= 1. Is the per-chunk deadline reset granularity in Patch 1 acceptable,=0A= or would a different idle-timeout strategy be preferred for large= =0A= responses?=0A= =0A= 2. The checkSizeLimit 20 MB cap was originally added to "accommodate= =0A= BMC dumps" but in practice blocks any dump larger than 20 MB f= rom=0A= being downloaded at all via the full-download path. Should this=0A= limit be revisited independently of this proposal?=0A= =0A= 3. Are there concerns about advertising Accept-Ranges: bytes on the=0A= BMC dump attachment endpoint that I have not considered?=0A= =0A= =0A= PATCHES=0A= -------=0A= =0A= The two patches are posted on Gerrit for review:=0A= =0A= - Patch 1: http: add per-chunk deadline reset for file-body responses=0A= </span><span style=3D"color: rgb(0, 0, 0);"><a href=3D"https://gerri= t.openbmc.org/c/openbmc/bmcweb/+/92612" data-outlook-id=3D"3af7d3cc-9a5e-42= bd-8525-fe0a8e4d8106">https://gerrit.openbmc.org/c/openbmc/bmcweb/+/92612</= a> </span><span style=3D"color: rgb(31, 35, 40);">=0A= =0A= - Patch 2: http: add ranged file-body support with send-limit enforcement= =0A= </span><span style=3D"color: rgb(0, 0, 0);"><a href=3D"https://gerri= t.openbmc.org/c/openbmc/bmcweb/+/92729" data-outlook-id=3D"c45ff4ee-a10b-4b= dd-8219-fbef91c2c626">https://gerrit.openbmc.org/c/openbmc/bmcweb/+/92729</= a> </span><span style=3D"color: rgb(31, 35, 40);">=0A= =0A= Both are independent and can be reviewed and merged separately.=0A= =0A= Any feedback, concerns, or alternative approaches are very welcome.=0A= =0A= Thanks,=0A= Abhilash Raju</span></div></pre> <div style=3D"direction: ltr; font-family: -apple-system, "Segoe UI&qu= ot;, system-ui, sans-serif; font-size: 13.5px; color: rgb(0, 0, 0);"> <span style=3D"background-color: rgb(247, 248, 250);"><br> </span></div> </body> </html> --_000_MW4PR15MB4732A5606EC4B75814BC9421E5CC2MW4PR15MB4732namp_--