[PR] fftools/ffmpeg_dec: do not substitute a different hardware format (PR #24291)

PR #24291 opened by Diego de Souza (ddesouza)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24291
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24291.patch

A hardware -hwaccel_output_format was treated as a preference:
get_format() settled for another hardware format of the same device type
when the requested one was unavailable. CUDA and CUARRAY share a device
type, so this is reachable in practice.

The frames were then unusable. hwaccel_retrieve_data() asked
av_hwframe_transfer_data() for the requested format, but the destination
frame carries no frames context, so the transfer took the allocating
path and failed on a hardware format. No frames were emitted at all.

Substituting is not desirable even where the frames would be usable: the
option exists so that the rest of the pipeline can rely on receiving a
specific format, and quietly producing a different one defeats that.

Treat a hardware -hwaccel_output_format as a constraint on format
selection and fall back to software decoding when it cannot be
satisfied, as already happens when an hwaccel cannot be set up at all.
A software -hwaccel_output_format still only requests a download.


>From 9aa2cb15b9115d41f5befd9165a49c8233f4ff5d Mon Sep 17 00:00:00 2001
From: Diego de Souza <[email protected]>
Date: Tue, 25 Aug 2026 18:04:52 +0200
Subject: [PATCH] fftools/ffmpeg_dec: do not substitute a different hardware
 format

A hardware -hwaccel_output_format was treated as a preference:
get_format() settled for another hardware format of the same device type
when the requested one was unavailable. CUDA and CUARRAY share a device
type, so this is reachable in practice.

The frames were then unusable. hwaccel_retrieve_data() asked
av_hwframe_transfer_data() for the requested format, but the destination
frame carries no frames context, so the transfer took the allocating
path and failed on a hardware format. No frames were emitted at all.

Substituting is not desirable even where the frames would be usable: the
option exists so that the rest of the pipeline can rely on receiving a
specific format, and quietly producing a different one defeats that.

Treat a hardware -hwaccel_output_format as a constraint on format
selection and fall back to software decoding when it cannot be
satisfied, as already happens when an hwaccel cannot be set up at all.
A software -hwaccel_output_format still only requests a download.

Signed-off-by: Diego de Souza <[email protected]>
---
 fftools/ffmpeg_dec.c | 41 ++++++++++++++++++++---------------------
 1 file changed, 20 insertions(+), 21 deletions(-)

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 1498afa880..5e7467797b 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -1314,6 +1314,7 @@ static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat
 {
     DecoderPriv  *dp = s->opaque;
     const enum AVPixelFormat *p;
+    int hwaccel_output_is_hw;
     int ret;
 
     ret = multiview_setup(dp, s);
@@ -1325,6 +1326,14 @@ static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat
 
     dp->hwaccel_pix_fmt = AV_PIX_FMT_NONE;
 
+    /* A hardware hwaccel_output_format constrains which hardware format may
+     * be selected, rather than requesting a conversion: frames are not
+     * implicitly transferred between two hardware formats. A software one
+     * only asks for a download and imposes no such constraint. */
+    hwaccel_output_is_hw = dp->hwaccel_output_format != AV_PIX_FMT_NONE &&
+                           (av_pix_fmt_desc_get(dp->hwaccel_output_format)->flags &
+                            AV_PIX_FMT_FLAG_HWACCEL);
+
     for (p = pix_fmts; *p != AV_PIX_FMT_NONE; p++) {
         const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(*p);
         const AVCodecHWConfig  *config = NULL;
@@ -1332,6 +1341,9 @@ static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat
         if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL))
             break;
 
+        if (hwaccel_output_is_hw && *p != dp->hwaccel_output_format)
+            continue;
+
         if (dp->hwaccel_id == HWACCEL_GENERIC ||
             dp->hwaccel_id == HWACCEL_AUTO) {
             for (int i = 0;; i++) {
@@ -1347,31 +1359,18 @@ static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat
         }
         if (config && config->device_type == dp->hwaccel_device_type) {
             dp->hwaccel_pix_fmt = *p;
-            /* Stop at the first matching hardware format unless the user
-             * explicitly requested a different *hardware* output format
-             * (e.g. CUARRAY vs CUDA, which share a device type) - in that
-             * case keep scanning for the exact match. A software
-             * hwaccel_output_format requests a download and imposes no such
-             * preference, so it must not switch us off the default (first)
-             * hardware format. */
-            if (dp->hwaccel_output_format == AV_PIX_FMT_NONE ||
-                dp->hwaccel_output_format == *p ||
-                !(av_pix_fmt_desc_get(dp->hwaccel_output_format)->flags & AV_PIX_FMT_FLAG_HWACCEL))
-                break;
+            break;
         }
     }
 
-    if (dp->hwaccel_pix_fmt != AV_PIX_FMT_NONE) {
-        if (dp->hwaccel_output_format != AV_PIX_FMT_NONE &&
-            dp->hwaccel_output_format != dp->hwaccel_pix_fmt &&
-            (av_pix_fmt_desc_get(dp->hwaccel_output_format)->flags & AV_PIX_FMT_FLAG_HWACCEL))
-            av_log(dp, AV_LOG_WARNING,
-                   "Requested hwaccel output format '%s' not available, "
-                   "falling back to '%s'\n",
-                   av_get_pix_fmt_name(dp->hwaccel_output_format),
-                   av_get_pix_fmt_name(dp->hwaccel_pix_fmt));
+    if (dp->hwaccel_pix_fmt != AV_PIX_FMT_NONE)
         return dp->hwaccel_pix_fmt;
-    }
+
+    if (hwaccel_output_is_hw && *p != AV_PIX_FMT_NONE)
+        av_log(dp, AV_LOG_WARNING,
+               "Requested hwaccel output format '%s' is not available, "
+               "falling back to software decoding\n",
+               av_get_pix_fmt_name(dp->hwaccel_output_format));
 
     return *p;
 }
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]