[PR] avfilter/vf_amf_common: fix component Init() format with hwaccel input frames (PR #23874)

PR #23874 opened by JuliusBairaktaris
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23874
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23874.patch

All AMF filters failed component Init() when fed D3D11VA/DXVA2 hw frames because the hw pixel format was passed to av_av_to_amf_format(). Root cause, fix rationale and a verification matrix (RX 9070 XT, Windows 11) are in the commit message. Follow-up to #23536: frc_amf and dxva2 input are fully fixed; compute-based components additionally need shader-capable decode textures (SHADER=1 device option).


>From 3f04efbb71b5b3804518b76e2b53bfba3e19181c Mon Sep 17 00:00:00 2001
From: Julius Bairaktaris <[email protected]>
Date: Thu, 23 Jul 2026 01:18:23 +0200
Subject: [PATCH] avfilter/vf_amf_common: fix component Init() format with
 hwaccel input frames

amf_init_filter_config() returned the hardware pixel format (e.g.
AV_PIX_FMT_D3D11) as in_format when the input link carries D3D11VA or
DXVA2 hw frames. av_av_to_amf_format() maps hw formats to
AMF_SURFACE_UNKNOWN, so every AMF filter failed its component Init()
with AMF_INVALID_ARG (VQEnhancer, FRC) or AMF_NOT_SUPPORTED
(Converter, HQScaler). Return the underlying software format instead,
which is already validated against AMF a few lines above. Apply the
same mapping to the default output sw format, which otherwise ends up
as the hw pixel format in hwframes_out->sw_format when the output link
uses a hwaccel format.

Fixes e.g.:
  ffmpeg -hwaccel d3d11va -hwaccel_output_format d3d11 -i in.mp4          -vf frc_amf -c:v hevc_amf out.mp4

Note: with D3D11VA input, the compute-based components (VQEnhancer,
HQScaler, VideoConverter) additionally require decode textures created
with D3D11_BIND_SHADER_RESOURCE, e.g.:
  -init_hw_device d3d11va=dx11:,SHADER=1 -hwaccel_device dx11
FRC and DXVA2 input work with default decode textures.
---
 libavfilter/vf_amf_common.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/libavfilter/vf_amf_common.c b/libavfilter/vf_amf_common.c
index e18c858eda..40c6fbceea 100644
--- a/libavfilter/vf_amf_common.c
+++ b/libavfilter/vf_amf_common.c
@@ -24,6 +24,7 @@
 #include "formats.h"
 #include "libavutil/mem.h"
 #include "libavutil/imgutils.h"
+#include "libavutil/pixdesc.h"
 
 #include "AMF/components/VideoDecoderUVD.h"
 #include "libavutil/hwcontext_amf.h"
@@ -333,8 +334,9 @@ int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format)
         AMF_RETURN_IF_FALSE(avctx, res == 0, res, "Failed to create  hardware device context (AMF) : %s\n", av_err2str(res));
 
     }
-    if(out_sw_format == AV_PIX_FMT_NONE){
-        if(outlink->format == AV_PIX_FMT_AMF_SURFACE)
+    if (out_sw_format == AV_PIX_FMT_NONE) {
+        const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(outlink->format);
+        if (desc && (desc->flags & AV_PIX_FMT_FLAG_HWACCEL))
             out_sw_format = in_sw_format;
         else
             out_sw_format = outlink->format;
@@ -351,11 +353,10 @@ int amf_init_filter_config(AVFilterLink *outlink, enum AVPixelFormat *in_format)
     hwframes_out->format    = AV_PIX_FMT_AMF_SURFACE;
     hwframes_out->sw_format = out_sw_format;
 
-    if (inlink->format == AV_PIX_FMT_AMF_SURFACE) {
-        *in_format = in_sw_format;
-    } else {
-        *in_format = inlink->format;
-    }
+    // in_sw_format is inlink->format for software input and the underlying
+    // sw_format for any hw frames input (AMF, D3D11, DXVA2); the component
+    // must always be initialized with a software surface format.
+    *in_format = in_sw_format;
     outlink->w = ctx->width;
     outlink->h = ctx->height;
 
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]