[PR] avfilter/vf_transpose_npp: restore dimensions on recycled frames (PR #24293)

PR #24293 opened by Forgejo_Fairy
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24293
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24293.patch

Fixes #21153.

`av_hwframe_get_buffer()` initializes the replacement frame dimensions from the aligned hardware frames context. `npptranspose_filter()` then recycles that replacement without restoring the final stage's visible dimensions, so later frames can advertise the allocation size rather than the image size.

Restore `width` and `height` from the final NPP stage before recycling the replacement frame.

Testing was performed on an RTX 5090 with NVIDIA driver 595.84 using the sample-independent reproducer from #21153:

```sh
ffmpeg -y \
  -init_hw_device cuda=cu -filter_hw_device cu \
  -f lavfi -i testsrc2=s=640x360 \
  -vf "format=yuv420p,hwupload_cuda,transpose_npp=cclock,transpose_npp=cclock,select='gte(n,1)'" \
  -c:v h264_nvenc -frames:v 10 out.mp4
```

The current release/7.1 head 3978a28d5bdd produces 640×384. With 51987710d9e applied, both visible and coded dimensions are 640×360. An additional 638×358 case likewise retains exactly 638×358. The branch builds successfully and passes `git diff --check`.

The same faulty code remains in release/8.0 and release/8.1. The filter was removed from master by 9f58e3014cf, which is why this fix targets an affected release branch.


From 51987710d9ecefdabdecf38a6307954b6635b15e Mon Sep 17 00:00:00 2001
From: Forgejo_Fairy <forgejo_fairy@localhost>
Date: Thu, 27 Aug 2026 16:36:21 +0000
Subject: [PATCH] avfilter/vf_transpose_npp: restore dimensions on recycled
 frames

av_hwframe_get_buffer() initializes the replacement frame dimensions from the aligned hardware frames context. Reset them to the final stage visible dimensions before recycling the frame, so later output frames do not advertise their allocation size.

Fixes #21153.
---
 libavfilter/vf_transpose_npp.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavfilter/vf_transpose_npp.c b/libavfilter/vf_transpose_npp.c
index 1706267b16..a1485d3fd4 100644
--- a/libavfilter/vf_transpose_npp.c
+++ b/libavfilter/vf_transpose_npp.c
@@ -376,6 +376,9 @@ static int npptranspose_filter(AVFilterContext *ctx, AVFrame *out, AVFrame *in)
     if (ret < 0)
         return ret;
 
+    s->tmp_frame->width  = s->stages[last_stage].planes_out[0].width;
+    s->tmp_frame->height = s->stages[last_stage].planes_out[0].height;
+
     av_frame_move_ref(out, src);
     av_frame_move_ref(src, s->tmp_frame);
 
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]