[PR] libswscale/ppc/swscale_vsx: fix heap buffer overflow in WRITERGB for RGB24/BGR24 (PR #24219)

PR #24219 opened by yongdev
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24219
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24219.patch

In WRITERGB for AV_PIX_FMT_RGB24 and AV_PIX_FMT_BGR24, vec_vsx_st(out1, 16, dest)
performs a 16-byte vector store starting at offset 16 (writing 32 bytes in total
into dest). Because RGB24/BGR24 pixels require only 24 bytes per iteration, the
16-byte write overshoots by 8 bytes, causing an out-of-bounds heap buffer overflow
on the trailing slice pixels.

Fix this by storing out1 into an aligned temporary buffer and copying only the
remaining 8 bytes to dest + 16 via memcpy, preventing the 8-byte buffer overrun.

Signed-off-by: yongdev <[email protected]>

# Summary of changes

Briefly describe what this PR does and why.

<!--
If this PR requires new FATE test samples, attach them to the PR and
list their target paths below (relative to the fate-suite root).

Attached filenames must match the sample's filename:

```fate-samples
# e.g. vorbis/new-sample.ogg
```
-->



>From 92958d405bbe572e37ddc41615b684c6f759c4ac Mon Sep 17 00:00:00 2001
From: yongdev <[email protected]>
Date: Mon, 17 Aug 2026 17:20:43 +0000
Subject: [PATCH] libswscale/ppc/swscale_vsx: fix heap buffer overflow in
 WRITERGB for RGB24/BGR24

In WRITERGB for AV_PIX_FMT_RGB24 and AV_PIX_FMT_BGR24, vec_vsx_st(out1, 16, dest)
performs a 16-byte vector store starting at offset 16 (writing 32 bytes in total
into dest). Because RGB24/BGR24 pixels require only 24 bytes per iteration, the
16-byte write overshoots by 8 bytes, causing an out-of-bounds heap buffer overflow
on the trailing slice pixels.

Fix this by storing out1 into an aligned temporary buffer and copying only the
remaining 8 bytes to dest + 16 via memcpy, preventing the 8-byte buffer overrun.

Signed-off-by: yongdev <[email protected]>
---
 libswscale/ppc/swscale_vsx.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/libswscale/ppc/swscale_vsx.c b/libswscale/ppc/swscale_vsx.c
index 8bbe6cde65..f61d79a625 100644
--- a/libswscale/ppc/swscale_vsx.c
+++ b/libswscale/ppc/swscale_vsx.c
@@ -457,28 +457,32 @@ yuv2NBPSX(16, LE, 0, 16, int32_t)
         bd = vec_packsu(bd16, zero16); \
 \
         switch(target) { \
-        case AV_PIX_FMT_RGB24: \
+        case AV_PIX_FMT_RGB24: { \
+            DECLARE_ALIGNED(16, uint8_t, tmp_out)[16]; \
             out0 = vec_perm(rd, gd, perm3rg0); \
             out0 = vec_perm(out0, bd, perm3tb0); \
             out1 = vec_perm(rd, gd, perm3rg1); \
             out1 = vec_perm(out1, bd, perm3tb1); \
 \
             vec_vsx_st(out0, 0, dest); \
-            vec_vsx_st(out1, 16, dest); \
+            vec_vsx_st(out1, 0, tmp_out); \
+            memcpy(dest + 16, tmp_out, 8); \
 \
             dest += 24; \
-        break; \
-        case AV_PIX_FMT_BGR24: \
+        break; } \
+        case AV_PIX_FMT_BGR24: { \
+            DECLARE_ALIGNED(16, uint8_t, tmp_out)[16]; \
             out0 = vec_perm(bd, gd, perm3rg0); \
             out0 = vec_perm(out0, rd, perm3tb0); \
             out1 = vec_perm(bd, gd, perm3rg1); \
             out1 = vec_perm(out1, rd, perm3tb1); \
 \
             vec_vsx_st(out0, 0, dest); \
-            vec_vsx_st(out1, 16, dest); \
+            vec_vsx_st(out1, 0, tmp_out); \
+            memcpy(dest + 16, tmp_out, 8); \
 \
             dest += 24; \
-        break; \
+        break; } \
         case AV_PIX_FMT_BGRA: \
             out0 = vec_mergeh(bd, gd); \
             out1 = vec_mergeh(rd, ad); \
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]