[PATCH 3/4] media: vicodec: clamp visible dimensions on S_FMT to coded bounds

From: Junrui Luo <[email protected]>

Setting a smaller CAPTURE format on the stateful decoder lets the visible
dimensions exceed the coded ones, and decode_plane() then writes past the
end of the capture buffer -- a kernel heap out-of-bounds write
controllable from unprivileged userspace.

vidioc_s_fmt() writes coded_width, coded_height and sizeimage but never
touches visible_width or visible_height, which vicodec_open() initialised
to 1280x720, so shrinking the coded pair leaves the visible pair larger.
job_ready() compares the bitstream header against the stale visible
dimensions, so no source-change event fires and decode_plane() iterates
over the visible height at a stride derived from coded_width.

On a KASAN-enabled kernel, a 1280x720 I-frame decoded into a 640x368
CAPTURE buffer triggers:

  BUG: KASAN: vmalloc-out-of-bounds in decode_plane+0x1bb8/0x34b0
  Write of size 1 at addr ffffc900004b8080 by task trigger_bin/69

  Call Trace:
   decode_plane+0x1bb8/0x34b0
   fwht_decode_frame+0x173/0x620
   v4l2_fwht_decode+0x595/0xea0

Clamp visible_width and visible_height to the new coded bounds after
every S_FMT, maintaining the invariant visible <= coded that
update_capture_data_from_header() relies on.

Fixes: 3b15f68e19c2 ("media: vicodec: Add support for resolution change event.")
Reported-by: Yuhao Jiang <[email protected]>
Assisted-by: Claude:claude-opus-5
Cc: [email protected]
Signed-off-by: Junrui Luo <[email protected]>
---
 drivers/media/test-drivers/vicodec/vicodec-core.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/media/test-drivers/vicodec/vicodec-core.c b/drivers/media/test-drivers/vicodec/vicodec-core.c
index 318e8330f16a..36b92f68ac42 100644
--- a/drivers/media/test-drivers/vicodec/vicodec-core.c
+++ b/drivers/media/test-drivers/vicodec/vicodec-core.c
@@ -1029,6 +1029,11 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		return -EINVAL;
 	}
 
+	q_data->visible_width = min(q_data->visible_width,
+				    q_data->coded_width);
+	q_data->visible_height = min(q_data->visible_height,
+				     q_data->coded_height);
+
 	dprintk(ctx->dev,
 		"Setting format for type %d, coded wxh: %dx%d, fourcc: 0x%08x\n",
 		f->type, q_data->coded_width, q_data->coded_height,

-- 
2.51.2