Python access to video frames

I need to alter video frames, and Python is my happy place.
my gst skills need some help.

I think I need these 2 things:

1. pipeline:
read a file,
decode the video,
hook? to read/write pixels,
encode (keep the original audio)
save to disk.

2. that hook code...
I have done something before: use the wavelevel element to get audio
data and generate static png images.
https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72

So I am assuming some element / message will give me a frame buffer
thing of data, and somehow i can detect black pixels.


End goal: fix some video;s I made of a live event where my setup
introduced black squares - I want to replace the black with the pixels
from the previous frame.
example of what needs fixing:
https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148

The slide is mostly a static image, so the previous frame will be fine.

-- 
Carl K