Re: Python access to video frames (Carl Karsten)
Marianna Smidth Buschle via gstreamer-devel <[email protected]> Tue, 17 Oct 2023 11:19:21 +0200
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Organization | Qtechnology |
| Message-ID | <[email protected]> |
The demuxer divides the pipeline into audio and video, you get one in each branch. Right now you seem to have both an audioparser and a videoencoder in the same branch. Try instead: gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 ! queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue ! mpegaudioparse ! mux1. And generally, when in doubt about pipeline errors it is always useful to simplify it and then build up. Fx: filesrc ! fakesink then filesrc ! demux ! fakesink then filesrc ! demux ! decode ! fakesink then filesrc ! demux ! decode ! videosink ... Best Regards Marianna On 16/10/2023 20.00, Carl Karsten wrote: > Ima need a little help with the pipeline... > > # probably needs an aacparse or parser for whatever the codec of the > audio stream is > # also you need queues after both branches of the demuxer > > gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 ! > queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux > name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue ! > mux1. > > > ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1 > '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!' > mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1. > '!' queue '!' mux1. > Setting pipeline to PAUSED ... > Pipeline is PREROLLING ... > ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal > data stream error. > Additional debug info: > ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop (): > /GstPipeline:pipeline0/GstTSDemux:demux1: > streaming stopped, reason not-negotiated (-4) > ERROR: pipeline doesn't want to preroll. > Setting pipeline to NULL ... > ERROR: from element > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind: > Stream doesn't contain enough data. > Additional debug info: > ../plugins/elements/gsttypefindelement.c(1012): > gst_type_find_element_chain_do_typefinding (): > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind: > Can't typefind stream > ERROR: pipeline doesn't want to preroll. > Freeing pipeline ... > > 13_25_49.ts came from: > > ffmpeg \ > -nostdin -y \ > -analyzeduration 10000 \ > -thread_queue_size 512 \ > -i tcp://localhost:11000?timeout=3000000 \ > -aspect 16:9 \ > -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4 > -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5 > -me_method:v:0 dia \ > -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \ > -flags +global_header \ > -f segment -segment_time $segment_time \ > -segment_format mpegts \ > -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts" > > https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54 > > and in case more info helps: > > $ mediainfo 13_25_49.ts > General > ID : 1 (0x1) > Complete name : 13_25_49.ts > Format : MPEG-TS > Commercial name : HDV 720p > File size : 838 MiB > Duration : 30 min 0 s > Overall bit rate mode : Variable > Overall bit rate : 3 903 kb/s > Frame rate : 30.000 FPS > > Video > ID : 256 (0x100) > Menu ID : 1 (0x1) > Format : MPEG Video > Commercial name : HDV 720p > Format version : Version 2 > Format profile : Main@High 1440 > Format settings, BVOP : No > Format settings, Matrix : Default > Format settings, GOP : N=5 > Codec ID : 2 > Duration : 30 min 0 s > Bit rate mode : Variable > Bit rate : 3 323 kb/s > Width : 1 280 pixels > Height : 720 pixels > Display aspect ratio : 16:9 > Frame rate : 30.000 FPS > Color space : YUV > Chroma subsampling : 4:2:0 > Bit depth : 8 bits > Scan type : Progressive > Compression mode : Lossy > Bits/(Pixel*Frame) : 0.120 > Time code of first frame : 01:29:59:29 > Time code source : Group of pictures header > GOP, Open/Closed : Open > Stream size : 713 MiB (85%) > Color primaries : BT.709 > Transfer characteristics : BT.709 > Matrix coefficients : BT.709 > > Audio > ID : 257 (0x101) > Menu ID : 1 (0x1) > Format : MPEG Audio > Format version : Version 1 > Format profile : Layer 2 > Codec ID : 3 > Duration : 30 min 0 s > Bit rate mode : Constant > Bit rate : 384 kb/s > Channel(s) : 2 channels > Sampling rate : 48.0 kHz > Frame rate : 41.667 FPS (1152 SPF) > Compression mode : Lossy > Delay relative to video : -29 ms > Stream size : 82.4 MiB (10%) > Language : English > > Menu > ID : 4096 (0x1000) > Menu ID : 1 (0x1) > Format : MPEG Video / MPEG Audio > Duration : 30 min 0 s > List : 256 (0x100) (MPEG Video) / > 257 (0x101) (MPEG Audio, English) > Language : / English > Service name : Service01 > Service provider : FFmpeg > Service type : digital television > > > On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <[email protected]> wrote: >> As suggested you can go some different ways: >> >> >> - Make a python gst app: >> >> filesrc ! decodebin ! appsink >> >> opencv code >> >> appsrc ! encode ! filesink >> >> Passing the audio as well might be tricky... >> >> >> - Make a python gst app: >> >> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1 >> ! filesink demux1. ! mux1 >> >> and use a probe to snatch frames at the identify element and do what you >> want there with opencv >> >> >> - Make a filter element to do your stuff >> >> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 ! >> filesink demux1. ! mux1 >> >> >> Best Regards >> >> Marianna >> >> >> On 13/10/2023 14.00, [email protected] wrote: >>> I need to alter video frames, and Python is my happy place. >>> my gst skills need some help. >>> >>> I think I need these 2 things: >>> >>> 1. pipeline: >>> read a file, >>> decode the video, >>> hook? to read/write pixels, >>> encode (keep the original audio) >>> save to disk. >>> >>> 2. that hook code... >>> I have done something before: use the wavelevel element to get audio >>> data and generate static png images. >>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72 >>> >>> So I am assuming some element / message will give me a frame buffer >>> thing of data, and somehow i can detect black pixels. >>> >>> >>> End goal: fix some video;s I made of a live event where my setup >>> introduced black squares - I want to replace the black with the pixels >>> from the previous frame. >>> example of what needs fixing: >>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148 >>> >>> The slide is mostly a static image, so the previous frame will be fine. >>> >>> -- Carl K >> -- >> Best regards / Med venlig hilsen >> “Marianna Smidth Buschle” >> > -- Best regards / Med venlig hilsen “Marianna Smidth Buschle”