Re: Python access to video frames (Carl Karsten)
Carl Karsten via gstreamer-devel <[email protected]> Tue, 17 Oct 2023 12:32:57 -0500
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Message-ID | <CADmzSSifedhQk8zyrBgsRE+2RPHu_nYstQz8FdJgMH9GMTAYXQ@mail.gmail.com> |
no error, but no play. gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1 '!' queue '!' decodebin3 '!' avenc_mpeg2video '!' mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1. '!' queue '!' mpegaudioparse '!' mux1. Setting pipeline to PAUSED ... Pipeline is PREROLLING ... htop shows no cpu and ls -l shows no bytes. On Tue, Oct 17, 2023 at 4:19 AM Marianna Smidth Buschle <[email protected]> wrote: > > The demuxer divides the pipeline into audio and video, you get one in > each branch. > > Right now you seem to have both an audioparser and a videoencoder in the > same branch. > > Try instead: > > gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 ! > queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts > demux1. ! queue ! mpegaudioparse ! mux1. > > > And generally, when in doubt about pipeline errors it is always useful > to simplify it and then build up. > Fx: filesrc ! fakesink > then filesrc ! demux ! fakesink > then filesrc ! demux ! decode ! fakesink > then filesrc ! demux ! decode ! videosink > ... > > Best Regards > Marianna > > On 16/10/2023 20.00, Carl Karsten wrote: > > Ima need a little help with the pipeline... > > > > # probably needs an aacparse or parser for whatever the codec of the > > audio stream is > > # also you need queues after both branches of the demuxer > > > > gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 ! > > queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux > > name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue ! > > mux1. > > > > > > ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1 > > '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!' > > mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1. > > '!' queue '!' mux1. > > Setting pipeline to PAUSED ... > > Pipeline is PREROLLING ... > > ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal > > data stream error. > > Additional debug info: > > ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop (): > > /GstPipeline:pipeline0/GstTSDemux:demux1: > > streaming stopped, reason not-negotiated (-4) > > ERROR: pipeline doesn't want to preroll. > > Setting pipeline to NULL ... > > ERROR: from element > > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind: > > Stream doesn't contain enough data. > > Additional debug info: > > ../plugins/elements/gsttypefindelement.c(1012): > > gst_type_find_element_chain_do_typefinding (): > > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind: > > Can't typefind stream > > ERROR: pipeline doesn't want to preroll. > > Freeing pipeline ... > > > > 13_25_49.ts came from: > > > > ffmpeg \ > > -nostdin -y \ > > -analyzeduration 10000 \ > > -thread_queue_size 512 \ > > -i tcp://localhost:11000?timeout=3000000 \ > > -aspect 16:9 \ > > -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4 > > -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5 > > -me_method:v:0 dia \ > > -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \ > > -flags +global_header \ > > -f segment -segment_time $segment_time \ > > -segment_format mpegts \ > > -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts" > > > > https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54 > > > > and in case more info helps: > > > > $ mediainfo 13_25_49.ts > > General > > ID : 1 (0x1) > > Complete name : 13_25_49.ts > > Format : MPEG-TS > > Commercial name : HDV 720p > > File size : 838 MiB > > Duration : 30 min 0 s > > Overall bit rate mode : Variable > > Overall bit rate : 3 903 kb/s > > Frame rate : 30.000 FPS > > > > Video > > ID : 256 (0x100) > > Menu ID : 1 (0x1) > > Format : MPEG Video > > Commercial name : HDV 720p > > Format version : Version 2 > > Format profile : Main@High 1440 > > Format settings, BVOP : No > > Format settings, Matrix : Default > > Format settings, GOP : N=5 > > Codec ID : 2 > > Duration : 30 min 0 s > > Bit rate mode : Variable > > Bit rate : 3 323 kb/s > > Width : 1 280 pixels > > Height : 720 pixels > > Display aspect ratio : 16:9 > > Frame rate : 30.000 FPS > > Color space : YUV > > Chroma subsampling : 4:2:0 > > Bit depth : 8 bits > > Scan type : Progressive > > Compression mode : Lossy > > Bits/(Pixel*Frame) : 0.120 > > Time code of first frame : 01:29:59:29 > > Time code source : Group of pictures header > > GOP, Open/Closed : Open > > Stream size : 713 MiB (85%) > > Color primaries : BT.709 > > Transfer characteristics : BT.709 > > Matrix coefficients : BT.709 > > > > Audio > > ID : 257 (0x101) > > Menu ID : 1 (0x1) > > Format : MPEG Audio > > Format version : Version 1 > > Format profile : Layer 2 > > Codec ID : 3 > > Duration : 30 min 0 s > > Bit rate mode : Constant > > Bit rate : 384 kb/s > > Channel(s) : 2 channels > > Sampling rate : 48.0 kHz > > Frame rate : 41.667 FPS (1152 SPF) > > Compression mode : Lossy > > Delay relative to video : -29 ms > > Stream size : 82.4 MiB (10%) > > Language : English > > > > Menu > > ID : 4096 (0x1000) > > Menu ID : 1 (0x1) > > Format : MPEG Video / MPEG Audio > > Duration : 30 min 0 s > > List : 256 (0x100) (MPEG Video) / > > 257 (0x101) (MPEG Audio, English) > > Language : / English > > Service name : Service01 > > Service provider : FFmpeg > > Service type : digital television > > > > > > On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <[email protected]> wrote: > >> As suggested you can go some different ways: > >> > >> > >> - Make a python gst app: > >> > >> filesrc ! decodebin ! appsink > >> > >> opencv code > >> > >> appsrc ! encode ! filesink > >> > >> Passing the audio as well might be tricky... > >> > >> > >> - Make a python gst app: > >> > >> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1 > >> ! filesink demux1. ! mux1 > >> > >> and use a probe to snatch frames at the identify element and do what you > >> want there with opencv > >> > >> > >> - Make a filter element to do your stuff > >> > >> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 ! > >> filesink demux1. ! mux1 > >> > >> > >> Best Regards > >> > >> Marianna > >> > >> > >> On 13/10/2023 14.00, [email protected] wrote: > >>> I need to alter video frames, and Python is my happy place. > >>> my gst skills need some help. > >>> > >>> I think I need these 2 things: > >>> > >>> 1. pipeline: > >>> read a file, > >>> decode the video, > >>> hook? to read/write pixels, > >>> encode (keep the original audio) > >>> save to disk. > >>> > >>> 2. that hook code... > >>> I have done something before: use the wavelevel element to get audio > >>> data and generate static png images. > >>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72 > >>> > >>> So I am assuming some element / message will give me a frame buffer > >>> thing of data, and somehow i can detect black pixels. > >>> > >>> > >>> End goal: fix some video;s I made of a live event where my setup > >>> introduced black squares - I want to replace the black with the pixels > >>> from the previous frame. > >>> example of what needs fixing: > >>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148 > >>> > >>> The slide is mostly a static image, so the previous frame will be fine. > >>> > >>> -- Carl K > >> -- > >> Best regards / Med venlig hilsen > >> “Marianna Smidth Buschle” > >> > > > -- > Best regards / Med venlig hilsen > “Marianna Smidth Buschle” > -- Carl K