Re: Python access to video frames (Carl Karsten)

Again, you need to start simplifying the pipeline in order to easier 
figure out what is going on.

Either you start with just a src and a sink and grow from there as I 
showed or you start with the whole pipeline and remove elements/branches 
one by one...

A good start is just to divide the src and decode from the encode and 
save. Or just start by discarding the audio...

Also, add --gst-debug=*:3 to the pipeline to get some debug output.

gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts --gst-debug=*:3

gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! fakesink --gst-debug=*:3

gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
queue ! decodebin3 ! videoconvert ! autovideosink sync=0 --gst-debug=*:3

And if you start suspecting some of the elements you can do: 
--gst-debug=*:3,elem:5 to get even more debug

On 17/10/2023 19.32, Carl Karsten wrote:
> no error, but no play.
>
> gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
> '!' queue '!' decodebin3 '!' avenc_mpeg2video '!' mpegtsmux name=mux1
> '!' filesink location=fixed_13_25_49.ts demux1. '!' queue '!'
> mpegaudioparse '!' mux1.
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
>
> htop shows no cpu and ls -l shows no bytes.
>
> On Tue, Oct 17, 2023 at 4:19 AM Marianna Smidth Buschle <[email protected]> wrote:
>> The demuxer divides the pipeline into audio and video, you get one in
>> each branch.
>>
>> Right now you seem to have both an audioparser and a videoencoder in the
>> same branch.
>>
>> Try instead:
>>
>> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
>> queue ! decodebin3 ! avenc_mpeg2video ! mpegtsmux name=mux1 ! filesink location=fixed_13_25_49.ts
>> demux1. ! queue ! mpegaudioparse ! mux1.
>>
>>
>> And generally, when in doubt about pipeline errors it is always useful
>> to simplify it and then build up.
>> Fx: filesrc ! fakesink
>> then filesrc ! demux ! fakesink
>> then filesrc ! demux ! decode ! fakesink
>> then filesrc ! demux ! decode ! videosink
>> ...
>>
>> Best Regards
>> Marianna
>>
>> On 16/10/2023 20.00, Carl Karsten wrote:
>>> Ima need a little help with the pipeline...
>>>
>>> # probably needs an aacparse or parser for whatever the codec of the
>>> audio stream is
>>> # also you need queues after both branches of the demuxer
>>>
>>> gst-launch-1.0 filesrc location=13_25_49.ts ! tsdemux name=demux1 !
>>> queue ! mpegaudioparse ! decodebin3 ! avenc_mpeg2video ! mpegtsmux
>>> name=mux1 ! filesink location=fixed_13_25_49.ts demux1. ! queue !
>>> mux1.
>>>
>>>
>>> ++ gst-launch-1.0 filesrc location=13_25_49.ts '!' tsdemux name=demux1
>>> '!' queue '!' mpegaudioparse '!' decodebin3 '!' avenc_mpeg2video '!'
>>> mpegtsmux name=mux1 '!' filesink location=fixed_13_25_49.ts demux1.
>>> '!' queue '!' mux1.
>>> Setting pipeline to PAUSED ...
>>> Pipeline is PREROLLING ...
>>> ERROR: from element /GstPipeline:pipeline0/GstTSDemux:demux1: Internal
>>> data stream error.
>>> Additional debug info:
>>> ../gst/mpegtsdemux/mpegtsbase.c(1760): mpegts_base_loop ():
>>> /GstPipeline:pipeline0/GstTSDemux:demux1:
>>> streaming stopped, reason not-negotiated (-4)
>>> ERROR: pipeline doesn't want to preroll.
>>> Setting pipeline to NULL ...
>>> ERROR: from element
>>> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
>>> Stream doesn't contain enough data.
>>> Additional debug info:
>>> ../plugins/elements/gsttypefindelement.c(1012):
>>> gst_type_find_element_chain_do_typefinding ():
>>> /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstTypeFindElement:typefind:
>>> Can't typefind stream
>>> ERROR: pipeline doesn't want to preroll.
>>> Freeing pipeline ...
>>>
>>> 13_25_49.ts came from:
>>>
>>> ffmpeg \
>>>       -nostdin -y \
>>>       -analyzeduration 10000 \
>>>       -thread_queue_size 512 \
>>>       -i tcp://localhost:11000?timeout=3000000 \
>>>       -aspect 16:9 \
>>>       -map 0:v -c:v:0 mpeg2video -pix_fmt:v:0 yuv420p -qscale:v:0 4
>>> -qmin:v:0 4 -qmax:v:0 4 -keyint_min:v:0 5 -bf:v:0 0 -g:v:0 5
>>> -me_method:v:0 dia \
>>>       -map 0:a -c:a mp2 -b:a 384k -ac:a 2 -ar:a 48000 \
>>>       -flags +global_header \
>>>           -f segment -segment_time $segment_time \
>>>            -segment_format mpegts \
>>>            -strftime 1 "$dest_dir/%Y-%m-%d/%H_%M_%S.ts"
>>>
>>> https://github.com/CarlFK/voctomix-outcasts/blob/master/record-timestamp.sh#L30C1-L41C54
>>>
>>> and in case more info helps:
>>>
>>> $ mediainfo 13_25_49.ts
>>> General
>>> ID                                       : 1 (0x1)
>>> Complete name                            : 13_25_49.ts
>>> Format                                   : MPEG-TS
>>> Commercial name                          : HDV 720p
>>> File size                                : 838 MiB
>>> Duration                                 : 30 min 0 s
>>> Overall bit rate mode                    : Variable
>>> Overall bit rate                         : 3 903 kb/s
>>> Frame rate                               : 30.000 FPS
>>>
>>> Video
>>> ID                                       : 256 (0x100)
>>> Menu ID                                  : 1 (0x1)
>>> Format                                   : MPEG Video
>>> Commercial name                          : HDV 720p
>>> Format version                           : Version 2
>>> Format profile                           : Main@High 1440
>>> Format settings, BVOP                    : No
>>> Format settings, Matrix                  : Default
>>> Format settings, GOP                     : N=5
>>> Codec ID                                 : 2
>>> Duration                                 : 30 min 0 s
>>> Bit rate mode                            : Variable
>>> Bit rate                                 : 3 323 kb/s
>>> Width                                    : 1 280 pixels
>>> Height                                   : 720 pixels
>>> Display aspect ratio                     : 16:9
>>> Frame rate                               : 30.000 FPS
>>> Color space                              : YUV
>>> Chroma subsampling                       : 4:2:0
>>> Bit depth                                : 8 bits
>>> Scan type                                : Progressive
>>> Compression mode                         : Lossy
>>> Bits/(Pixel*Frame)                       : 0.120
>>> Time code of first frame                 : 01:29:59:29
>>> Time code source                         : Group of pictures header
>>> GOP, Open/Closed                         : Open
>>> Stream size                              : 713 MiB (85%)
>>> Color primaries                          : BT.709
>>> Transfer characteristics                 : BT.709
>>> Matrix coefficients                      : BT.709
>>>
>>> Audio
>>> ID                                       : 257 (0x101)
>>> Menu ID                                  : 1 (0x1)
>>> Format                                   : MPEG Audio
>>> Format version                           : Version 1
>>> Format profile                           : Layer 2
>>> Codec ID                                 : 3
>>> Duration                                 : 30 min 0 s
>>> Bit rate mode                            : Constant
>>> Bit rate                                 : 384 kb/s
>>> Channel(s)                               : 2 channels
>>> Sampling rate                            : 48.0 kHz
>>> Frame rate                               : 41.667 FPS (1152 SPF)
>>> Compression mode                         : Lossy
>>> Delay relative to video                  : -29 ms
>>> Stream size                              : 82.4 MiB (10%)
>>> Language                                 : English
>>>
>>> Menu
>>> ID                                       : 4096 (0x1000)
>>> Menu ID                                  : 1 (0x1)
>>> Format                                   : MPEG Video / MPEG Audio
>>> Duration                                 : 30 min 0 s
>>> List                                     : 256 (0x100) (MPEG Video) /
>>> 257 (0x101) (MPEG Audio, English)
>>> Language                                 :  / English
>>> Service name                             : Service01
>>> Service provider                         : FFmpeg
>>> Service type                             : digital television
>>>
>>>
>>> On Mon, Oct 16, 2023 at 2:05 AM Marianna Smidth Buschle <[email protected]> wrote:
>>>> As suggested you can go some different ways:
>>>>
>>>>
>>>> - Make a python gst app:
>>>>
>>>> filesrc ! decodebin ! appsink
>>>>
>>>> opencv code
>>>>
>>>> appsrc ! encode ! filesink
>>>>
>>>> Passing the audio as well might be tricky...
>>>>
>>>>
>>>> - Make a python gst app:
>>>>
>>>> filesrc ! demux name=demux1 ! decode ! identity ! encode ! mux name=mux1
>>>> ! filesink demux1. ! mux1
>>>>
>>>> and use a probe to snatch frames at the identify element and do what you
>>>> want there with opencv
>>>>
>>>>
>>>> - Make a filter element to do your stuff
>>>>
>>>> filesrc ! demux name=demux1 ! decode ! filter ! encode ! mux name=mux1 !
>>>> filesink demux1. ! mux1
>>>>
>>>>
>>>> Best Regards
>>>>
>>>> Marianna
>>>>
>>>>
>>>> On 13/10/2023 14.00, [email protected] wrote:
>>>>> I need to alter video frames, and Python is my happy place.
>>>>> my gst skills need some help.
>>>>>
>>>>> I think I need these 2 things:
>>>>>
>>>>> 1. pipeline:
>>>>> read a file,
>>>>> decode the video,
>>>>> hook? to read/write pixels,
>>>>> encode (keep the original audio)
>>>>> save to disk.
>>>>>
>>>>> 2. that hook code...
>>>>> I have done something before: use the wavelevel element to get audio
>>>>> data and generate static png images.
>>>>> https://github.com/CarlFK/veyepar/blob/master/dj/lib/gslevels.py#L72
>>>>>
>>>>> So I am assuming some element / message will give me a frame buffer
>>>>> thing of data, and somehow i can detect black pixels.
>>>>>
>>>>>
>>>>> End goal: fix some video;s I made of a live event where my setup
>>>>> introduced black squares - I want to replace the black with the pixels
>>>>> from the previous frame.
>>>>> example of what needs fixing:
>>>>> https://youtu.be/N5Anbq8vYNk?si=Gl89X3Uk5_2CrMu6&t=148
>>>>>
>>>>> The slide is mostly a static image, so the previous frame will be fine.
>>>>>
>>>>> -- Carl K
>>>> --
>>>> Best regards / Med venlig hilsen
>>>> “Marianna Smidth Buschle”
>>>>
>> --
>> Best regards / Med venlig hilsen
>> “Marianna Smidth Buschle”
>>
>
-- 
Best regards / Med venlig hilsen
“Marianna Smidth Buschle”