Re: GStreamer playback of MP4 files with and without audio
Terry Barnaby via gstreamer-devel <[email protected]> Wed, 2 Oct 2024 15:51:14 +0100
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --------------6L994H50R0ukrz1enJWeJHsU Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Thanks, I will have a play with adding to the pipeline on that event. Terry On 02/10/2024 08:26, Gareth Alldread via gstreamer-devel wrote: > I would add a handler for the "pad-added" event on the filesrc element > and only create/add the audio part of the pipeline if you get an audio > pad created. That is what I have done to handle rtspsrc streams that > may or may not have audio. Reply if you want more details. > > g_signal_connect(m_sourceElement, "pad-added", > G_CALLBACK(&SynxRTSPPipeline::onPadAdded), this); > > On Wed, 2 Oct 2024 at 07:30, Terry Barnaby via gstreamer-devel > <[email protected]> wrote: > > I am developing a C++ video inspection program that creates > MP4/H264/MP3 files and needs to play them back with the video > stream processed by various gstreamer elements and audio separately. > > In general this has been working fine when just video was being > recorded/played back, but I am just adding audio to the mix. Some > MP4 files will contain only a video stream and some will contain > both video and audio streams. > > I need some way to handle the playback of these MP4 files that may > or may not have MP3 audio streams. > > As a simple idea if I use something like (The real C++ code > constructs the gstreamer piple line and tees the video stream to > various gstreamer sub pipelines): > > gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux > > demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink > > demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink > > This plays back the video and audio streams fine (not sure how > well synchronised?) from an MP4 with video and audio streams, but > hangs if the MP4 only has a video stream. > > So I think I need to: > > 1. Maybe there is some gstreamer element or attribute that can > ignore the audio stream if not present somehow ? > > 2. Check if the MP4 file has an audio stream before creating the > gstreamer pipeline in C++. > > 3. Create the basic C++ pipleline in C++ and interrogate pads or > something somehow and add the "demux.audio_0 ! queue ! decodebin ! > audioconvert ! pulsesink" sub pipeline if it is seen an audio > stream is present. > > Any ideas on the simplest/most CPU efficient way of doing this > with gstreamer ? > > --------------6L994H50R0ukrz1enJWeJHsU Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <div class="moz-cite-prefix">Thanks, I will have a play with adding to the pipeline on that event.</div> <div class="moz-cite-prefix"><br> </div> <div class="moz-cite-prefix">Terry<br> </div> <div class="moz-cite-prefix">On 02/10/2024 08:26, Gareth Alldread via gstreamer-devel wrote:<br> </div> <blockquote type="cite" cite="mid:CAENg58qyeVDbzdJwNwdQ2pcxxajL2QN=U9KabvDqBV17R6ZhfQ@mail.gmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="ltr"> <div>I would add a handler for the "pad-added" event on the filesrc element and only create/add the audio part of the pipeline if you get an audio pad created. That is what I have done to handle rtspsrc streams that may or may not have audio. Reply if you want more details.</div> <div><br> </div> <div> g_signal_connect(m_sourceElement, "pad-added", G_CALLBACK(&SynxRTSPPipeline::onPadAdded), this);<br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Wed, 2 Oct 2024 at 07:30, Terry Barnaby via gstreamer-devel <<a href="mailto:[email protected]" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <p>I am developing a C++ video inspection program that creates MP4/H264/MP3 files and needs to play them back with the video stream processed by various gstreamer elements and audio separately.</p> <p>In general this has been working fine when just video was being recorded/played back, but I am just adding audio to the mix. Some MP4 files will contain only a video stream and some will contain both video and audio streams.</p> <p>I need some way to handle the playback of these MP4 files that may or may not have MP3 audio streams.<br> </p> <p>As a simple idea if I use something like (The real C++ code constructs the gstreamer piple line and tees the video stream to various gstreamer sub pipelines):<br> </p> <pre>gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux</pre> <pre> demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink</pre> <pre> demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink</pre> <p>This plays back the video and audio streams fine (not sure how well synchronised?) from an MP4 with video and audio streams, but hangs if the MP4 only has a video stream.</p> <p>So I think I need to:</p> <p>1. Maybe there is some gstreamer element or attribute that can ignore the audio stream if not present somehow ?</p> <p>2. Check if the MP4 file has an audio stream before creating the gstreamer pipeline in C++.</p> <p>3. Create the basic C++ pipleline in C++ and interrogate pads or something somehow and add the "demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink" sub pipeline if it is seen an audio stream is present.<br> </p> Any ideas on the simplest/most CPU efficient way of doing this with gstreamer ?<br> <p><br> </p> </div> </blockquote> </div> </blockquote> <p><br> </p> </body> </html> --------------6L994H50R0ukrz1enJWeJHsU--