Re: GStreamer playback of MP4 files with and without audio
Terry Barnaby via gstreamer-devel <[email protected]> Wed, 2 Oct 2024 15:52:48 +0100
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --------------Za2EbwuF3NOTx0kp0p0jrouX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Thanks, but in my real case I have to do a fair bit with the separate video and audio streams (multiple elements with tees, AppSinks etc). On 02/10/2024 12:36, Tim Müller via gstreamer-devel wrote: > Hi Terry, > >> 1. Maybe there is some gstreamer element or attribute that can ignore >> the audio stream if not present somehow ? > > Have you tried the playbin3 > <https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page> element > yet? > > You just set the file location in form of a URI (gst_filename_to_uri() > <https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri>) > on it via the "uri" property, and it will figure out the > demuxer/parsers/decoders itself. > > If there's video it will plug video decoders and a video sink, > otherwise not. > > If there's audio it will plug audio decoders and an audio sink, > otherwise not. > > You can configure specific sinks via the audio-sink and video-sink > properties as well. > > You can see how it behaves with the gst-play-1.0 command line utility > for what it's worth, which is a wrapper around playbin/playbin3. > > Cheers > Tim > > PS: most discussion has shifted from the mailing list to our shiny new > Discourse forum <https://discourse.gstreamer.org/> these days. > > On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel > wrote: >> >> I am developing a C++ video inspection program that creates >> MP4/H264/MP3 files and needs to play them back with the video stream >> processed by various gstreamer elements and audio separately. >> >> In general this has been working fine when just video was being >> recorded/played back, but I am just adding audio to the mix. Some MP4 >> files will contain only a video stream and some will contain both >> video and audio streams. >> >> I need some way to handle the playback of these MP4 files that may or >> may not have MP3 audio streams. >> >> As a simple idea if I use something like (The real C++ code >> constructs the gstreamer piple line and tees the video stream to >> various gstreamer sub pipelines): >> >> gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux >> demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink >> demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink >> >> This plays back the video and audio streams fine (not sure how well >> synchronised?) from an MP4 with video and audio streams, but hangs if >> the MP4 only has a video stream. >> >> So I think I need to: >> >> 1. Maybe there is some gstreamer element or attribute that can ignore >> the audio stream if not present somehow ? >> >> 2. Check if the MP4 file has an audio stream before creating the >> gstreamer pipeline in C++. >> >> 3. Create the basic C++ pipleline in C++ and interrogate pads or >> something somehow and add the "demux.audio_0 ! queue ! decodebin ! >> audioconvert ! pulsesink" sub pipeline if it is seen an audio stream >> is present. >> >> Any ideas on the simplest/most CPU efficient way of doing this with >> gstreamer ? >> >> > -- Dr Terry Barnaby BEAM Ltd Phone: +44 1454 324512 Northavon Business Center, Email:[email protected] Dean Rd, Yate Web:www.beam.ltd.uk Bristol, BS37 5NH, UK BEAM Engineering: Instrumentation, Electronics/Software/Systems --------------Za2EbwuF3NOTx0kp0p0jrouX Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <div class="moz-cite-prefix">Thanks, but in my real case I have to do a fair bit with the separate video and audio streams (multiple elements with tees, AppSinks etc).</div> <div class="moz-cite-prefix"><br> </div> <div class="moz-cite-prefix">On 02/10/2024 12:36, Tim Müller via gstreamer-devel wrote:<br> </div> <blockquote type="cite" cite="mid:[email protected]"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <style>pre,code,address { margin: 0px; }h1,h2,h3,h4,h5,h6 { margin-top: 0.2em; margin-bottom: 0.2em; }ol,ul { margin-top: 0em; margin-bottom: 0em; }blockquote { margin-top: 0em; margin-bottom: 0em; }</style> <div>Hi Terry,</div> <div><br> </div> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <div>1. Maybe there is some gstreamer element or attribute that can ignore the audio stream if not present somehow ?</div> </blockquote> <div><br> </div> <div>Have you tried the <a href="https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page" moz-do-not-send="true">playbin3</a> element yet?</div> <div><br> </div> <div>You just set the file location in form of a URI (<a href="https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri" moz-do-not-send="true">gst_filename_to_uri()</a>) on it via the "uri" property, and it will figure out the demuxer/parsers/decoders itself.</div> <div><br> </div> <div>If there's video it will plug video decoders and a video sink, otherwise not.</div> <div><br> </div> <div>If there's audio it will plug audio decoders and an audio sink, otherwise not.</div> <div><br> </div> <div>You can configure specific sinks via the audio-sink and video-sink properties as well.</div> <div><br> </div> <div>You can see how it behaves with the gst-play-1.0 command line utility for what it's worth, which is a wrapper around playbin/playbin3.</div> <div><br> </div> <div>Cheers</div> <div> Tim</div> <div><br> </div> <div>PS: most discussion has shifted from the mailing list to our shiny new <a href="https://discourse.gstreamer.org/" moz-do-not-send="true">Discourse forum</a> these days.</div> <div><br> </div> <div>On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel wrote:</div> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <div> </div> <p>I am developing a C++ video inspection program that creates MP4/H264/MP3 files and needs to play them back with the video stream processed by various gstreamer elements and audio separately.</p> <div> </div> <p>In general this has been working fine when just video was being recorded/played back, but I am just adding audio to the mix. Some MP4 files will contain only a video stream and some will contain both video and audio streams.</p> <div> </div> <p>I need some way to handle the playback of these MP4 files that may or may not have MP3 audio streams.<br> </p> <div> </div> <p>As a simple idea if I use something like (The real C++ code constructs the gstreamer piple line and tees the video stream to various gstreamer sub pipelines):<br> </p> <div> </div> <pre>gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux</pre> <div> </div> <pre> demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink</pre> <div> </div> <pre> demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink</pre> <div> </div> <p>This plays back the video and audio streams fine (not sure how well synchronised?) from an MP4 with video and audio streams, but hangs if the MP4 only has a video stream.</p> <div> </div> <p>So I think I need to:</p> <div> </div> <p>1. Maybe there is some gstreamer element or attribute that can ignore the audio stream if not present somehow ?</p> <div> </div> <p>2. Check if the MP4 file has an audio stream before creating the gstreamer pipeline in C++.</p> <div> </div> <p>3. Create the basic C++ pipleline in C++ and interrogate pads or something somehow and add the "demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink" sub pipeline if it is seen an audio stream is present.<br> </p> <div> Any ideas on the simplest/most CPU efficient way of doing this with gstreamer ?<br> </div> <p><br> </p> <div> </div> </blockquote> <div><br> </div> <div><span></span></div> </blockquote> <p><br> </p> <pre class="moz-signature" cols="72">-- Dr Terry Barnaby BEAM Ltd Phone: +44 1454 324512 Northavon Business Center, Email: <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> Dean Rd, Yate Web: <a class="moz-txt-link-abbreviated" href="http://www.beam.ltd.uk">www.beam.ltd.uk</a> Bristol, BS37 5NH, UK BEAM Engineering: Instrumentation, Electronics/Software/Systems</pre> </body> </html> --------------Za2EbwuF3NOTx0kp0p0jrouX--