Re: GStreamer playback of MP4 files with and without audio
Terry Barnaby via gstreamer-devel <[email protected]> Wed, 2 Oct 2024 16:07:12 +0100
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--------------xdCPY03Rp0mh0YQ66Hm25mVk
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Thanks, but in my real case I have to do a fair bit with the separate
video and audio streams (multiple elements with tees, AppSinks etc).
On 02/10/2024 12:36, Tim Müller via gstreamer-devel wrote:
> Hi Terry,
>
>> 1. Maybe there is some gstreamer element or attribute that can ignore
>> the audio stream if not present somehow ?
>
> Have you tried the playbin3
> <https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page> element
> yet?
>
> You just set the file location in form of a URI (gst_filename_to_uri()
> <https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri>)
> on it via the "uri" property, and it will figure out the
> demuxer/parsers/decoders itself.
>
> If there's video it will plug video decoders and a video sink,
> otherwise not.
>
> If there's audio it will plug audio decoders and an audio sink,
> otherwise not.
>
> You can configure specific sinks via the audio-sink and video-sink
> properties as well.
>
> You can see how it behaves with the gst-play-1.0 command line utility
> for what it's worth, which is a wrapper around playbin/playbin3.
>
> Cheers
> Tim
>
> PS: most discussion has shifted from the mailing list to our shiny new
> Discourse forum <https://discourse.gstreamer.org/> these days.
>
> On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel
> wrote:
>>
>> I am developing a C++ video inspection program that creates
>> MP4/H264/MP3 files and needs to play them back with the video stream
>> processed by various gstreamer elements and audio separately.
>>
>> In general this has been working fine when just video was being
>> recorded/played back, but I am just adding audio to the mix. Some MP4
>> files will contain only a video stream and some will contain both
>> video and audio streams.
>>
>> I need some way to handle the playback of these MP4 files that may or
>> may not have MP3 audio streams.
>>
>> As a simple idea if I use something like (The real C++ code
>> constructs the gstreamer piple line and tees the video stream to
>> various gstreamer sub pipelines):
>>
>> gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux
>> demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
>> demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
>>
>> This plays back the video and audio streams fine (not sure how well
>> synchronised?) from an MP4 with video and audio streams, but hangs if
>> the MP4 only has a video stream.
>>
>> So I think I need to:
>>
>> 1. Maybe there is some gstreamer element or attribute that can ignore
>> the audio stream if not present somehow ?
>>
>> 2. Check if the MP4 file has an audio stream before creating the
>> gstreamer pipeline in C++.
>>
>> 3. Create the basic C++ pipleline in C++ and interrogate pads or
>> something somehow and add the "demux.audio_0 ! queue ! decodebin !
>> audioconvert ! pulsesink" sub pipeline if it is seen an audio stream
>> is present.
>>
>> Any ideas on the simplest/most CPU efficient way of doing this with
>> gstreamer ?
>>
>>
>
--------------xdCPY03Rp0mh0YQ66Hm25mVk
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Thanks, but in my real case I have to
do a fair bit with the separate video and audio streams (multiple
elements with tees, AppSinks etc).</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 02/10/2024 12:36, Tim Müller via
gstreamer-devel wrote:<br>
</div>
<blockquote type="cite"
cite="mid:[email protected]">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<style>pre,code,address {
margin: 0px;
}h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}blockquote {
margin-top: 0em;
margin-bottom: 0em;
}</style>
<div>Hi Terry,</div>
<div><br>
</div>
<blockquote type="cite"
style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>1. Maybe there is some gstreamer element or attribute that
can ignore the audio stream if not present somehow ?</div>
</blockquote>
<div><br>
</div>
<div>Have you tried the <a
href="https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page"
moz-do-not-send="true">playbin3</a> element yet?</div>
<div><br>
</div>
<div>You just set the file location in form of a URI (<a
href="https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri"
moz-do-not-send="true">gst_filename_to_uri()</a>) on it via
the "uri" property, and it will figure out the
demuxer/parsers/decoders itself.</div>
<div><br>
</div>
<div>If there's video it will plug video decoders and a video
sink, otherwise not.</div>
<div><br>
</div>
<div>If there's audio it will plug audio decoders and an audio
sink, otherwise not.</div>
<div><br>
</div>
<div>You can configure specific sinks via the audio-sink and
video-sink properties as well.</div>
<div><br>
</div>
<div>You can see how it behaves with the gst-play-1.0 command line
utility for what it's worth, which is a wrapper around
playbin/playbin3.</div>
<div><br>
</div>
<div>Cheers</div>
<div> Tim</div>
<div><br>
</div>
<div>PS: most discussion has shifted from the mailing list to our
shiny new <a href="https://discourse.gstreamer.org/"
moz-do-not-send="true">Discourse forum</a> these days.</div>
<div><br>
</div>
<div>On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via
gstreamer-devel wrote:</div>
<blockquote type="cite"
style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div> </div>
<p>I am developing a C++ video inspection program that creates
MP4/H264/MP3 files and needs to play them back with the video
stream processed by various gstreamer elements and audio
separately.</p>
<div> </div>
<p>In general this has been working fine when just video was
being recorded/played back, but I am just adding audio to the
mix. Some MP4 files will contain only a video stream and some
will contain both video and audio streams.</p>
<div> </div>
<p>I need some way to handle the playback of these MP4 files
that may or may not have MP3 audio streams.<br>
</p>
<div> </div>
<p>As a simple idea if I use something like (The real C++ code
constructs the gstreamer piple line and tees the video stream
to various gstreamer sub pipelines):<br>
</p>
<div> </div>
<pre>gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux</pre>
<div> </div>
<pre> demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink</pre>
<div> </div>
<pre> demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink</pre>
<div> </div>
<p>This plays back the video and audio streams fine (not sure
how well synchronised?) from an MP4 with video and audio
streams, but hangs if the MP4 only has a video stream.</p>
<div> </div>
<p>So I think I need to:</p>
<div> </div>
<p>1. Maybe there is some gstreamer element or attribute that
can ignore the audio stream if not present somehow ?</p>
<div> </div>
<p>2. Check if the MP4 file has an audio stream before creating
the gstreamer pipeline in C++.</p>
<div> </div>
<p>3. Create the basic C++ pipleline in C++ and interrogate pads
or something somehow and add the "demux.audio_0 ! queue !
decodebin ! audioconvert ! pulsesink" sub pipeline if it is
seen an audio stream is present.<br>
</p>
<div> Any ideas on the simplest/most CPU efficient way of doing
this with gstreamer ?<br>
</div>
<p><br>
</p>
<div> </div>
</blockquote>
<div><br>
</div>
<div><span></span></div>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------xdCPY03Rp0mh0YQ66Hm25mVk--