Re: GStreamer playback of MP4 files with and without audio
Tim Müller via gstreamer-devel <[email protected]> Wed, 02 Oct 2024 12:36:13 +0100
| Newsgroups | gmane.comp.video.gstreamer.devel |
|---|---|
| Organization | Centricular Ltd |
| Message-ID | <[email protected]> |
--=-s7A00AfaDyiaYpgABDO1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Hi Terry,
> 1. Maybe there is some gstreamer element or attribute that can ignore
> the audio stream if not present somehow ?
Have you tried the=C2=A0playbin3 [1]=C2=A0element yet?
You just set the file location in form of a URI (gst_filename_to_uri()
[2]) on it via the "uri" property, and it will figure out the
demuxer/parsers/decoders itself.
If there's video it will plug video decoders and a video sink,
otherwise not.
If there's audio it will plug audio decoders and an audio sink,
otherwise not.
You can configure specific sinks via the audio-sink and video-sink
properties as well.
You can see how it behaves with the gst-play-1.0 command line utility
for what it's worth, which is a wrapper around playbin/playbin3.
Cheers
=C2=A0Tim
PS: most discussion has shifted from the mailing list to our shiny
new=C2=A0Discourse forum [3]=C2=A0these days.
On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel
wrote:
> =20
> I am developing a C++ video inspection program that creates
> MP4/H264/MP3 files and needs to play them back with the video stream
> processed by various gstreamer elements and audio separately.
> =20
> In general this has been working fine when just video was being
> recorded/played back, but I am just adding audio to the mix. Some MP4
> files will contain only a video stream and some will contain both
> video and audio streams.
> =20
> I need some way to handle the playback of these MP4 files that may or
> may not have MP3 audio streams.
> =20
> As a simple idea if I use something like (The real C++ code
> constructs the gstreamer piple line and tees the video stream to
> various gstreamer sub pipelines):
> =20
> gst-launch-1.0 -v filesrc location=3Dtemp.mp4 ! qtdemux name=3Ddemux
> =20
> =C2=A0demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
> =20
> =C2=A0demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
> =20
> This plays back the video and audio streams fine (not sure how well
> synchronised?) from an MP4 with video and audio streams, but hangs if
> the MP4 only has a video stream.
> =20
> So I think I need to:
> =20
> 1. Maybe there is some gstreamer element or attribute that can ignore
> the audio stream if not present somehow ?
> =20
> 2. Check if the MP4 file has an audio stream before creating the
> gstreamer pipeline in C++.
> =20
> 3. Create the basic C++ pipleline in C++ and interrogate pads or
> something somehow and add the "demux.audio_0 ! queue ! decodebin !
> audioconvert ! pulsesink" sub pipeline if it is seen an audio stream
> is present.
> Any ideas on the simplest/most CPU efficient way of doing this with
> gstreamer ?
>=20
> =20
> =20
[1] playbin3
https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?=
gi-language=3Dc#playbin3-page
[2] gst_filename_to_uri()
https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler=
.html?gi-language=3Dc#gst_filename_to_uri
[3] Discourse forum https://discourse.gstreamer.org/
--=-s7A00AfaDyiaYpgABDO1
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
<html><head>
=20
<style>pre,code,address {
margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}
ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}
blockquote {
margin-top: 0em;
margin-bottom: 0em;
}
</style></head>
<body><div>Hi Terry,</div><div><br></div><blockquote type=3D"cite" style=
=3D"margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div=
>1. Maybe there is some gstreamer element or attribute that can ignore the =
audio stream if not present somehow ?</div></blockquote><div><br></div><div=
>Have you tried the <a href=3D"https://gstreamer.freedesktop.org/docum=
entation/playback/playbin3.html?gi-language=3Dc#playbin3-page">playbin3</a>=
element yet?</div><div><br></div><div>You just set the file location =
in form of a URI (<a href=3D"https://gstreamer.freedesktop.org/documentatio=
n/gstreamer/gsturihandler.html?gi-language=3Dc#gst_filename_to_uri">gst_fil=
ename_to_uri()</a>) on it via the "uri" property, and it will figure out th=
e demuxer/parsers/decoders itself.</div><div><br></div><div>If there's vide=
o it will plug video decoders and a video sink, otherwise not.</div><div><b=
r></div><div>If there's audio it will plug audio decoders and an audio sink=
, otherwise not.</div><div><br></div><div>You can configure specific sinks =
via the audio-sink and video-sink properties as well.</div><div><br></div><=
div>You can see how it behaves with the gst-play-1.0 command line utility f=
or what it's worth, which is a wrapper around playbin/playbin3.</div><div><=
br></div><div>Cheers</div><div> Tim</div><div><br></div><div>PS: most =
discussion has shifted from the mailing list to our shiny new <a href=
=3D"https://discourse.gstreamer.org/">Discourse forum</a> these days.<=
/div><div><br></div><div>On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby v=
ia gstreamer-devel wrote:</div><blockquote type=3D"cite" style=3D"margin:0 =
0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div> </div><p>I =
am developing a C++ video inspection program that creates MP4/H264/MP3 file=
s and needs to play them back with the video stream processed by various gs=
treamer elements and audio separately.</p><div> </div><p>In general this ha=
s been working fine when just video was being recorded/played back, but I a=
m just adding audio to the mix. Some MP4 files will contain only a video st=
ream and some will contain both video and audio streams.</p><div> </div><p>=
I need some way to handle the playback of these MP4 files that may or may n=
ot have MP3 audio streams.<br> </p><div> </div><p>As a simple idea if I use=
something like (The real C++ code constructs the gstreamer piple line and =
tees the video stream to various gstreamer sub pipelines):<br> </p><div> </=
div><pre>gst-launch-1.0 -v filesrc location=3Dtemp.mp4 ! qtdemux name=3Ddem=
ux</pre><div> </div><pre> demux.video_0 ! queue ! h264parse ! openh264=
dec ! glimagesink</pre><div> </div><pre> demux.audio_0 ! queue ! decod=
ebin ! audioconvert ! pulsesink</pre><div> </div><p>This plays back the vid=
eo and audio streams fine (not sure how well synchronised?) from an MP4 wit=
h video and audio streams, but hangs if the MP4 only has a video stream.</p=
><div> </div><p>So I think I need to:</p><div> </div><p>1. Maybe there is s=
ome gstreamer element or attribute that can ignore the audio stream if not =
present somehow ?</p><div> </div><p>2. Check if the MP4 file has an audio s=
tream before creating the gstreamer pipeline in C++.</p><div> </div><p>3. C=
reate the basic C++ pipleline in C++ and interrogate pads or something some=
how and add the "demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesi=
nk" sub pipeline if it is seen an audio stream is present.<br> </p><div> An=
y ideas on the simplest/most CPU efficient way of doing this with gstreamer=
?<br> </div><p><br> </p><div> </div></blockquote><div><br></div><div><span=
></span></div></body></html>
--=-s7A00AfaDyiaYpgABDO1--