Re: GStreamer playback of MP4 files with and without audio

--000000000000ebfaff0623795c4e
Content-Type: text/plain; charset="UTF-8"

I would add a handler for the "pad-added" event on the filesrc element and
only create/add the audio part of the pipeline if you get an audio pad
created.  That is what I have done to handle rtspsrc streams that may or
may not have audio.  Reply if you want more details.

g_signal_connect(m_sourceElement, "pad-added",
G_CALLBACK(&SynxRTSPPipeline::onPadAdded), this);

On Wed, 2 Oct 2024 at 07:30, Terry Barnaby via gstreamer-devel <
[email protected]> wrote:

> I am developing a C++ video inspection program that creates MP4/H264/MP3
> files and needs to play them back with the video stream processed by
> various gstreamer elements and audio separately.
>
> In general this has been working fine when just video was being
> recorded/played back, but I am just adding audio to the mix. Some MP4 files
> will contain only a video stream and some will contain both video and audio
> streams.
>
> I need some way to handle the playback of these MP4 files that may or may
> not have MP3 audio streams.
>
> As a simple idea if I use something like (The real C++ code constructs the
> gstreamer piple line and tees the video stream to various gstreamer sub
> pipelines):
>
> gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux
>
>  demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
>
>  demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
>
> This plays back the video and audio streams fine (not sure how well
> synchronised?) from an MP4 with video and audio streams, but hangs if the
> MP4 only has a video stream.
>
> So I think I need to:
>
> 1. Maybe there is some gstreamer element or attribute that can ignore the
> audio stream if not present somehow ?
>
> 2. Check if the MP4 file has an audio stream before creating the gstreamer
> pipeline in C++.
>
> 3. Create the basic C++ pipleline in C++ and interrogate pads or something
> somehow and add the "demux.audio_0 ! queue ! decodebin ! audioconvert !
> pulsesink" sub pipeline if it is seen an audio stream is present.
> Any ideas on the simplest/most CPU efficient way of doing this with
> gstreamer ?
>
>
>

--000000000000ebfaff0623795c4e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>I would add a handler for the &quot;pad-added&quot; e=
vent on the filesrc element and only create/add the audio part of the pipel=
ine if you get an audio pad created.=C2=A0 That is what I have done to hand=
le rtspsrc streams that may or may not have audio.=C2=A0 Reply if you want =
more details.</div><div><br></div><div>	g_signal_connect(m_sourceElement, &=
quot;pad-added&quot;, G_CALLBACK(&amp;SynxRTSPPipeline::onPadAdded), this);=
<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gm=
ail_attr">On Wed, 2 Oct 2024 at 07:30, Terry Barnaby via gstreamer-devel &l=
t;<a href=3D"mailto:[email protected]">gstreamer-devel@=
lists.freedesktop.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex"><u></u>

 =20

   =20
 =20
  <div>
    <p>I am developing a C++ video inspection program that creates
      MP4/H264/MP3 files and needs to play them back with the video
      stream processed by various gstreamer elements and audio
      separately.</p>
    <p>In general this has been working fine when just video was being
      recorded/played back, but I am just adding audio to the mix. Some
      MP4 files will contain only a video stream and some will contain
      both video and audio streams.</p>
    <p>I need some way to handle the playback of these MP4 files that
      may or may not have MP3 audio streams.<br>
    </p>
    <p>As a simple idea if I use something like (The real C++ code
      constructs the gstreamer piple line and tees the video stream to
      various gstreamer sub pipelines):<br>
    </p>
    <pre>gst-launch-1.0 -v filesrc location=3Dtemp.mp4 ! qtdemux name=3Ddem=
ux</pre>
    <pre>=C2=A0demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesin=
k</pre>
    <pre>=C2=A0demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink=
</pre>
    <p>This plays back the video and audio streams fine (not sure how
      well synchronised?) from an MP4 with video and audio streams, but
      hangs if the MP4 only has a video stream.</p>
    <p>So I think I need to:</p>
    <p>1. Maybe there is some gstreamer element or attribute that can
      ignore the audio stream if not present somehow ?</p>
    <p>2. Check if the MP4 file has an audio stream before creating the
      gstreamer pipeline in C++.</p>
    <p>3. Create the basic C++ pipleline in C++ and interrogate pads or
      something somehow and add the &quot;demux.audio_0 ! queue ! decodebin=
 !
      audioconvert ! pulsesink&quot; sub pipeline if it is seen an audio
      stream is present.<br>
    </p>
    Any ideas on the simplest/most CPU efficient way of doing this with
    gstreamer ?<br>
    <p><br>
    </p>
  </div>

</blockquote></div>

--000000000000ebfaff0623795c4e--