Re: Problems when creating pipeline outside main thread

--0000000000001c75ef0629f935c4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Le lun. 23 d=C3=A9c. 2024, 06 h 15, Daniel Spiessberger via gstreamer-devel=
 <
[email protected]> a =C3=A9crit :

> Hi everyone,
>
> I=E2=80=99m working on a QtQuick application that uses GStreamer to displ=
ay video
> streams. On my target hardware, creating the video decoder element
> (v4l2h264dec) takes some time, which blocks the main thread and freezes t=
he
> GUI.
>
> To avoid this, I=E2=80=99ve moved the pipeline creation, starting, and st=
opping to
> a QThread. However, I=E2=80=99m encountering an occasional problem (~1 in=
 100
> runs): the v4l2h264dec element gradually accumulates latency, leading to
> delays of up to ~20 seconds. Strangely, this issue doesn=E2=80=99t occur =
when the
> same pipeline is initialized and run from the main thread.
>

20s is a lot more then v4l2h264dec element can allocate memory.

Here=E2=80=99s the pipeline: udpsrc port=3D5000 ! application/x-rtp, payloa=
d=3D96,
> encoding-name=3DH264 ! queue max-size-buffers=3D0 ! rtph264depay ! h264pa=
rse !
> v4l2h264dec ! imxvideoconvert_g2d ! qtvideosink sync=3Dfalse
> I=E2=80=99ve tried repositioning the queue element, but it doesn=E2=80=99=
t affect the
> outcome.
>
By not using rtpjitterbuffer and ideally an proper rtpbin with RTCP, your
pipeline is enable to compensate delays and time shift. It's possible you
simply have been lucky so far, and changing the timing triggered the issue.

My questions are:
>
>    1. What is the recommended approach for creating and managing
>    GStreamer elements outside the main thread?
>
> One initialized, the pipeline will run on its own thread. Nothing special
should be needed.



>    1. Are there any specific considerations for synchronization or thread
>    safety when using QThread for this purpose?
>
>
If you handle asynchronous messages in that new thread, then you need
internal threadsafety in you app.


>
> Any advice or insights would be greatly appreciated!
>
> Thank you,
> Daniel Spie=C3=9Fberger
>
>

--0000000000001c75ef0629f935c4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote gmail_quote_contai=
ner"><div dir=3D"ltr" class=3D"gmail_attr">Le lun. 23 d=C3=A9c. 2024, 06 h =
15, Daniel Spiessberger via gstreamer-devel &lt;<a href=3D"mailto:gstreamer=
[email protected]">[email protected]</a>&gt;=
 a =C3=A9crit=C2=A0:<br></div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>

 =20

   =20
 =20
  <div>
    <p>Hi everyone,</p>
    <p>I=E2=80=99m working on a QtQuick application that uses GStreamer to
      display video streams. On my target hardware, creating the video
      decoder element (v4l2h264dec) takes some time, which blocks the
      main thread and freezes the GUI.</p>
    <p>To avoid this, I=E2=80=99ve moved the pipeline creation, starting, a=
nd
      stopping to a <code>QThread</code>. However, I=E2=80=99m encountering=
 an
      occasional problem (~1 in 100 runs): the <code>v4l2h264dec</code>
      element gradually accumulates latency, leading to delays of up to
      ~20 seconds. Strangely, this issue doesn=E2=80=99t occur when the sam=
e
      pipeline is initialized and run from the main thread.</p></div></bloc=
kquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">20s is a l=
ot more then v4l2h264dec element can allocate memory.</div><div dir=3D"auto=
"><br></div><div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_contain=
er"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef=
t:1px #ccc solid;padding-left:1ex"><div>
    <p>Here=E2=80=99s the pipeline: udpsrc port=3D5000 ! application/x-rtp,
      payload=3D96, encoding-name=3DH264 ! queue max-size-buffers=3D0 !
      rtph264depay ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d !
      qtvideosink sync=3Dfalse<br>
      I=E2=80=99ve tried repositioning the <code>queue</code> element, but =
it
      doesn=E2=80=99t affect the outcome.</p></div></blockquote></div></div=
><div dir=3D"auto">By not using rtpjitterbuffer and ideally an proper rtpbi=
n with RTCP, your pipeline is enable to compensate delays and time shift. I=
t&#39;s possible you simply have been lucky so far, and changing the timing=
 triggered the issue.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><d=
iv class=3D"gmail_quote gmail_quote_container"><blockquote class=3D"gmail_q=
uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e=
x"><div>
    <p>My questions are:</p>
    <ol>
      <li>What is the recommended approach for creating and managing
        GStreamer elements outside the main thread?</li></ol></div></blockq=
uote></div></div><div dir=3D"auto">One initialized, the pipeline will run o=
n its own thread. Nothing special should be needed.</div><div dir=3D"auto">=
<br></div><div dir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail=
_quote gmail_quote_container"><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><ol>
      <li>Are there any specific considerations for synchronization or
        thread safety when using <code>QThread</code> for this purpose?</li=
></ol></div></blockquote></div></div><div dir=3D"auto"><br></div><div dir=
=3D"auto">If you handle asynchronous messages in that new thread, then you =
need internal threadsafety in you app.</div><div dir=3D"auto"><br></div><di=
v dir=3D"auto"><div class=3D"gmail_quote gmail_quote_container"><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex"><div><ol>
    </ol>
    <p>Any advice or insights would be greatly appreciated!</p>
    <p>Thank you,<br>
      Daniel Spie=C3=9Fberger<br>
    </p>
    <p></p>
  </div>

</blockquote></div></div></div>

--0000000000001c75ef0629f935c4--