Re: Problem with NDISRC, WHIPSINK, and audio

Ok, you will need Gstreamer version 1.22 for this, where webrtcbin has this
signal 'request-aux-sender' implemented.

On Sat, 15 Jul, 2023, 19:53 GST Developer, <[email protected]> wrote:

> Thanks Tarun,
>
> *we are making progress *- this pipeline *does* startup, but it fails
> after a few seconds:
>
> *gst-launch-1.0  ndisrc ndi-name="NDIPE8 (SIGGEN)" ! ndisrcdemux
>  name=demux demux.video ! queue ! videoconvert ! whipwebrtcsink name=ws
> signaller::whip-endpoint="https://director.millicast.com/api/whip/myStreamName
> <https://director.millicast.com/api/whip/myStreamName>",auth-token=“..redacted.."
> demux.audio ! queue ! audioconvert ! ws.*
>
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> Redistribute latency...
> Redistribute latency…
>
> *<it runs fine here for a few seconds with the numbers running>*
>
> thread 'tokio-runtime-worker' panicked at 'Signal 'request-aux-sender' of
> type 'GstWebRTCBin' not found',
> /home/ubuntu/.cargo/git/checkouts/gtk-rs-core-7be42ca38bd6361c/44c4ab3/glib/src/object.rs:2642
> :32
> note: run with `RUST_BACKTRACE=1` environment variable to display a
> backtrace
> thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err`
> value: PoisonError { .. }', net/webrtc/src/webrtcsink/imp.rs:3013:47
> ERROR: from element /GstPipeline:pipeline0/GstWhipWebRTCSink:ws: Panicked:
> called `Result::unwrap()` on an `Err` value: PoisonError { .. }
> thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err`
> value: PoisonError { .. }', net/webrtc/src/webrtcsink/imp.rs:3013Execution
> ended after 0:00:01.886767934
> :47Setting pipeline to NULL ...
>
> ERROR: from element /GstPipeline:pipeline0/GstWhipWebRTCSink:ws: Panicked
> ERROR: from element /GstPipeline:pipeline0/GstWhipWebRTCSink:ws: Panicked
> ERROR: from element /GstPipeline:pipeline0/GstWhipWebRTCSink:ws: Panicked:
> called `Result::unwrap()` on an `Err` value: PoisonError { .. }
> ERROR: from element /GstPipeline:pipeline0/GstWhipWebRTCSink:ws: Panicked
> Freeing pipeline …
>
>
> On 15 Jul 2023, at 14:47, Tarun Tej K <[email protected]> wrote:
>
> auth-token=“…redacted…."
>
>
> which doesnt appear to be a supported parameter in whipwebrtcsink so
> perhaps I cant use that in place of whipsink
>
> whipwebrtcsink does have 'auth-token' property.
>
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/blob/main/net/webrtc/src/whip_signaller/imp.rs#L577
>
> Did you try setting it like this `whipwebrtcsink
> signaller::auth-token="wxyz"` ?
>
> On Sat, Jul 15, 2023 at 6:44 PM GST Developer <[email protected]>
> wrote:
>
>
> Thanks Tarun
>
> The Dolby.io workflow also requires
>
> auth-token=“…redacted…."
>
> which doesnt appear to be a supported parameter in whipwebrtcsink so
> perhaps I cant use that in place of whipsink
>
> On 15 Jul 2023, at 13:53, Tarun Tej K <[email protected]> wrote:
>
> The same ndisrc  / ndisrcdemux works fine into a webrtcsink pipeline with
> A+V
>
> We now have a whip signaller implementation in the webrtcsink -
>
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1168
> .
> But I don't think this is a part of a release yet, so you will have to
> set up an uninstalled version.
>
> On Sat, Jul 15, 2023 at 5:18 PM GST Developer <[email protected]>
> wrote:
>
>
>
>
> On 15 Jul 2023, at 12:36, Tarun Tej K <[email protected]> wrote:
>
> Ok. I am not familiar with ndisrcdemux, does this mean the buffers are
> being dropped at the demuxer?
>
>
> This also new to me, but its certainly warning that the input buffer is
> filling, presumably because its not draining.  I did try with a much larger
> receive buffer but it does the same thing, so this does look like some sort
> of deadlock.
>
> The same ndisrc  / ndisrcdemux works fine into a webrtcsink pipeline with
> A+V,  so it looks like some quirk where it doesnt like whipsink in A+V
> workflow.  I can use gst videotestsrc and audiotestsrc into whipsink fine,
> so I know there is nothing fundamentally wrong with either of the
> components, ndisrc, ndisrcdemux and whipsink.
>
> Also what is the GST_DEBUG value set in the environment
>
>
> in the log, it was *.7 I believe,  the TL:DR was *.3 I think.
>
> and gstreamer
> version you are using?
>
>
> 1.20.3
>
> Thanks !
>
>
>
> On Sat, Jul 15, 2023 at 4:45 PM GST Developer <[email protected]>
> wrote:
>
>
> besides the full log (second one) that i shared,
> http://www.gallery.co.uk/gstlog2.txt.zip
>
> TL:DR this appears to be the crux of the issue:
>
> Pipeline is live and does not need PREROLL ...
> 0:00:00.036747546 11562 0x558781cb5400 FIXME                default
> gstutils.c:4025:gst_pad_create_stream_id_internal:<ndisrc0:src> Creating
> random stream-id, consider implementing a deterministic way of creating a
> stream-id
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> Redistribute latency...
> Redistribute latency...
> 0:00:01.220267971 11562 0x7fe028007330 WARN             ndireceiver
> net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0>
> Dropping old buffer -- queue has 11 items
> 0:00:01.232472921 11562 0x7fe028007330 WARN             ndireceiver
> net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0>
> Dropping old buffer -- queue has 11 items
> 0:00:01.239202906 11562 0x7fe028007330 WARN             ndireceiver
> net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0>
> Dropping old buffer -- queue has 11 items
>
>
>
>
>
> On Sat, 15 Jul, 2023, 03:30 GST Developer via gstreamer-devel, <
> [email protected]> wrote:
>
>
> Hi Folks.
>
> I am attempting use gstreamer to send content from an NDI Source to a WHIP
> end point at Dolby.io.
>
> If I just send VIDEO, its working fine:
>
> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux
> demux.video ! queue ! videoconvert ! x264enc !
> video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !
>   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000'
> !   whip.sink_0 whipsink name=whip   auth-token=“…redacted….."
>  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>
> This works FINE !! we get the signal to the WHIP server and the end to end
> latency is about 2 seconds.
>
> Now I want to add AUDIO to that pipeline, and I am trying:
>
> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux
> demux.video ! queue ! videoconvert ! x264enc !
> video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !
>   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000'
> !  whip.sink_0 demux.audio ! audioconvert ! opusenc ! rtpopuspay !
> 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'
>  ! whip.sink_1  whipsink name=whip   auth-token="…redacted….."
>  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>
> Now, it starts up with:
>
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> Redistribute latency...
> Redistribute latency…
>
> But it stalls right there, and never gets to counting time, and we dont
> see anything arrive at the WHIP server.
>
> Might anyone know what I am doing wrong ?
>
> Many thanks !!!
>
>
>
>
>
>
>
>
>