Re: Problem with NDISRC, WHIPSINK, and audio

GST Developer via gstreamer-devel <[email protected]> Sat, 15 Jul 2023 12:15:36 +0100
Newsgroups	gmane.comp.video.gstreamer.devel
Message-ID	<[email protected]>
besides the full log (second one) that i shared,   http://www.gallery.co.uk/gstlog2.txt.zip

 TL:DR this appears to be the crux of the issue:

Pipeline is live and does not need PREROLL ...
0:00:00.036747546 11562 0x558781cb5400 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<ndisrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
0:00:01.220267971 11562 0x7fe028007330 WARN             ndireceiver net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0> Dropping old buffer -- queue has 11 items
0:00:01.232472921 11562 0x7fe028007330 WARN             ndireceiver net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0> Dropping old buffer -- queue has 11 items
0:00:01.239202906 11562 0x7fe028007330 WARN             ndireceiver net/ndi/src/ndisrc/receiver.rs:850:gstndi::ndisrc::receiver::Receiver::receive_thread:<ndisrc0> Dropping old buffer -- queue has 11 items




> On 15 Jul 2023, at 11:44, Tarun Tej K <[email protected]> wrote:
> 
> This log snippet does not give much. information. Could you please share the full log?
> 
> On Sat, 15 Jul, 2023, 14:00 GST Developer, <[email protected] <mailto:[email protected]>> wrote:
>> Thanks for your reply.
>> 
>> Sadly, this doesn't change the behaviour - when attempting to send video *and* audio, it stalls after “Redistribute latency…”
>> 
>> So, Just video works:
>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !   whip.  whipsink name=whip   auth-token=“…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>> 
>> 
>> and If I try *just* audio:
>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whip.  whipsink name=whip   auth-token=“...redacted..."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>> This also works !
>> 
>> but when I try to combine them : 
>> 
>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip.  demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>> 
>> it stalls after “Redistribute latency…”
>> 
>> This does feel like some sort of demux deadlock between the ndisrcdemux and the whipsink.  I have pasted the debug output for the stall loop below, which repeats over and over:
>> 
>> 0:00:19.425157202  9470 0x557d29708a00 TRACE              structure gststructure.c:292:gst_structure_new_id_empty_with_size: created structure 0x557d298c1aa0
>> 0:00:19.425181034  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'long-name'
>> 0:00:19.433174494  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 0:00:19.433194340  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'klass'
>> 0:00:19.433204782  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 0:00:19.433216613  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'description'
>> 0:00:19.433223154  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 0:00:19.433232379  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'author'
>> 0:00:19.433246113  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 0:00:19.433254115  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:627:gst_registry_chunks_load_feature: Element factory : npadtemplates=2
>> 0:00:19.433261379  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:526:gst_registry_chunks_load_pad_template: Reading/casting for GstRegistryChunkPadTemplate at address 0x7f3208f72f30
>> 0:00:19.433271543  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:540:gst_registry_chunks_load_pad_template: Added pad_template src
>> 0:00:19.433279923  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:526:gst_registry_chunks_load_pad_template: Reading/casting for GstRegistryChunkPadTemplate at address 0x7f3208f72fb0
>> 0:00:19.433289253  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:540:gst_registry_chunks_load_pad_template: Added pad_template sink
>> 0:00:19.433302373  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistry.c:592:gst_registry_add_feature:<registry0> adding feature 0x557d298bf6d0 (avdec_wmavoice)
>> 0:00:19.433311993  9470 0x557d29708a00 DEBUG        GST_REFCOUNTING gstobject.c:708:gst_object_set_parent:<avdec_wmavoice> set parent (ref and sink)
>> 0:00:19.433322596  9470 0x557d29708a00 TRACE        GST_REFCOUNTING gstobject.c:292:gst_object_ref_sink:<avdec_wmavoice> 0x557d298bf6d0 ref_sink 1->2
>> 0:00:19.433330585  9470 0x557d29708a00 LOG             GST_REGISTRY gstregistry.c:610:gst_registry_add_feature:<registry0> emitting feature-added for avdec_wmavoice
>> 0:00:19.433341108  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:741:gst_registry_chunks_load_feature: Added feature avdec_wmavoice, plugin 0x557d298724d0 libav
>> 0:00:19.433350373  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:583:gst_registry_chunks_load_feature: Plugin 'libav' feature 'avdec_ws_snd1' typename : 'GstElementFactory'
>> 0:00:19.433362464  9470 0x557d29708a00 TRACE        GST_REFCOUNTING gstobject.c:208:gst_object_init:<GstObject@0x557d298c2050> 0x557d298c2050 new
>> 0:00:19.433374983  9470 0x557d29708a00 LOG             GST_REGISTRY gstregistrychunks.c:610:gst_registry_chunks_load_feature: Reading/casting for GstRegistryChunkElementFactory at address 0x7f3208f72ff0
>> 0:00:19.433385348  9470 0x557d29708a00 TRACE              structure gststructure.c:292:gst_structure_new_id_empty_with_size: created structure 0x557d298c1e40
>> 0:00:19.433394095  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'long-name'
>> 0:00:19.433402420  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 0:00:19.433411420  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'klass'
>> 0:00:19.433419783  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
>> 
>> 
>> 
>> 
>>> On 15 Jul 2023, at 01:17, Tarun Tej K <[email protected] <mailto:[email protected]>> wrote:
>>> 
>>> Hi,
>>> 
>>> What is the idea behind fixing the sink pad number i.e., sink_0 to video and sink_1 to audio ? The pads would be linked dynamically anyway.
>>> 
>>> Can you try the below modified pipeline. I have also added a queue in audio branch after demux.
>>> 
>>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip. demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>>> 
>>> 
>>> On Sat, 15 Jul, 2023, 03:30 GST Developer via gstreamer-devel, <[email protected] <mailto:[email protected]>> wrote:
>>>> Hi Folks.
>>>> 
>>>> I am attempting use gstreamer to send content from an NDI Source to a WHIP end point at Dolby.io.
>>>> 
>>>> If I just send VIDEO, its working fine:
>>>> 
>>>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !   whip.sink_0 whipsink name=whip   auth-token=“…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>>>> 
>>>> This works FINE !! we get the signal to the WHIP server and the end to end latency is about 2 seconds.
>>>> 
>>>> Now I want to add AUDIO to that pipeline, and I am trying: 
>>>> 
>>>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip.sink_0 demux.audio ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  ! whip.sink_1  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>>>> 
>>>> Now, it starts up with:
>>>> 
>>>> Setting pipeline to PAUSED ...
>>>> Pipeline is live and does not need PREROLL ...
>>>> Pipeline is PREROLLED ...
>>>> Setting pipeline to PLAYING ...
>>>> New clock: GstSystemClock
>>>> Redistribute latency...
>>>> Redistribute latency...
>>>> Redistribute latency…
>>>> 
>>>> But it stalls right there, and never gets to counting time, and we dont see anything arrive at the WHIP server.
>>>> 
>>>> Might anyone know what I am doing wrong ?
>>>> 
>>>> Many thanks !!!
>>>> 
>>>> 
>>>> 
>>