Re: Using shmsink/shmsrc with RTP packets

Hi,

Le mercredi 12 juillet 2023 à 14:59 +0530, Anand Sivaram via gstreamer-devel a
écrit :
> Dear All,
> 
> Whenever I use the raw I420 video frame through shmsink/shmsrc, then it is
> working fine.
> But, I need to use RTP packets and I tried the following pipeline.
> 
> gst-launch-1.0 videotestsrc pattern=0 ! capsfilter caps='video/x-
> raw,format=I420,width=640,height=360,framerate=30/1' ! videoscale ! videorate
> ! 
> videoconvert ! timeoverlay ! \
> x264enc key-int-max=30 ! capsfilter caps='video/x-h264,stream-format=byte-
> stream' ! rtph264pay pt=96 ! \
> shmsink socket-path=/tmp/gstshm sync=true wait-for-connection=false
> 
> gst-launch-1.0 shmsrc socket-path=/tmp/gstshm ! \
> capsfilter caps='application/x-rtp,media=(string)video,clock-rate=(int)90000'
> ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideos
> ink

The actual error didn't make it into the error message (a bug probably?), but if
you set GST_DEBUG=2 you can see:

  ERROR rtpbasedepayload
gstrtpbasedepayload.c:970:gst_rtp_base_depayload_handle_event:<rtph264depay0>
Segment with non-TIME format not supported

You can switch shmsrc to use TIME format with the "do-timestamp=true" property.
I would also make it behave like a live source with the is-live property. After
fixing this, you may get warning like such (and dropped frames):

   WARN videodecoder
gstvideodecoder.c:3668:gst_video_decoder_clip_and_push_buf:<avdec_h264-0>
Dropping frame due to QoS. start:0:00:04.677027371 deadline:0:00:04.677027371
earliest_time:0:00:04.68462098

You can fix this using the rtpjitterbuffer. You may want to reduce the default
latency though. A working pipeline:

GST_DEBUG=2 gst-launch-1.0 \
  shmsrc is-live=1 do-timestamp=1 socket-path=/tmp/gstshm ! capsfilter
caps='application/x-rtp,media=(string)video,clock-rate=(int)90000' \
  ! rtpjitterbuffer ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert !
autovideosink

regards,
Nicolas