Re: Adapting H264 encoding rate

Hello,

First of all many thanks for your various suggestions, that helped a lot!

Here is a little followup,

I've been able using mp4mux with varying caps/resolution, with the help
of h264timestamper (to prevent this error:
"gst_qt_mux_add_buffer:<dry_recorder> error: Buffer has no PTS"), I also
switched to h264 main profile and removed zerolatency.

I think your feedback "2 to 3 second IDR frame is good idea for general
streaming" is the default value for nvh264enc here:
https://gstreamer.freedesktop.org/documentation/nvcodec/GstNvBaseEnc.html?gi-language=c#GstNvBaseEnc:gop-size

What I could not do is:

   - enable rc-lookahead (20): video freezes and does not recover
   - I could not find how to enable CVBR mode for the encoder

In our scenario, we both record input streams and try to send them back
modified (with a video effect) with as little latency as possible over
webrtc. Controlling the encoder bitrate is helpful when network conditions
vary, but we've lowered the bitrate update period (from 1 to 8 seconds).
Anyway, it seems the encoder does not respect the max/target bitrate,
especially when the limit is already high.

Thanks again for your leads,
Guillaume

Le ven. 2 juin 2023 à 20:18, vinod kesti <[email protected]> a écrit :

> x264 supports dynamic bitrate but I am not sure about the nvenc.
>
> Here are my thoughts:
> 1) Updating bitrate every 1 second is bad idea. Use CVBR instead of VBR so
> that you have better control over the datarate and quality.
> 2) 2 to 3 second IDR frame is good idea for general streaming. May be you
> can increase if bitrate is too low. In case of
> 3) I would suggest control resolution/framerate based on the based bitrate
> you are choosing.
>
> By the way latest mp4 muxer supports the dynamic resolution change
> recording. You can consider using so the dry recording happens without any
> transcoding.
> I am wondering why you are using zero latency and baseline for recording,
> You can go with rc-lookahead and main profile (Which is supported by most
> of the devices in todays world) so that you have better quality for same
> bitrate
>
>
>
> Sent from Yahoo Mail. Get the app <https://yho.com/148vdq>
>
>
> On Thursday, 1 June, 2023 at 02:18:19 am GMT-5, Guillaume Denis via
> gstreamer-devel <[email protected]> wrote:
>
>
> Hello,
>
> We are using GStreamer within a WebRTC app (the WebRTC part is done with
> https://github.com/pion/webrtc) where we estimate the best target bitrate
> depending on network conditions (congestion control).
>
> I was wondering if anyone has experience about the best way to control
> encoders (x264enc and nvcodec' nvh264enc) in particular:
>
>    - when we update the target bitrate: should it be done at a slow pace?
>    (or: is updating every second could be useless or even harmful in any way)
>    - can it have an impact on keyframes (other than their quality)?
>    - are there other encoders' properties worth controlling?
>
> Just in case anyone is interested, below is a typical pipeline we use.
>
> Thanks,
> Guillaume
>
> ## Pipeline with RTP in/out (appsrc/appsink)
> ## and two filesinks (one for the in/dry stream, one after applying a
> video FX, here facedetect)
> ## we force streams to a 800x600 resolution not to crash muxers with
> changing resolutions
>
> appsrc name=video_src is-live=true format=GST_FORMAT_TIME
> min-latency=33333333
> appsink name=video_sink qos=true
> matroskamux name=dry_recorder ! filesink location=data/dry.mkv
> matroskamux name=wet_recorder ! filesink location=data/wet.mkv
>
> video_src. !
> application/x-rtp,encoding-name=H264 !
> rtpjitterbuffer name=video_buffer do-lost=1
> add-reference-timestamp-meta=true latency=200 !
> rtph264depay wait-for-keyframe=true !
> h264parse !
> nvh264sldec min-force-key-unit-interval=3000000000
> discard-corrupted-frames=true qos=true !
> cudaupload !
> cudaconvertscale !
> cudadownload !
> videorate !
> video/x-raw, framerate=30/1, width=800, height=600, format=I420,
> colorimetry=bt601, chroma-site=jpeg, pixel-aspect-ratio=1/1 !
>
> tee name=tee_video_in !
> queue !
> nvh264enc name=video_encoder_dry rc-mode=3 preset=4 gop-size=15
> zerolatency=true b-adapt=0 bframes=0 rc-lookahead=0
> min-force-key-unit-interval=3000000000 qos=true ! video/x-h264,
> profile=constrained-baseline !
> h264parse !
> dry_recorder.
>
> tee_video_in. !
> queue !
> videoconvert !
> facedetect !
> queue !
> cudaupload !
> cudaconvertscale !
> cudadownload !
> video/x-raw, format=I420, colorimetry=bt601, chroma-site=jpeg,
> pixel-aspect-ratio=1/1 !
> nvh264enc name=video_encoder_wet rc-mode=3 preset=4 gop-size=15
> zerolatency=true b-adapt=0 bframes=0 rc-lookahead=0
> min-force-key-unit-interval=3000000000 qos=true !
> video/x-h264, profile=constrained-baseline !
> h264parse !
>
> tee name=tee_video_out !
> queue !
> wet_recorder.
>
> tee_video_out. !
> queue !
> rtph264pay config-interval=-1 min-ptime=30000000 !
> video_sink.
>