Re: Buffering in the Helix Client

Mahmoud Hammoud wrote:
> __Dear Greg,
>  
> I found your proposed solution to my problem very appealing, but could 
> you please elaborate a little more on that? I will formulate my 
> questions in a list so that you can quickly provide me with some brief 
> remarks, and I really appreciate you taking time off your very busy 
> schedule to answer me:
>  
> 1) How to know whether client side or server side rate control is being 
> used?

You can look in the RTSP chat headers or code. It can be controlled
via these preferences (protocol/rtsp/rateadaptinfo.cpp):

             ReadPrefBOOL( pPrefs, "HelixAdaptation", m_bHlxAdaptEnabled );
             ReadPrefBOOL( pPrefs, "3GPPAdaptation", bNADU );
             ReadPrefBOOL( pPrefs, "ServerSideRateControl", bSSRC );


>  
> 2) What modules contain the implementation of these rate control 
> mechanisms? (I'm guessing rateadaptinfo.cpp is one of the essential 
> source files but I'm sure there's more to it than that). That being 
> said, is a rate adaptation request (e.g. tell the server to send at a 
> higher bit rate) done at the RTSP level? Can you tell me which functions 
> usually perform that?

Yes, just a take a look for 'SetDeliveryBandwidth' messages in the
RTSP protocol code.

>  
> 3) When the server sends at a higher/lower bit-rate, how does that 
> affect the playback quality? 

It does not, it simple changes the amount of data the client
has on hand.

 > Is that taken into consideration? From what
> I know a client generally has a playout schedule which is a delayed 
> version of the encoding/sending curve at the transmitter. So I guess the 
> client initially starts playing at the original clip-rate 
> (i.e. according to the initially agreed upon playout schedule) and in my 
> case tells the server at some point to send at a higher bit-rate 
> (because bad connection quality will be encountered soon). What does the 
> client do then, when its rate adaptation request is "accepted", with 
> respect to its playback rate?  

The client will simple store the extra packets until they are
needed at the renderers/codecs for playback. The engine time-delivers
packets to the renderers for processing on an as-needed basis.

This mechanism is actually a bit complicated in practice, so be very
careful if you try to tweak it. However, I don't believe you would
have to.

The big picture is this is not easy stuff. We have never had any
code in place that takes future-quality of the connection into any API
consideration that I am aware of, and so, you will not find easy
hooks to just add it in.

My suggestion of just over sending data if  you know the
connection is going to be bad is a quick easy thing to do,
but it does not take into account latency, memory requirments,
server side load, server side rate adaptation (Helix or 3GPP) or
any other of a number of important factors.

Perhaps a more sophisticated approach would include re-routing the
connections between server and client, changing protocols or
changing to a less demanding version of the clip (sure-stream is
already in place for this). Things like this are done all the
time in the mobile device world where you can drop from 3G to
EDGE, for example or wireless to cell-network.

--greg.



>  
> Thank you a lot,
>  
> Mahmoud
> 
> On Tue, Jul 29, 2008 at 7:33 PM, Greg Wright <[email protected] 
> <mailto:[email protected]>> wrote:
> 
>     Mahmoud Hammoud wrote:
> 
>         Dear Greg,
>          I am writing you directly because I didnt really get meaningful
>         answers the last time I tried the mailing list, and my problem
>         is really urgent, so I appreciate your understanding and you
>         taking time to answer me. I am using the hxclient_1_5_0_cayenne
>         branch (Profile helix-client-all-defines, SYSTEM_ID =
>         win32-i386-vc6), and I am trying to implement a way of
>         controlling the amount of buffering to be done by the client
>         based on external measurements (independent of the state of the
>         player, or the congestion/connection quality it experiences).
>         For example I would like to be able to pass a "prediction"
>         (supposedly obtained from an outside module that interfaces with
>         the Helix core) telling/forcing the core to buffer data for a
>         certain amount of time (say because we will experience bad
>         connection quality soon). After reading the "devdocs" related to
>         buffering in the Helix client, and thoroughly examining the
>         corresponding parts of the code, some of my issues remained
>         unanswered and here is where I need your help:
>          1) Rebuffering seems to be done only for audio streams (when
>         accelerated delivery doesnt make up for the data shortage).
>         Where does the rebuffering of video streams take place? Here I
>         am making reference to HXSource::ReportRebufferStatus which gets
>         exclusively called by audio renderes (so the else clause where
>         DoRebuffer( ) is called for video streams is somehow never
>         executed).
> 
> 
>     We never rebuffer due to video. The idea is that the audio playback,
>     what the person is hearing at any given instant, is the master time
>     line. It drives everything else. So, it is imperative that we never
>     allow the audio to run dry or otherwise break up. This is why we
>     rebuffer the audio stream. For video, it is not as important. We have
>     chosen to go ahead and let the video drop frames (even back to showing
>     only key frames) as long as we can keep the audio from breaking up.
> 
>     We could have equally well decided to rebuffer if we are unable to
>     keep the video playing well. However, given the history of where
>     and when the media engine was used, that is not a decision we made.
> 
>     It would not be hard to change this (see client/audiosvc/*) behavior
>     and have rebuffering caused by video renderers as well. That would be
>     a valid choice in a lot of scenarios I would guess; where video has
>     to be shown with all frames or none at all.
> 
> 
>         2) When rebuffering is to be done for an audio stream,
>         accelerated packet delivery is stopped, and the chain
>         DoRebufferIfNeeded( ) - DoRebuffer( ) - ReBuffer( ) -
>         m_pBufferManager->Rebuffer(preroll increment). Is this accurate?
>         Does this mean that rebuffering consists of merely incrementing
>         our preroll by a constant amount of time? Is there some actual
>         buffering of the media packets that I am not seeing here? I
>         appreciate it if you would point it out to me.
> 
> 
>     If we encounter a rebuffering event we assume that the connection is
>     not that good and that rebuffering more data will help keep the
>     amount of rebuffers down. So, that is why you see an increment there.
>     This increment only goes up to some maximum value. It is easy to
>     change how much, if any, of an increment you get.
> 
>     Rebuffering itself is just the act of pausing playback until we have
>     satisfied all the pre-roll requirements of each renderer in the
>     playback chain. This includes any *extra* we have added (above).
> 
> 
>         3) If a simple solution to my initial problem description comes
>         to your mind, please be kind to provide me with your valuable
>         thoughts.
> 
> 
>     If you can somehow detect impending degradation of the connection
>     ahead of time, then I would just look at accelerating delivery of
>     the packets from the server. You can ask the server to send you
>     data at just about any rate. We start off very fast (to fulfill
>     the pre-roll as fast as we can) and then change it to just about
>     the clip bit-rate. If the engine senses the data getting low it
>     will tell the server to increase the rate. This applies only to
>     client side rate control. If the connection is using server side
>     rate control then you really can't do anything except maybe lie
>     to the server about how much data you have.
> 
>     --greg.
> 
>     Adding helix-client-dev, please keep all discussion on a helix
>     list. If you want you can always email and point me to a
>     discussion, but I don't have that much time right now and emails
>     directed only to me may not get answered.
> 
> 
> 
>          Best Regards,
>          Mahmoud Hammoud
> 
> 
> 


_______________________________________________
Helix-client-dev mailing list
[email protected]
http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev