Synchronization Audio Video

-----Message d'origine-----
De : Erwan Gouyette [mailto:[email protected]] 
Envoyé : mercredi 13 mai 2009 18:37
À : 'Greg Wright'
Objet : RE: [Helix-client-dev] Synchronization Audio Video

Ok, thanks for replying. 
I've already taken a look at those documents and done other searches. 

I've found out what's following from different sources (most of them on
Helix Community site and forum):


===========================================================================

-------------------------------------------------------------------------
                        Things happen in this order:
-------------------------------------------------------------------------

   1. After having created the file header and the stream header, the file
format creates the stream packets using an IHXPacket interface. In this
interface is put the deliveryTime (Time stamp for the packet delivery). Then
a pointer to this interface is given to the renderer. 
 
   2. Helix Renderer gives the audio/video compressed data to the decoder.

   3. Decoder returns the raw data (decoded) back to the renderer.

   4. This data is put into the buffers (audio/video separately) with
related time stamps.
  
   5. Renderer sends this raw data to respective output device drivers
depending on the time stamps.



I/ --> Is this good (the order things happen, what each element does...)?
Did I forget something?


-------------------------------------------------------------------------
               Other important things taking place in this process:
-------------------------------------------------------------------------

   a) At startup time, each renderer requests a time synchronization
interval in milliseconds using the unInitialGranularity attribute. The
client then calls IHXRenderer::OnTimeSync each interval.

   b) Time stamps (in packets) matter to the client core in order to pump
data from the data source to the renderers (pumping is time based). Time
stamps matter to audio services (audio stream) to detect gaps and overlap in
uncompressed audio data. Time stamps matter to the audio rendering component
to determine gaps in coded audio data. 
      The time stamps assigned to packets are relative to the start time of
the stream. Packet times are adjusted by the stream start time and the
preroll:

      delivery time = packet time - stream preroll + stream start


II/ --> I guess that when a packet has to be played earlier or later, the
delivery time has to be changed. Which element is calculating the moment it
has to be played (the Audio Services like it is told below?)? When does this
occur? Is there any change concerning the delivery time information inside
the stream packet?


   c) Helix synchronizes playback to a presentation's audio track. If there
is no audio track, it synchronizes playback based on the system time.
      Audio Services provides playback synchronization to all rendering
plug-ins. 
      The synchronization values are based on the actual playback time in
the audio hardware. The audio hardware generates the synchronization signals
based on the actual playback of the audio track. The AudioDevice object
passes these signals back to the client, which then issues callbacks to the
rendering plug-in through IHXRenderer::OnTimeSync. The rendering plug-in's
IHXRenderer::GetRendererInfo method specifies the granularity of the time
synchronization that the plug-in needs.


III/ --> So when the playback of a source containing audio and video is
done, two renderers are instantiated. The audio part is played and the video
part is synchronized with the audio. Only Audio Services are in charge for
the synchronization? When there's only video, Audio Services are still in
charge of the synchronization? 

IV/ --> It seems that in this process, only granularity is affected. But, if
you want to adjust the playback, don't we need to change the delivery time
(time stamps) in the stream packets? 


===========================================================================

Thanks for helping me, apologies for any inconveniences. Regards,

Erwan 


-----Message d'origine-----
De : Greg Wright [mailto:[email protected]] 
Envoyé : mercredi 13 mai 2009 15:46
À : Erwan Gouyette
Cc : [email protected]
Objet : Re: [Helix-client-dev] Synchronisation Audio Video

Erwan Gouyette wrote:
> Hi! Could somebody explain to me what are the mechanisms to synchronize 
> audio and video from the same source? Thanks,

Have you taken a look at the client architecture documents
we have on the Helix site?

https://client.helixcommunity.org/2003/devdocs/intro/client_intro.htm

Give that try first. Also, take a look around at the other
documents we have on there.


--greg.