Re: Arpi: we need to discuss MPlayer's A/V sync

Hi,

> Albeu said that MPlayer needs timestamp for every audio frame and that 
> it cannot just decode those frames and use the duration given by the 
> decoder for sync.
It's false.
Mpeg2 has not timestamps for all frames, and progresisve streams has no
constant duration but playback works well. same for realvideo.

Normally mplayer uses only the duration from the decoders to do teh timing,
and uses timestamps only to detect desync and slowly correct it by changing
video speed. Anyway, for variable speed video (no constant FPS but every
frame has duration or timestamp) it requires timestamps as duration is
calculated from that *if no FPS given).
> 
> So, how exactly does this thing work? How does it handle most current 
> containers (AVI, Ogg) that usually contain audio frames without timecodes?

avi is simple - it has either CBR audio, so you have a fixed compression
ratio (timestamp=bytepos*constant), or has VBR audio with constant
packets/sec rate. Byte size of packets varies, but each packet contains the
same amount of PCM samples.

Ogg is problematic, as for low bitrates (especially at black video frames or
silence sound) there are no timestamps for many seconds, so a-v may shift a
bit. Also, ogg files has no constant FPS, so you need timestamps to
calculate durations. Anyway, mplayer plays ogg files well, at least the
samples i tried.

> Adding timecode (or even time offset) for every frame makes overhead 
> significantly bigger.
you MUST have either fixed rate (bitrate or packet rate for audio, and
FPS for video) or duration for every video frame. Audio don't have to have
timestamps too often, but it should have timestamps for keyframes
for seeking (i mean if there is a video keyframe with timestamp X, there
should be an audio timestamp X (or close to X) too).

> Would it be possible to just handle frames of a single Block as one big 
> frame, when doing sync? This way you wouldn't need to quess timecodes 
> for frames which don't already have one.

No.
But if you don't have timestamps nor FPS, how should teh decoder know the
display time of the frames??? Or you mean fixed FPS for a block of frames?

MPlayer currently increases previous timestamp by previous frame duration if
no timestamp for the current frame, so it works even with zero timestamps,
but no av sync _correction_ then.

> In case you are not familiar with MCF, Block is an element that contains 
> timecode and data. With audio it normally contains several frames and 
> then you only have timecode for the first one. Of course a Block only 
> contains audio for around 50 ms, so there is no trouble with seek 
> accuracy nor sync drift.

audio is not problem while you have constant audio samplerate (constant
number of PCM samples per second - note that quicktime allows variable
samplerate too - but i haven't seen any file using this feature)

for constant FPS video or vidoe frames with duration value i don't need
timestamps.

the problem, again, with ogg files that no fixed FPS and no timestamp for
every video frame.


A'rpi / Astral & ESP-team

--
Developer of MPlayer, the Movie Player for Linux - http://www.MPlayerHQ.hu