Re: Timestamp, flags, YUV, play VOB files

Bj|rn Englund wrote:

> The timestamps should be set to the time you want the picture to be
> displayed. I don't know if dropping frames would be good, depends on how
> you intend to make the deinterlacing I suppose.

This is for the inverse telecine case, which you have described.

> The time stamps are measured in a 90kHz clock (stored in an uint64_t).

OK, this will be in the API. 

It will probably be a new struct type, codein_frame_t, including a 
yuv_image_t, the timestamp, and the flags.

> The mpeg decoder can, if the stream is field coded, output
> fields at 60 fields per second (1500) but what ogle does now is
> to combine 2 fields to one frame and output at 30fps (3000).
> This could (maybe) be changed if you think it is better for
> the deinterlacer with 60 fields per second in case the stream is coded so.

No need. This would only speed things up if we do deinterlacing by expanding 
one field and discarding the other; but this method is handled by the 
fast-deinterlacer (based on your patch; I will soon send you a version of 
that with all the fixes discussed later).

> A 24 fps progressive film will have the progressive frame flag set
> and the repeat_first_field set on every other picture.
> What should be output in that case (to an interlaced display) 

To be more precise: to an NTSC interlaced display, which takes 30 frames/sec. 
A PAL interlaced display is 25 frames/sec and this process (known in other 
sources as telecine) is not needed.

> Sometimes this 3/2 conversion is done before coding so you want to do
> the inverse to deinterlace.

Actually, to "inverse telecine". 

> You have (30fps):
> 0.0 0.1, 1.0 1.1, 2.0 2.1, 3.0 3.1, 4.0 4.1, ..., 29.0 29.1, 30.0 30.1
> timestamps:
> 0      , 3000   , 6000   , 9000   , 12000  , ..., 87000    , 90000
> But what this really is (and hopefully the deinterlacer will detect) is:
> 0.0 0.1, 1.0 1.1, 1.0 2.0, 2.1 3.0, 3.1 3.0 , ...,23.1 23.0, 24.0 24.1
> or in other notation 0, 1*, 2, 3*, ...
> Which should result in (24fps):
> 0,    1,     2,     3, ...,    23, 24
> timestamps:
> 0, 3750, 7500, 11250, ... , 86250, 90000

Thanks for the hint on the timestamps. Now I know how to go about them in this 
case. (Or at least I have some ideas.)

> If we have pictures that are truly interlaced (not from film material)
> then you obviously have to do something more complex.
> If you for instance for every field in a 30 fps ( 60 fields/s) stream,
> output a whole frame, resulting in a 60frames/s stream. Then you should
> timestamp the new pictures:
> input (30 fps):
> 0,1,2,3,...
> timstamps:
> 0,3k,6k,9k,...
> output (60fps)
> 0,1,2,3,4,5,6,7,...
> timestamps:
> 0,1.5k,3k,4.5k,6k,7.5k,9k,10.5k,...

I know this double framerate trick. I don't really want to do that; I suspect 
it won't make things any better compared to the fast-deinterlace. The 
noticeable part is the level of detail, not the smoothness of movement (30fps 
is enough for movement anyway).And the level of detail is not increased.

So I have (from transcode) an algorithm which only deinterlaces (by expanding 
data from one field) the parts of the picture where deinterlacing is visible 
(i.e. the parts in motion). This is CPU intensive but preserves maximum 
detail where no motion is happening.

In fact I'll probably start by just adapting this algorithm. This will yield a 
working version. I will then turn to the addition of inverse telecine, based 
on detection.

> > In the YUV model used in ogle, is every plane one-byte-per-pixel?
>
> It's as coded in the mpeg stream 4:2:0
> Y = 1 byte per pixel
> U and V is 0.25 byte per pixel each.
> U and V are byte values but they 'map' to 4 Y values.

OK.

> I also think the mapping to Y values is different depending on if
> the picture is frame or field coded.

But the output of the decoder is always frame coded, right?

If I can not rely on the sizes of a pixel in U and V, then I can only 
deinterlace Y... which might be enough for most cases anyway.

> Another note.
> There can be still pictures in the mpeg stream.

Question: why deinterlace them at all? A still picture has no motion. 
Interlacing is therefore not seen in it. I would suggest not calling the 
deinterlacer for still pictures.

> So for every picture input to the deinterlacer, that picture
> must be returned without having to input any more pictures.
> Is this a problem? 

Not really. I will provide for this in the API and code.

> Is there any case when you have to look at future 
> pictures to deinterlace, instead of only previous ones?

Inverse telecine. You do have to have a buffer in order to check whether it's 
telecined. But then, if there are no buffered pictures, I just won't check 
for telecine. and will deinterlace normally instead.

> It might be possible to make a call when ogle detects a still
> picture so you can output it directly in that case.

Doesn't ogle know if a picture is still, by the time it should call the 
deinterlacer? If it does, I suggest not calling - even though I will support 
this case.

In fact, if ogle reaches a still picture, it will need to do something 
*before* displaying it. Namely, take all the available output from the 
deinterlacer and show that. And only then show the still picture.

This is because, while the decoder has reached the still picture, the 
deinterlacer has output LAG pictures less. Before real displaying reaches the 
still, the LAG pictures must be shown,

-- 
Yours, Mikhail Ramendik