Re: Deinterlacing proposal

Thu Sep 29 2005, Mikhail Ramendik wrote:

> It's not only better than nothing. It's probably the fastest mode possible, as 
> there is no extra load on the CPU for deinterlacing. So I think this should 
> be kept as "Fast deinterlacing".

Yes, I agree.

> 
> Now, I have spent some time thinking, and I have an idea for a really strong 
> deinterlacer. It will use code from several Transcode plugins, and will 
> hopefully produce the best quality possible for all sorts of incoming 
> material (interlaced, telecined, non-interlaced). "Best for all cases", the 
> user does not have to set up a plugin chain differently for different kinds 
> of films (as in, for example, mplayer - if you want to get real good 
> quality).
> 
> The downsides:
> 
> - It will introduce a fixed lag of, probably, 3 to 7 frames (exact number to 
> be determined). It's desirable to delay the sound accordingly; this should be 
> simple as the lag is really fixed.

The lag is not a problem, because each picture has a timestamp, so the sync
will work even if the lag varies,
as long as the pictures are decoded/deinterlaced in time to be displayed
at the timestamp.


> 
> - It will be really CPU-intensive. Bigtime. Perhaps it can be optimized by 
> using MMX and SSE/SSE2, but I don't think I know how to optimize such code 
> now.The reason: it will heavily rely on motion/interlace detection, sometimes 
> done more than once per frame.

Once the code is working, I'm sure someone that knows 
MMX/SSE/3dnow/Altivec/... can optimize it (it will probably be needed to
do full frame rate even on a relatively fast machine).

> 
> It will take some time in development, but a simple prototype can be developed 
> quickly in order to debug the API (it will provide trivial deinterlacing 
> too).
> 
> To get this done, I will need a "hook". Here's the entire API as planned now:
> 
> DEINTERLACE_LAG will be a hardcoded #define-d value showing the lag (in 
> frames).
> 
This is not needed, see my answer above.


> int deinterlace_init (int x, int y, int d_mode);    this will be the init 
> procedure; x and y show the image size; d_mode is the "deinterlace mode" 
> which the user will be able to define in the config fle, but in early 
> versions just hardcode this to 0. Returns 0 if initialization has failed, 1 
> on success.
Keep in mind that the size of the picture can change at any time
so make it easy to reinit.
And the deinterlace mode should also be possible to change on the fly,
either from a user override, or from information from the mpeg decoder.
Each picture has flags that tells if it is a progressive or an interlaced
picture. For progressive pictures no deinterlacing is needed.
These flags are known to not be 100% correctly coded on all films though.
But it should be an option to use them.

> 
> int deinterlace_frame (yuv_image_t *frame, double framerate);   this will be 
> the main routine. Call this before displaying every frame. Returns 0 if the 
> frame should be dropped, 1 if it should be displayed. (Note: this can result 
> in a varying frame rate, but it's OK for playback.But if the sound sync is 
> too complicated for this case, then let the frame always be displayed). The 
> framerate parameter is indeed needed, and should contain the current 
> framerate. (Will be used to determine if the system should attempt inverse 
> telecine).
> 
> int deinterlace_frame_frombuf(yuv_image_t *frame);  this takes an empty buffer 
> and fills it with the next "lagging" frame, if that is available. Returns 1 
> if a frame was available, 0 if none was available. This should be used to 
> "clear out" the buffers when the playback comes to a stop. 

So for each picture that comes from the mpeg decoder I call deinterlace_frame()
and then directly deinterlace_frame_frombuf() which might return
a picture or not. If it returns a picture it will be displayed at the
correct timestamp. If it doesn't return a picture don't display.
Continue to next decoded picture.

Did I understand this correctly?

Can I call deinterlace_frame several times to buffer up pictures
before frombuf is called or only one time per frombuf?
Will you buffer the deinterlaced picture internally and deinterlace_frame_
frombuf() will only copy the deinterlaced frame, or will it be that
function that does the deinterlacing directly to the specified (empty) buffer?
Also as a note, input frames mustn't be changed, only read from, because
they are used for prediction if they are I or P pictures.
If the input is a B picture you may change the data, this might reduce
the need for copying data in some cases perhaps.

> 
> Can you implement these calls? 

Yes, they might need some small changes to handle the timestamps.
> 
> If so, I can send you stub deinterlace.h and deinterlace.c files . 
> Unfortunately I don't understand autotools, so I'll have to ask you to 
> include them in the build process. (This can be a one-time custom version if 
> you don't want the unfinished stuff in CVS; I just need to be able to compile 
> it all). After that I'll be able to work on the deinterlace code.
> 
Maybe we should make a branch in CVS?

/Bj|rn