Re: Advanced Motion Compensation for sub 500kbit DVD Video

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 19 July 2002 13:32, Dustin Stuart wrote:
> Hi everyone,
>
> This is my first post to the Tarkin list after a quick skimming
> of archived aritcles.
>
<snip - brilliant idea>
>
> This method presents huge advantages for decoding because the
> idea of a video 'frame' can be dropped completely! A video
> recorded at 24 frames/second can be played at 60 or 70 on a
> decoder fast enough. But the complexity of my method (and the
> numerous stretchings that would take place) may prohibit further
> development.
>
> But I have done the math. If the keyframe consumes only 30kb,
> there should be more than 100kb for reencoded triangles and MVs
> if keyframes occur every couple of seconds...

Hi Dustin,

That sounds good. And somewhat familiar 
(http://www.xiph.org/archives/vorbis-dev/200102/0135.html) but 
ofcourse that doesn't take away the fun :-). I've been working on 
doing motion compensation for such a scheme, since we'd have to 
have some sort of vector field mapping one frame to another. We'll 
need that anyway, for any kind of codec, unless someone comes up 
with a transform that deals with translation (now that would be 
interesting).

A slightly different idea would be to do away with the concept of 
keyframes, and instead use separate textures that are streamed 
alongside the motion data. This would require a lot of memory for 
texture caching on the decoder side though, and ideally the decoder 
would need to know which textures to keep for how long (ie when 
they are going to be needed). It shouldn't be too hard to put that 
information into the bitstream, unless we are doing live encoding 
and streaming, in which case the information simply doesn't exist 
at the moment of broadcasting.

My current brilliant idea to create a vector field with motion 
information is to take two (grayscale) frames and create a bunch of 
copies of each. These copies are then gaussian blurred with 
variously sized kernels. Starting with the blurriest (is that a 
word?) frame, we attempt to deform frame1 to match frame2. This is 
done by calculating the difference of the frames, and moving the 
point where the difference is biggest in the opposite direction of 
the gradient at that point in the frame. It's a sort of morph, in 
that point in the vicinity of this point also move along for a bit. 
All morphs are recorded in a cumulative vector field. Once frame1 
matches frame2, we have a motion field for the two frames. We then 
repeat the process with a sharper version of the two frames, all 
the way up to a very slightly blurred (to get rid of noise) version 
of the frames.

The reason for the multi-resolution approach is that we want to 
avoid getting stuck in local minima while morphing if there is a 
lot of difference between the frames (ie fast motion).

I'm working on an implementation of that now, and I hope to have 
some results still today, otherwise I'll probably finish it on 
monday.

Lourens
- -- 
GPG public key: http://home.student.utwente.nl/l.e.veen/lourens.key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD4DBQE9N/65vmNyqZHWDvURAt1mAJ4wwhMURilj0zdhVkk11I2bgOOEOgCXQgnU
qwLe7+yKVDXMjOmeBv3a8Q==
=DWe/
-----END PGP SIGNATURE-----


--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to '[email protected]'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.