Re: Advanced Motion Compensation for sub 500kbit DVD Video

not to pop any bubbles, but --

we went down a substantially similar path several
years ago (triangular
MV's, texture compression, feature matching, etc).  As
a model for generating images with a
small number of bits, it's very seductive.  One can
imagine encodings that
are extremely economical.

I think if you pursue this, you will find that the
central issue is how to
encode the material.  it proves to be surprisingly
difficult to generate
watchable 'motion fields' in this manner.  The central
problem seemed to
us that when you try to build a motion field (which is
essentially what you are doing with feature
recognition & triangulation), you have a global
optimization problem.  Any instability anywhere in the
frame can easily spread and make the other parts of
the frame look bad.  The practical result is that,
with every algorithm we came up with to create the
field from source material, the resulting compression
had a very wierd 'warbly' look to it, as if you were
looking at the image through an old piece of glass
with irregular thickness, or a funhouse mirror.

In stark contrast, the standard technique of
rectilinear tesselation (typically 4x4 to 16x16
blocks), where each motion vector is local to the
block, followed by DCT type transforms, which are also
strictly local, has the advantage that the problem now
becomes a purely local optimization: for each block in
the image, find the best combination of motion
prediction and residual correction, using RD
optimization theory, or a hacked bunch of thresholds,
whatever.  In any case, the result you get can have
artifacts (as we all know), but they tend to be
strictly local artifacts; a mistake coding one block
has no negative effect on any other blocks (that's not
100% true in some of the more sophisticated,
context-sensitive algorithms such as H.26L, but it's
close enough).  Errors do not propagate; and the
problem is conveniently broken down into modular,
disconnected pieces.

None of this is meant to discourage the vibrant Tarkin
community from its ambitious goals; I in fact believe
an algorithm of this sort could be made to work.  I
just hope by pointing out some existing results that I
can save someone a little time, or give some hints
about how to proceed.

-dbm

--- Dustin Stuart <[email protected]> wrote:
> Hi everyone,
> 
> This is my first post to the Tarkin list after a
> quick skimming
> of archived aritcles.
> 
> My brilliant idea is a method of compressing very
> high quality video. 
> Firstly, 'keyframes' are identified in a video
> sequence and compressed at 
> high resolution (1024x432) with a wavelet coder at
> around 20:1 / 25:1. 
> (30kb) Then all subsequent frames are analyzed to
> find key points and areas 
> (eg. the edge around a character or the features of
> his face) and these 
> points are joined to make a series of triangles.
> After that, the 
> correspponding areas in other frames are found (eg.
> a characters left eye in 
> one frame is mapped to the left eye in the next).
> When all triangles are 
> mapped and found the encoder warps each triangle
> using an interpolation 
> alogrithm and compares it to the subsequent
> triangles. If the results are 
> similar enough, an MV is created to defing each
> point's movement and speed 
> over subsequent frames. If the triangles do not
> match enough, the encoder 
> simply reencodes the triangle with a wavelet
> algorithm. The process 
> continues for all keyframes in the video.
> 
> This method presents huge advantages for decoding
> because the idea of a 
> video 'frame' can be dropped completely! A video
> recorded at 24 
> frames/second can be played at 60 or 70 on a decoder
> fast enough. But the 
> complexity of my method (and the numerous
> stretchings that would take place) 
> may prohibit further development.
> 
> But I have done the math. If the keyframe consumes
> only 30kb, there should 
> be more than 100kb for reencoded triangles and MVs
> if keyframes occur every 
> couple of seconds...
> 
> Look forward to your feedback...
> Dustin Stuart
> 
>
_________________________________________________________________
> Chat with friends online, try MSN Messenger:
> http://messenger.msn.com
> 
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to
> '[email protected]'
> containing only the word 'unsubscribe' in the body. 
> No subject is needed.
> Unsubscribe messages sent to the list will be
ignored/filtered.


__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to '[email protected]'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.