Re: Advanced Motion Compensation for sub 500kbit DVD Video

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 19 July 2002 16:47, dan miller wrote:
> not to pop any bubbles, but --

Don't worry. Poppable bubbles are no good anyway, and sharing 
experience only save work and time. There's no point in 
re-inventing the wheel (although there seems to be a point in 
patenting the wheel, but that's a wholly different issue).

> we went down a substantially similar path several
> years ago (triangular
> MV's, texture compression, feature matching, etc).  As
> a model for generating images with a
> small number of bits, it's very seductive.  One can
> imagine encodings that
> are extremely economical.

Yep.

> I think if you pursue this, you will find that the
> central issue is how to
> encode the material.  it proves to be surprisingly
> difficult to generate
> watchable 'motion fields' in this manner.  The central
> problem seemed to
> us that when you try to build a motion field (which is
> essentially what you are doing with feature
> recognition & triangulation), you have a global
> optimization problem.  Any instability anywhere in the
> frame can easily spread and make the other parts of
> the frame look bad.  The practical result is that,
> with every algorithm we came up with to create the
> field from source material, the resulting compression
> had a very wierd 'warbly' look to it, as if you were
> looking at the image through an old piece of glass
> with irregular thickness, or a funhouse mirror.

Hmm, interesting. What did the motionfields look like, were they 
correct in the sense that the pixels in one frame moved the way 
they did in the movie? I can see this going wrong if the wrong 
pixels are matched, but not if the matching is correct.

Wait a second, I can see it going wrong. If the surface the triangle 
is mapped to isn't really flat, the linear interpolation across the 
traingle is going to be incorrect. You'd have to have some sort of 
displacement mapping for the texture instead of only the motion 
information from the traingles vertices. I think :-). Comments?

> In stark contrast, the standard technique of
> rectilinear tesselation (typically 4x4 to 16x16
> blocks), where each motion vector is local to the
> block, followed by DCT type transforms, which are also
> strictly local, has the advantage that the problem now
> becomes a purely local optimization: for each block in
> the image, find the best combination of motion
> prediction and residual correction, using RD
> optimization theory, or a hacked bunch of thresholds,
> whatever.  In any case, the result you get can have
> artifacts (as we all know), but they tend to be
> strictly local artifacts; a mistake coding one block
> has no negative effect on any other blocks (that's not
> 100% true in some of the more sophisticated,
> context-sensitive algorithms such as H.26L, but it's
> close enough).  Errors do not propagate; and the
> problem is conveniently broken down into modular,
> disconnected pieces.

I see. But it's rather hard to extrapolate motion vectors if they're 
jumping all over the place because there happened to be a better 
match somewhere else. Which potentially costs bits. Thanks for the 
warning anyway, I hadn't considered local vs. global issues yet.

> None of this is meant to discourage the vibrant Tarkin
> community from its ambitious goals; I in fact believe
> an algorithm of this sort could be made to work.  I
> just hope by pointing out some existing results that I
> can save someone a little time, or give some hints
> about how to proceed.

Vibrant? That may be a bit too optimistic :-). The way I see it is 
that with VP3 soon available, there's not really a 'market' for 
Tarkin unless it's substantially better than VP3, and other current 
codecs (someone once quipped "a movie on a zip-disk" and that's 
what I think Tarkin should aim for). I'm not quite sure where VP3 
stands performance-wise, and from what I gather from the web it's 
not quite up to par with DivX, but it's a lot cheaper for sure, and 
with disk space in abundance the size difference only really 
matters in online distribution. There is OpenDivX, but that's not 
Free Software and also has patent issues, being an MPEG-4 encoder. 
So that's where something slightly better than VP3 would come in 
handy, but otherwise I don't think it would be that useful.

Tarkin at the moment has been described by Monty as a research 
project, and that's what it is. I feel the goal should be to create 
a codec that is substantially better than anything currently 
available, which means we'll have to try new things (which we have 
to anyway to avoid patent problems). If something doesn't work, we 
simply have to try something else. Any help with figuring out what 
does and what doesn't work is very much appreciated, so keep 'em 
coming :-).

Lourens
- -- 
GPG public key: http://home.student.utwente.nl/l.e.veen/lourens.key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9ODUPvmNyqZHWDvURAgwxAJ9gn6zQ1ndNHP1+dbGUCfBZihNvkQCeNdeP
ehLEsO9lYyc1ucbWd+qanyU=
=6IPP
-----END PGP SIGNATURE-----


--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to '[email protected]'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.