yet another compression idea based on human 3D-viewing

Hello,

I have read the last posts concerning symbol recognizing and and an enormous
data reduction as a possible result.

As many people have said this before symbol recognizing isn't much more than 
an impressive idea. Although many attemps where and are made into this 
direction none has succeded despite from very special tasks (for example OCR, 
or easy structured rooms where robots find some objects) that need in most 
times supercomputers.

When I read them I remembered an article in german science magazine "Spektrum
der Wissenschaft" (www.wissenschaft-online.de/spektrum related to "Scientific
American") that was also pointing out the "symbol recognition" and the 
"related points" problem and pointed out a very interesting solution to some 
of the aspects (April/2002, page 10 - 16, title: "Stereosehen und das 
zyklopische Auge").

The article was about a new theory of human 3D-viewing and how nature made it
possible that our brain was able to make out of two images from the two eyes a
new one with perfect overlaping of all elements and 3 dimensional view.

The answer is: We don't need symbol recognition for that it is much more
easier:

Beginning with an old fact that we view our environement out of the 
perspective of a virtual third eye (called "zyklopisches Auge" in german) in 
the middle between our two eyes (just try it, close one eye look around open 
it and close the other and compare the perspectives with that from both eyes 
open or look at babies that always take an object in between of their eyes 
instead in front of one eye) there was build up a theory that explained our 
3D-viewing and the middle eye perspective.

But what has this all to do with 2D-Films (I hear someone of you thinking ;-)
):

Imagine a scenery with some objects in foreground and a far away background.
Then take a camera and make a film during moving the camera in the same hight
with constanst speed from the left to the right. The objects in foreground
will have another position in an image after the movement but the far away
ones are still at the same place in images from the beginning and the end. The
same effect can be seen when you close your left eye and the you right one and
watching your fingers in front of the wall. The fingers seem to jump. So a
movement and the two human perspectives are the same despite from the fact
that you have only the first and the last image instead of a plenty of images
if you move the camera. Untill here everything is obvious.

Now imagine all the images the camera has made for the film, think them as
photographs and put them one onto the other in the order they where made. In
the end you have a little tower of photographs. Now lets take one point of the
foreground and one of the background in a horizontal line of the top photo and
take a knife an make a cut through our tower just in the line where our two
points are from the top to the bottom. Now lets take the related points of our
two ones in every image (with our human brain it is no problem here ;-) )and
make on the cut edge with a felt tip or what ever take a line through all
related points from the foregrund point and one line through the related
points of our background point. Now you have two lines. (just read it again if
you can't imagine, if you have the idea one time it is simple). The foreground
point line is diagonal and the background point line is vertical.

Now lets make this for all points and you will recive an image where only
diagonal and vertical lines are crossing.

So now one idea is to take a "cut image" of an image tower and search for our
lines. That way we have reduced 3D viewing and image movements recognition to
a texture recognition. We now need and algorithm to search for lines in our
cut image (each contrast border creates a line) and dependent from how
diagonal our lines are we can determine how far our points are. Now make this
for every scanline (like cutting bread) in our photo tower. And If we have one
time the distance for every point we also knwo much better which points belong
together and which not (so our symbol recognition would be much easier now).
But that's only a sideffect. 

Now there is my idea: If you have all the lines detected you can for example
take the first an the last image and combine all related points (now you know
by our TODO texture algorithm which points are related) with vectors. All
images between can bee thrown away. They only contain information we now can
reconstruct from our two images and our vectors. This idea can be improved if
you only take vectors that are very different and interpolate all the similar
ones in between (that would be a primtive object recognition).

But that's not the way human eye recongizes our nice textures. Since it only
has two images it has to do it another way (our lines would be hard to find):

Let us take one scanline from our images (our cut image) and combine wildly 
all
points from the scanline of the first photo with the ones from last one and
let us calculate a distance. Most of the distances will be wrong, since the
points do not belong together. But our images have a inner structure (we only
don't know that at the moment). So some of the point relations will be right
and cause our object has a certain size there will be other points in the 
neighbourhood with the same (or very similar) calculated distance. So there 
are some coherent calculations and lost of wrong ones. But the wrong ones 
aren't coherent, they are just statistically. Human neurons that are coherent 
are now stopping the statistically ones and make our statistical noise 
smaller and smaller. So we could make an algorithm that finds our coherent 
ones and throws our wrong ones away. That way we know how far object are and 
by the way the perspective of that brain created mix image is exactly the one 
from our virtual middle eye.

My second idea concerning our 2D-Films is, that you take two following images 
out of the image tower and follwo for each followning pair the above 
described steps. That way we get again our related points vectors through our 
image tower that can also be improved (concerning similar vectors) like in 
the first idea.

For now this only works for linear horizontal movements but I'm convinced that 
can be solved by a little bit more complexity of our algorithms. I presume 
that the second idea has much smaller problems with that cause you take two 
subsequent images and between them you can make linear movements. You only 
don't know in which direction. And that way you can aproximate any complex 
movement of the camera. In the first idea our linear lines will be curves in 
3D-space, I don't know if this is a much greater problem.

And by the way, the 3D stereo viewing with coherent points has already been 
solved in hardware (for example as PC card) by "3D-Image-Processing" in 
Augsburg, Germany.

So I hope I didn't explain it to complicated and that these ideas can help you 
on building up a great video codec. 

What do you think of it?

--
happy coding, Daniel Arnold 
 

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to '[email protected]'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.