Re: Projective image transformations

Sorry for the delay -- I have only once-a-day email access at the moment.
I hope this is still relevant.

First, beware of feature creep.  You're really getting into 3D territory and
image-processing territory now.  Both are littered with wheels waiting to be
reinvented.

On Thu, Sep 12, 2002 at 05:06:44PM -0700, Keith Packard wrote:
| 
| Sampling theory says we use a filter to resample the original source 
| pixels into the virtual source pixels.  ...

My understanding is that resampling really means "reconstruct (a continuous
representation of the signal), filter, then sample."  Most efficient algorithms
blur the distinctions (so to speak), but it's still worthwhile to keep them in
mind.  For example, in graphics practice, the reconstruction step usually has
more impact on the quality of the result than the filtering step when
magnifying; the reverse is usually true when minifying.

|                                    ...  Current hardware says you get two 
| choices for filtering; nearest neighbor and bilinear interpolation.  

There are others.  There are anisotropic variants for minification, and bicubic
reconstruction is sometimes used for magnification.  Hard to tell where this
technology is going from a point of view outside the IHVs, but I'd bet we'll
see more options.

|                                                        ...  The projective
| transformation will make that particular filter quite an adventure;

Current 3D hardware generally supports projective tranformations of texture
coordinates before texel lookup.  (It's needed to avoid artifacts when
geometric primitives are broken into triangles.  Simple perspective-correct
interpolation of texture coordinates from the vertices doesn't yield the right
result.)  You may need a 4x4 matrix (like the OpenGL texture matrix) for this,
though.

| The final question about filtering is how to handle the edge pixels. There
| are several traditional answers -- reflecting the image across the border,
| filling the exterior with a constant color.  ...

OpenGL has several ways to handle edge pixels.  There's the "repeat" mode, in
which the texture repeats but doesn't reflect.  Then there are the "clamp"
modes, which boil down to clamping to the edge of the texture image, clamping
to the border of the texture image, or clamping to a mix of both.  The border
may either be a 1-pixel-wide "frame" image around the texture image, or a solid
color.

Clamp-to-edge is pretty obvious (and is often what people use).
Clamp-to-border is helpful when you expect the meaningful part of a texture to
be smaller than the primitive on which it's drawn, and (as you mentioned) you
might well want the excess pixels to be transparent.  Clamping to a mix is used
when available texture memory is much smaller than the size of the texture
image, and you must use multipass techniques to draw a piece of the texture
image per pass.  This latter case is rarely supported in hardware these days.

|                  ....  OpenGL provides mipmapping to "fix" bilinear 
| interpolation at the expense of a lot of memory.  ...

The full mipmap stack costs only 1/3 as much memory as the base texture.  If
you're doing a lot of minification, it's usually considered a good tradeoff.

|                                              ...  I'm avoiding that 
| because I think our images are more dynamic than GL textures.

OpenGL has automatic mipmap generation support these days, largely because
people are using dynamic textures heavily (video and environment maps).

| Yeah, using a box filter always sucks.  Why is it that hardware never 
| bothers to provide anything better?

Current consumer hardware does, both for full-scene antialiasing and for
anisotropic filtering of textures.  Current high-end hardware can do a lot
better, so maybe we'll see that filter down to consumer hardware before long.

Allen