Re: Filtering

You guessed right. The loss of precision is in the texture units. Unfortunately, 8 bit components are filtered to 8 bit results (even though they show up as floating point values in the shader). This is true for nvidia gpus for sure and probably all other gpus.

-mike
  ----- Original Message ----- 
  From: Stefan Sandberg 
  To: Game Development Algorithms 
  Sent: Friday, October 01, 2010 1:45 AM
  Subject: Re: [Algorithms] Filtering


  Assuming you're after precision, what's wrong with doing it manually? :)
  If performance is what you're after, and you're working on textures as they were intended(ie, game textures or video or something like that, not 'data'), you could separate contrast & color separately, keeping high contrast resolution, and downsampled color, and 
  you'd save both bandwidth and instr.
  If you simply want to know 'why', I'm guessing loss of precision in the tex units?
  You've already ruled out shader precision from your own manual filtering, so doesn't leave much else, imo..
  Other than manipulating the data you're working on, which is the only thing you -can- change I guess, I cant really see a solution,
  but far greater minds linger here than mine, so hold on for what I assume will be a lengthy description of floating point math as 
  it is implemented in modern gpu's :)






  On Fri, Oct 1, 2010 at 9:57 AM, Andreas Brinck <[email protected]> wrote:

    Hi,

    I have a texture in which I use the R, G and B channel to store a
    value in the [0, 1] range with very high precision. The value is
    extracted like this in the (Cg) shader:

    float
    extractValue(float2 pos) {
       float4 temp = tex2D(buffer, pos);
       return (temp.x * 16711680.0 + temp.y * 65280.0 + temp.z * 255.0) *
    (1.0 / 16777215.0);
    }

    I now want to sample this value with bilinear filtering but when I do
    this I don't get a correct result. If I do the filtering manually like
    this:

    float
    sampleValue(float2 pos) {
           float2 ipos = floor(pos);
           float2 fracs = pos - ipos;
           float d0 = extractValue(ipos);
           float d1 = extractValue(ipos + float2(1, 0));
           float d2 = extractValue(ipos + float2(0, 1));
           float d3 = extractValue(ipos + float2(1, 1));
           return lerp(lerp(d0, d1, fracs.x), lerp(d2, d3, fracs.x), fracs.y);
    }

    everything works as expected. The values in the buffer can be seen as
    a linear combination of three constants:

    value = (C0 * r + C1 * g + C2 * b)

    If we use the built in texture filtering we should get the following
    if we sample somewhere between two texels: {r0, g0, b0} and {r1, g1,
    b1}. For simplicity we just look at filtering along one axis:

    filtered value = lerp(r0, r1, t) * C0 + lerp(g0, g1, t) * C1 +
    lerp(b0, b1, t) * C2;

    Doing the filtering manually:

    filtered value = lerp(r0 * C0 + b0 * C1 + g0 * C2, r1 * C0 + g1 * C1 +
    b1 * C2, t)  =
                      = (r0 * C0 + b0 * C1 + g0 * C2) * (1 - t) + (r1 *
    C0 + g1 * C1 + b1 * C2) * t =
                      = (r0 * C0) * (1 - t) + (r1 * C0) * t + ... =
                      = lerp(r0, r1, t) * C0 + ...

    So in the world of non floating point numbers these two should be
    equivalent right?

    My theory is that the error is caused by an unfortunate order of
    floating point operations. I've tried variations like:

    (temp.x * (16711680.0 / 16777215.0) + temp.y * (65280.0/16777215.0) +
    temp.z * (255.0/16777215.0))

    and

    (((temp.x * 256.0 + temp.y) * 256.0 + temp.z) * 255.0) * (1.0 / 16777215.0)

    but all exhibit the same problem. What do you think; is it possible to
    solve this problem?

    Regards Andreas

    ------------------------------------------------------------------------------
    Start uncovering the many advantages of virtual appliances
    and start using them to simplify application deployment and
    accelerate your shift to cloud computing.
    http://p.sf.net/sfu/novell-sfdev2dev
    _______________________________________________
    GDAlgorithms-list mailing list
    [email protected]
    https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
    Archives:
    http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list





------------------------------------------------------------------------------


  ------------------------------------------------------------------------------
  Start uncovering the many advantages of virtual appliances
  and start using them to simplify application deployment and
  accelerate your shift to cloud computing.
  http://p.sf.net/sfu/novell-sfdev2dev


------------------------------------------------------------------------------


  _______________________________________________
  GDAlgorithms-list mailing list
  [email protected]
  https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
  Archives:
  http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev

_______________________________________________
GDAlgorithms-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list