Re: Truncate then average? Or average then truncate?

OK, imagine this. We have a noisy two lowest bits, so we're removing
the bottom three bits of the value and averaging four samples. If the
value is around 180 we get:

        10110101b
      + 10110100b
      + 10110110b
      + 10110101b
  ---------------
      1011010100b >> 2
  ---------------
      = 1011010b

But if the value is close to a power of two, e.g. 64, we get unstable values:

      1000000b
    + 0111111b
    + 1000001b
    + 0111111b
  ------------
     11111111b >> 2
  ------------
    = 0111111b

The noise in the LSBs ends up bits flipping bits high up in the number
in a way that truncating doesn't solve. Mr Bunnell is right, I should
probably look more into numerical filtering rather than bit tricks.
Duh.

- Robin Green.


On Fri, May 28, 2010 at 9:06 AM, Jeremiah Zanin
<[email protected]> wrote:
> Broken divider? I don't see how the noise bits can bleed into the other bits after dividing unless it's rounding up. Have you tried taking a power of two samples and shifting instead of dividing?
>

------------------------------------------------------------------------------

_______________________________________________
GDAlgorithms-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list