Re: transfrom dynamic range

> The inverse, gains 4.
                    ^^^^
    This is correct?

    Let me recapitulate:

    residual [9 bits] -> transform -> [15 bits] -> quantize -> [X bits] -> inv quantize -> [15 bits] -> int transform -> [9 bits]

    The standard says that input to IT is in range [-2^15...( 2^15)-1] ( as Indrajit said), and output in [-2^15...( 2^15)-33].
Then, the normalization of EQ 8-278 takes it back to 9 bits.

    Another thing is that inside quantization ( using the reference software approach to change divisions by right shifts) dynamic
range go up to nearly 32 bit because of the multiplication with 14 bit constants.  As i see it, this could be problematic for optimized
MMX code unless both integer multiplication/divisions are used.

    /AITOR
  ----- Original Message ----- 
  From: [email protected] 
  To: [email protected] ; [email protected] 
  Sent: Wednesday, April 02, 2003 12:56 PM
  Subject: Re: [Hdot264-devel] transfrom dynamic range



        The forward transfrom gaines 6 bits over the range of the input data ,
        The inverse, gains 4.

              <[email protected]>
              [Hdot264-devel] transfrom dynamic range 

              Hi there!,

                  Which is the real dynamic range of the values output by the forward transform?  Full 16 bit,
              or even in a worst case less than that?

                  /AITOR
             


------------------------------------------------------------------------





------------------------------------------------------------------------
       


------------------------------------------------------------------------------
  Walla! Mail, Get Your Private, Free E-mail from Walla! at:
  http://mail.walla.co.il