Re: Re: transfrom dynamic range

> So I think the correct chart should look something like that :
>
> residual [9 bits] -> transform -> [15 bits] -> quantize -> [X bits] ->
> inv quantize -> [11 bits] -> int transform -> [16 bits] -> downscaling -> [9 bits]
>

    Then, if the output from the IQ is 11 bits, why does the standard say the following?:

    "The bitstream shall not contain data that results in a value of wij that exceeds the range
    of integer values from -2^15 to 2^15-1, inclusive." ( just after EQ 8-267).

 ----- Original Message ----- 
  From: [email protected] 
  To: [email protected] ; [email protected] 
  Sent: Wednesday, April 02, 2003 3:22 PM
  Subject: Re: Re: [Hdot264-devel] transfrom dynamic range


        >   residual [9 bits] -> transform -> [15 bits] -> quantize -> [X bits] -> inv quantize -> [15 bits] -> int transform -> [9 bits]
        for this equation to be correct the inverse transform should be 
        0.25   0.2   0.25    0.1
        0.25   0.1  -0.25  -0.2
        0.25  -0.1  -0.25   0.2
        0.25  -0.2   0.25  -0.1
        which isnt the case :(
        and the f/i quant are not inverse of each other as well
        please note that the result of the itrans is downscaled by 6   



              <[email protected]>
              Re: [Hdot264-devel] transfrom dynamic range 
              > The inverse, gains 4.
                                  ^^^^
                  This is correct?

                  Let me recapitulate:

                  residual [9 bits] -> transform -> [15 bits] -> quantize -> [X bits] -> inv quantize -> [15 bits] -> int transform -> [9 bits]

                  The standard says that input to IT is in range [-2^15...( 2^15)-1] ( as Indrajit said), and output in [-2^15...( 2^15)-33].
              Then, the normalization of EQ 8-278 takes it back to 9 bits.

                  Another thing is that inside quantization ( using the reference software approach to change divisions by right shifts) dynamic
              range go up to nearly 32 bit because of the multiplication with 14 bit constants.  As i see it, this could be problematic for optimized
              MMX code unless both integer multiplication/divisions are used.

                  /AITOR
                ----- Original Message ----- 
                From: [email protected] 
                To: [email protected] ; [email protected] 
                Sent: Wednesday, April 02, 2003 12:56 PM
                Subject: Re: [Hdot264-devel] transfrom dynamic range



                      The forward transfrom gaines 6 bits over the range of the input data ,
                      The inverse, gains 4.

                            <[email protected]>
                            [Hdot264-devel] transfrom dynamic range 

                            Hi there!,

                                Which is the real dynamic range of the values output by the forward transform?  Full 16 bit,
                            or even in a worst case less than that?

                                /AITOR
                           


----------------------------------------------------------





----------------------------------------------------------
                     


----------------------------------------------------------------
                Walla! Mail, Get Your Private, Free E-mail from Walla! at:
                http://mail.walla.co.il
             

------------------------------------------------------------------------



------------------------------------------------------------------------
       


------------------------------------------------------------------------------
  Walla! Mail, Get Your Private, Free E-mail from Walla! at:
  http://mail.walla.co.il