Re: problem about precision in transform and quantization
"Aitor Garay" <[email protected]> Tue, 22 Apr 2003 10:07:16 +0200
| Newsgroups | gmane.comp.video.h264.devel |
|---|---|
| Message-ID | <000b01c308a6$311096f0$bcb31fac@HAJC0062> |
VCEG-025 is superseded by JVT-B038. Even the final standard is not strictly in sync with JVT-B038.
/AITOR
----- Original Message -----
From: hell yu
To: [email protected]
Sent: Friday, April 18, 2003 11:30 AM
Subject: [Hdot264-devel] problem about precision in transform and quantization
In document VCEG-O25, Tabele 2 is given as following:
----------------------------------------------------------------------------
Luma and chroma transform adn quantization
---------------------------------------------------------------------------
| Input bits | Output bits
-----------------------------|----------------------|----------------------
Forword transform | 9, 11, 13 | 15, 16, 16
Quantization | 15, 16, 16 | 10, 12, 14
Dequantization | 10, 12, 14 | 15, 17, 19
Inverse transform | 15, 17, 19 | 9, 11, 13
----------------------------------------------------------------------------
In quantization, input data is 15-bits wide. Becasue the max quantMat 13107 is 14-bits wide, the output data should be 12 bits wide after a 17 right shifting. But the it is "10" in the table.
Is my understanding wrong?
Thanks for your help.
------------------------------------------------------------------------------
Do You Yahoo!?
"雅虎邀你参与CCTV-2对话节目调查：移动、联通、灵通，要说爱你不容易