problem about precision in transform and quantization
hell yu <[email protected]> Fri, 18 Apr 2003 17:30:55 +0800 (CST)
| Newsgroups | gmane.comp.video.h264.devel |
|---|---|
| Message-ID | <[email protected]> |
In document VCEG-O25, Tabele 2 is given as following:----------------------------------------------------------------------------Luma and chroma transform adn quantization--------------------------------------------------------------------------- | Input bits | Output bits-----------------------------|----------------------|----------------------Forword transform | 9, 11, 13 | 15, 16, 16Quantization | 15, 16, 16 | 10, 12, 14Dequantization | 10, 12, 14 | 15, 17, 19Inverse transform | 15, 17, 19 | 9, 11, 13----------------------------------------------------------------------------In quantization, input data is 15-bits wide. Becasue the max quantMat 13107 is 14-bits wide, the output data should be 12 bits wide after a 17 right shifting. But the it is "10" in the table. Is my understanding wrong?Thanks for your help. --------------------------------- Do You Yahoo!? "禄垄没茫虏毛CCTV-2露禄掳陆驴碌梅虏茅拢潞露炉隆垄陋篓隆垄茅篓拢卢陋碌掳庐茫虏禄