Re: IMDCT accuracy

On Dec 1, 2003, at 7:50 AM, Rafael Avila de Espíndola wrote:
> Testing the  Rob Leslie implementation of the Szu-Wei Lee algorithm I 
> notice
> that his modification (multiplying the input of the DCTIV by 2 and 
> dividing
> the output) generally improves the accuracy but, in some cases, may 
> cause a
> overflow.
>
> I have not found a .mp3 file that triggers a overflow, but the 
> following data
> does. The present IMDCT implementation in libmad 0.15 return a valid 
> result.
>
> 0x442D46F
> 0x217CC40
> 0x8E26853
> 0xF1888C6
> 0x921BF17
> 0xD282
> 0xF060CB8
> 0x3E82D68
> 0xBCA6124
> 0x1217700
> 0x9500F6D
> 0x3E9B4B6
> 0xC4B7675
> 0x6AE1645
> 0x56C266C
> 0x4463FD6
> 0x7EE5176
> 0xC408F90
>
> Is there some evidence that no real mp3 may trigger an overflow?

Only that in order to overflow, I think some of the transformed sample 
values would have to be at least 12dB above full scale. For example, 
when the above data is transformed, many samples above full scale 
result, including two samples more than 14dB above (-5.200011).

Having said that, I would be very interested to learn of a legitimate 
Layer III frame that does cause an overflow.

-- 
Rob Leslie
[email protected]