IMDCT accuracy

Rafael Avila de Espíndola <[email protected]>
Newsgroups gmane.comp.audio.mad.devel
Message-ID <[email protected]>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Testing the  Rob Leslie implementation of the Szu-Wei Lee algorithm I notice 
that his modification (multiplying the input of the DCTIV by 2 and dividing 
the output) generally improves the accuracy but, in some cases, may cause a 
overflow.

I have not found a .mp3 file that triggers a overflow, but the following data 
does. The present IMDCT implementation in libmad 0.15 return a valid result.

0x442D46F
0x217CC40
0x8E26853
0xF1888C6
0x921BF17
0xD282
0xF060CB8
0x3E82D68
0xBCA6124
0x1217700
0x9500F6D
0x3E9B4B6
0xC4B7675
0x6AE1645
0x56C266C
0x4463FD6
0x7EE5176
0xC408F90

Is there some evidence that no real mp3 may trigger an overflow?

Thanks for any help.
Rafael Espíndola
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/y2NVLlrfGJ8JUHwRAibvAJ9OtOIAoFi3UlKMqscFYUZIquCeGwCcCFLC
563s6VSqX5vnPxdTZNZAR/g=
=yIV7
-----END PGP SIGNATURE-----
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.