[Hardmad-devel] Re: [mad-dev] IMDCT accuracy

[Rob L. <ro...@ma...>]

On Dec 1, 2003, at 7:50 AM, Rafael Avila de Esp=EDndola wrote:
> Testing the  Rob Leslie implementation of the Szu-Wei Lee algorithm I=20=

> notice
> that his modification (multiplying the input of the DCTIV by 2 and=20
> dividing
> the output) generally improves the accuracy but, in some cases, may=20
> cause a
> overflow.
>
> I have not found a .mp3 file that triggers a overflow, but the=20
> following data
> does. The present IMDCT implementation in libmad 0.15 return a valid=20=

> result.
>
> 0x442D46F
> 0x217CC40
> 0x8E26853
> 0xF1888C6
> 0x921BF17
> 0xD282
> 0xF060CB8
> 0x3E82D68
> 0xBCA6124
> 0x1217700
> 0x9500F6D
> 0x3E9B4B6
> 0xC4B7675
> 0x6AE1645
> 0x56C266C
> 0x4463FD6
> 0x7EE5176
> 0xC408F90
>
> Is there some evidence that no real mp3 may trigger an overflow?

Only that in order to overflow, I think some of the transformed sample=20=

values would have to be at least 12dB above full scale. For example,=20
when the above data is transformed, many samples above full scale=20
result, including two samples more than 14dB above (-5.200011).

Having said that, I would be very interested to learn of a legitimate=20
Layer III frame that does cause an overflow.

--=20
Rob Leslie
ro...@ma...