Re: [mpg123-devel] fixed point decoders

[Taihei M. <tm...@ma...>]

On 2009/05/31, at 4:06, Taihei Monma wrote:

> fl15.bit is still weird... I should look into more.

Gah, I made a stupid mistake... The layer2 decoder is correct now, and  
all tests pass limited accuracy :)

==== Layer 1 ====
--> 16 bit signed integer output
fl1.bit:        RMS=9.153515e-05 (LIMITED) maxdiff=3.705621e-04 (FAIL)
fl2.bit:        RMS=9.362248e-05 (LIMITED) maxdiff=4.243851e-04 (FAIL)
fl3.bit:        RMS=9.387823e-05 (LIMITED) maxdiff=3.730059e-04 (FAIL)
fl4.bit:        RMS=7.200389e-05 (LIMITED) maxdiff=3.411770e-04 (FAIL)
fl5.bit:        RMS=1.051527e-04 (LIMITED) maxdiff=4.037619e-04 (FAIL)
fl6.bit:        RMS=1.180426e-04 (LIMITED) maxdiff=4.869699e-04 (FAIL)
fl7.bit:        RMS=8.529281e-05 (LIMITED) maxdiff=3.802776e-04 (FAIL)
fl8.bit:        RMS=9.816843e-05 (LIMITED) maxdiff=4.596114e-04 (FAIL)

==== Layer 2 ====
--> 16 bit signed integer output
fl10.bit:       RMS=1.146784e-04 (LIMITED) maxdiff=4.718304e-04 (FAIL)
fl11.bit:       RMS=1.131210e-04 (LIMITED) maxdiff=4.780293e-04 (FAIL)
fl12.bit:       RMS=1.119295e-04 (LIMITED) maxdiff=4.792809e-04 (FAIL)
fl13.bit:       RMS=1.054379e-04 (LIMITED) maxdiff=3.585219e-04 (FAIL)
fl14.bit:       RMS=9.804458e-05 (LIMITED) maxdiff=3.951192e-04 (FAIL)
fl15.bit:       RMS=1.097113e-04 (LIMITED) maxdiff=4.411340e-04 (FAIL)
fl16.bit:       RMS=1.202716e-04 (LIMITED) maxdiff=8.831620e-04 (FAIL)

==== Layer 3 ====
--> 16 bit signed integer output
compl.bit:      RMS=1.303472e-04 (LIMITED) maxdiff=1.008749e-03 (FAIL)

Thanks,
Taihei Monma