Re: [Audacity-devel] Floating point values > 1.0 ??

[Dominic M. <do...@mi...>]

You shouldn't be getting any floating point values > 1.0.

Are you getting values which are just barely over 1.0,
or are you getting values like 1.1 or higher?

If the problem happens only with MP3, then maybe we're
not using the right scaling factor in ImportMP3.cpp.
The function to look at is:

inline float scale(mad_fixed_t sample)
{
    return (float) (sample / (float) (1L << MAD_F_FRACBITS));
}

Try doing an assertion here to see if the result is
outside of the range.

Actually it's very possible that we're doing the
conversion correctly, but that the MP3 file is actually
encoding samples out of range.  This is possible because
slight errors in the frequency domain could cause the
maximum sample value to change by a couple of %.

In that case, the solution is to clip inside this
function.

- Dominic

Vince Busam wrote:
> I've been doing more testing with my AVC Effect and have been getting some
> program exceptions.  I've tracked it down to getting floating point sample
> values that are > 1.0.  I thought they are supposed to be from -1 to +1.  Am
> I correct in this?
> 
> I took a wav file from the Microsoft Speech SDK (seven.wav) and am seeing
> this problem.  I'm using "floatSample" with WaveTrack::Get(...).
> 
> If I change to use "int16Sample" I don't see any value < -32768 or > 32767.
> (I do have some -32768 values.)
> 
> My last update of Audacity code looks like Aug. 8, so if this is a recently
> fixed problem I should get the latest.  But since I haven't seen any mention
> of this in the group's emails I suspect that my download of code is still
> satisfactory for my purposes.
> 
> Any ideas of what is going on here?  Is this a MP3 decode problem?  Any
> suggestions on how to track down the problem?
> 
> Thanks,
> Vince
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by: OSDN - Tired of that same old
> cell phone?  Get a new here for FREE!
> https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390
> _______________________________________________
> Audacity-devel mailing list
> Aud...@li...
> https://lists.sourceforge.net/lists/listinfo/audacity-devel