Re: [OctDev] Test failures due to tolerance in fftfilt.m

[Daniel J S. <dan...@ie...>]

On 09/06/2012 02:59 PM, Daniel J Sebald wrote:
> I'll toss this one to Ed and Rik, since we were just talking about
> precision issues for svds test failures...
>
> I checked the current state of tests and found this failure:
>
>>>>>> processing
>>>>>> /usr/local/src/octave/octave/octave/scripts/signal/fftfilt.m
> ***** test

There is a bit more to this, and I've put a patch on Savannah:

https://savannah.gnu.org/bugs/index.php?37297

The routine will round the output if the inputs are integers and will 
truncate the imaginary component if both inputs are real.  That seems 
fair, I suppose.  (I do wonder though if there should be an option to 
remove this behavior because some might not want such a thing.  Any 
thoughts maintainers or OctDev?)  I've extended that concept to account 
for the other cases of real*imaginary, imaginary*real, and 
imaginary*imaginary.  I don't see why only the real*real case should be 
done...all or nothing, as I see it.  These conditions now have tests, 
and there are a couple more tests for tolerance on the 
imaginary/imaginary scenario, as well as the complex/complex scenario.

By making the integerization (rounding) test more stringent, I uncovered 
a bug whereby only the first element of the output single row vector was 
rounded.

Dan