Menu

#15 Floating point rounding failure in Apl

open
nobody
5
2002-02-06
2002-02-06
Beau Webber
No

When using Apl interpretor Apl.68000 in Stonx,
there is a mistake in the rounding for display
puropses of
floating point numbers.
0.001 + 0 1 2 3 10 100
returns :
0.001 12.111 13.111 14.111 121.111 1211.111

On the Atari ST, STE, TT it correctly returns
0.001 1.001 2.001 3.001 10.001 100.001

The internal representation is correct, as adding floor
to the
command-line correctly returns :
0 1 2 3 10 100

I and Marinos had a look at this some years ago but
failed
to sort it. I managed (by switching off the carry bit
in an instruction)
to further break rounding so numbers came out like
1.99999998,
but there is another printing routine where I could fix
this for
display purposes.

I still use Apl.68000 on a TT at home and on STonX at
work
as my preferred interractive data manipulation tool
(with Maple
and Tim Budd's Apl2c compiler). I use it for NMR and
Scattering
experiments, producing postscript directly for Latex
papers.
So a fix would be a great help, as this also screws up
the internal
representation of floating point numbers in programs.

If you can look at this, I can pass a copy of Apl.68000
for debugging purposes.
cheers, beau webber

Discussion


Log in to post a comment.

MongoDB Logo MongoDB