Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#2658 Working presicion

None
closed
nobody
5
2013-11-06
2013-11-06
ReneSci
No

Why is Maxima replying with there inaccuracies:

(%i1) kB : 1.38x10^-23;
(%o1) 1.3800000000000001x10^-23

and

(%i2) Ng1ppmFactor : 1e-6;
(%o2) 9.9999999999999995x10^-7

instead of
1.38x10^-23
and
10x10^-7

Discussion

  • Raymond Toy
    Raymond Toy
    2013-11-06

    1.38*10^-23 cannot be represented exactly as a (binary) floating-point number. The actual internal representation is slightly different. The printer itself may introduce some roundoff. This is dependent on the underlying lisp implementation.

    The same happens for 1e-6.

    Based on the printed result, I'm guessing you're using gcl. cmucl will print 1.38e-23 and 1.0e-6 in both cases. If you are using gcl, file a bug with the gcl maintainer to improve the printer.

     
  • Robert Dodier
    Robert Dodier
    2013-11-06

    Not a bug, therefore closing this report.

     
  • Robert Dodier
    Robert Dodier
    2013-11-06

    • labels: Presicion --> floating point, not-a-bug
    • status: unread --> closed