#373 Format bif does not round some small numbers


Running under xp home sp3 and v3.5 & 3.6 of Regina
the format bif converts some small numbers to exponential
notation rather than rounding them
the follow snippet demonstrates the problem

say format(2.499999999,1,0); say
say format(2.500000000,1,0); say
say format(0.000000001,1,0) ' error'; say
say format(0.00000001,1,0) ' error'; say
say format(0.0000001,1,0) ' error'; say
say format(0.0000009,1,0) ' error'; say
say format(0.000001,1,0); say


  • Mark Hessling

    Mark Hessling - 2012-08-27

    I've run each of the examples through the ANSI BIF tester: http://www.rexxla.org/cgi/ANSI/ansibifs.rex and Regina matches the results except for the first one, where Regina returns a value of 2 whereas ANSI returns 3. All tests done with the default NUMERIC DIGITS of 9.
    Now the first result is interesting; I believe Regina's answer of 2 to be correct. I will have to check the ANSI standard and the BIF tester code.

  • Mark Hessling

    Mark Hessling - 2012-08-27
    • assigned_to: nobody --> rexx
  • Mark Hessling

    Mark Hessling - 2012-08-28

    The ANSI BIF tester is correct and the answer to the first example is indeed 3. Regina does produce an answer of 3 if the OPTION; STRICT_ANSI is set.


Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

No, thanks