#90 Missing decimal digits in float var

closed
NXC (53)
5
2011-01-20
2011-01-20
Matthias
No

This simple test program:

task main()
{
float f1, f2;

f1 = atof\("0.123456789"\);
f2 = 0.123456789;

ClearScreen\(\);
NumOut\(0, LCD\_LINE1, f1\);
NumOut\(0, LCD\_LINE2, f2\);
Wait\(3000\);;

}

gives this output on the bricks' screen:

0.1235
0.1235

So it seems that only 4 of the decimal digits are used. The NXC help says that a float variable is of 32-bit IEEE 754 single precision, which should provide about 7 decimal digits. But -- where are those decimal digits in my example above?

Thanks!

Discussion

  • John Hansen

    John Hansen - 2011-01-20

    This is working as designed. When the float is converted to a string via the firmware's numtostr opcode it formats the value with fewer decimal places than may actually exist in the value. It used to do 2 digits. Recent firmware changes have extended it to 3 digits. In the enhanced NBC/NXC firmware I modified the numtostr opcode to use up to 4 digits. You can tell that the float variable contains the supported number of digits of precision by multiplying the value by 10 or 100 or 1000 and see that additional non-zero digits are in the variable. Or you could store a value with 4 zeros before the first non-zero significant digit and see what happen when you multiply a large non-zero number. If the value was actually zero (as shown via NumOut) then you would get zero no matter what you multiply it with. Or you could write the value to a file using Write to see if it is zero or not. If you need to draw the value to the LCD and show more decimal places than 4 you can use FormatNum with a standard C format string that will produce more than 4 decimal places. FormatNum requires the enhanced NBC/NXC firmware.

     
  • Matthias

    Matthias - 2011-01-20
    • status: open --> closed
     
  • Matthias

    Matthias - 2011-01-20

    thanks, that makes sense.
    I have tested it and it works as you have described.

    cheers!