Menu

#20 double to unsigned long long precision error

2.0
pending
nobody
None
1
2013-02-26
2013-02-25
Anonymous
No

Hi, there is a bug with unsigned long long values. At least on my system where sizeof(unsigned long long) = sizeof(double) = 8

The problem is that internally you use double for all intergral types, which might cause
loss of precision for big unsigned long long numbers.

this fails on my system:

unsigned long long ullValue = 63660062001020978;
 double doubleValue = 63660062001020978;

 double unsignedlonglong_to_double = static_cast<double>(ullValue);
 unsigned long long double_to_unsignedlonglong = static_cast<unsigned long             long>(doubleValue);

qDebug() << "unsigned long long: " << ullValue;
qDebug() << "double: " << doubleValue;
qDebug() << "unsigned long long to double: " << unsignedlonglong_to_double;
qDebug() << "double to unsigned long long: " << double_to_unsignedlonglong;

output is:

unsigned long long: 63660062001020978 <-- compare this
double: 6.36601e+16
unsigned long long to double: 6.36601e+16
double to unsigned long long: 63660062001020976 <-- with this

cast from double <--> unsigned long long might cause problems both ways.

Discussion

  • Artyom Beilis

    Artyom Beilis - 2013-02-26
    • status: open --> pending
     
  • Artyom Beilis

    Artyom Beilis - 2013-02-26

    CppCMS represents numeric values as double internally so it wouldn't be possible to fix in without chaning an interface. So at this point I can't fix it.

    You must be aware of this limitations.

     
  • Artyom Beilis

    Artyom Beilis - 2013-02-26

    Ticket moved from /p/cppcms/bugs/94/

     
  • Artyom Beilis

    Artyom Beilis - 2013-02-26
    • milestone: 1.2 --> 2.0
     

Anonymous
Anonymous

Add attachments
Cancel





Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.