Menu

Upgrade to double precision floating point

2015-03-23
2018-04-28
  • Tero Kontkanen

    Tero Kontkanen - 2015-03-23

    I checked the source codes and found out that calculations are made with single precision floating point (32bit float type) instead of double precision (64 bit, double type). While it might first look like 32 bits is enough, it isn't. The rounding error of floats accumulate during filter processing and may cause significant source of distortion in some filter sets.

    There is also no reason for single precision floats as CPU does double precision without any performance hit. Secondly, modifying code to use double instead of float should be almost as easy as "search & replace".

    BTW, thanks for the great project! I have been looking for such app for years.

    BR, Tero

     

    Last edit: Tero Kontkanen 2015-03-23
  • Ron Kuper

    Ron Kuper - 2018-04-28

    Was there a follow up / update on this?

     

Log in to post a comment.

MongoDB Logo MongoDB