Menu

#335 Floating point errors on bit and integer operations

v1.1.x
fixed
critical (96)
v1.1.0
Bug Fix
2018-05-23
2018-05-18
Erik Hänel
No

If one does calculations like 0.01 * 100 and tries to apply bit operations on it afterwards, the bit operations will not work as intended, because 0.01 * 100 = 0.9999999999 (or similar). Casted to an int, this number is identical to 0.

Analysis:
The description already contains all necessary information. It can be solved by replacing the direct cast to int with a function, which will use rint(), if the number is quite close to the near int.

Implementation:
The fix was implemented as proposed by the analysis. Implementation tests were passed successfully.

Documentation:
The fix was documented in the changes log. Further documentation not needed.

Tests:
The fix was added to the automatic tests. No deviation detected. Bug fix implemented successfully.

Related

Tickets: #335
Tickets: #336

Discussion

  • Erik Hänel

    Erik Hänel - 2018-05-18
    • status: open --> accepted
     
  • Erik Hänel

    Erik Hänel - 2018-05-18
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1 +1,9 @@
     If one does calculations like `0.01 * 100` and tries to apply bit operations on it afterwards, the bit operations will not work as intended, because `0.01 * 100 = 0.9999999999` (or similar). Casted to an `int`, this number is identical to `0`.
    +
    +**Analysis:**
    +
    +**Implementation:**
    +
    +**Documentation:**
    +
    +**Tests:**
    
    • status: accepted --> analyzing
     
  • Erik Hänel

    Erik Hänel - 2018-05-19
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1,6 +1,7 @@
     If one does calculations like `0.01 * 100` and tries to apply bit operations on it afterwards, the bit operations will not work as intended, because `0.01 * 100 = 0.9999999999` (or similar). Casted to an `int`, this number is identical to `0`.
    
     **Analysis:**
    +The description already contains all necessary information. It can be solved by replacing the direct cast to `int` with a function, which will use `rint()`, if the number is quite close to the near `int`.
    
     **Implementation:**
    
    • status: analyzing --> implementing
     
  • Erik Hänel

    Erik Hänel - 2018-05-19
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -4,7 +4,9 @@
     The description already contains all necessary information. It can be solved by replacing the direct cast to `int` with a function, which will use `rint()`, if the number is quite close to the near `int`.
    
     **Implementation:**
    +The fix was implemented as proposed by the analysis. Implementation tests were passed successfully.
    
     **Documentation:**
    +The fix was documented in the changes log. Further documentation not needed.
    
     **Tests:**
    
    • status: implementing --> testing
     
  • Erik Hänel

    Erik Hänel - 2018-05-23
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -10,3 +10,4 @@
     The fix was documented in the changes log. Further documentation not needed.
    
     **Tests:**
    +The fix was added to the automatic tests. No deviation detected. Bug fix implemented successfully.
    
    • status: testing --> fixed
     

Anonymous
Anonymous

Add attachments
Cancel





MongoDB Logo MongoDB