Menu

#3 Multiplication stops with two much decimal places

open
nobody
None
5
2013-04-02
2013-04-02
Christian
No

The recursive multiplication stops when there are to much positions after decimal separator and the number is to big. For example when multiplying 200000000000000000000000000000043200000 with 0,0000000000000000000000000000000009 with a scale of 100.

After debugging for a while, I discovered that the object, where the result is stored, at function to perform the recursive multiplication (prod) contains broken data, e.G. NaN instead of 0. And these NaNs couldn't be overwritten with 0.

Discussion


Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.