Menu

fractional digit precision

Chris
2013-07-21
2013-09-03
  • Chris

    Chris - 2013-07-21

    Hi everyone,

    First of all, many thanks to Pavel for this very useful library.

    I have some questions concerning the fractional digit precision. If I want to set a double cell value like '12345.6789' then this results in a value of '12345.7' in the xlsx file. Setting the numberOfDigitsAfterPoint (result e.g. '12345.70000') does not help, it is not a displaying issue but the precision is really missing in the xml file. I tried to track the problem down and found out that the precision is probably lost when streaming the doubles out, the default precision std::cout.precision(); seems to be only 5 in my case. Further I found out that in ximplexlsxwriter the precision is changed with the std::setprecision command, but only for date/time. I then tried to increase the general cout precision with std::cout.precision(10); but with no success, the numbers are the same also after this change.

    Questions:
    1. Is there a way to increase the decimals precision also for 'normal' numbers, not only date/time? A precision of five is not sufficient.
    2. If not, what do you think will be the easiest way to implement it, adding setprecision statements like for date/time?
    3. Is this small precision normal, that is, do you also have such a small presicion by default or I am the only one with this problem?

    I compiled the library r0.19 using MinGW on Windows.

    Thanks for your help,
    Chris

     
  • Chris

    Chris - 2013-07-22

    As a work around, I added a template specialization for double values in xmlwriter.h which enables full precision:

    template <> inline XmlStream& XmlStream::operator<< <double> (const double& value) {
        if (stateTagName == state)
            tagName << value;
        s << std::setprecision(std::numeric_limits<double>::digits10) << value;
        return *this;
    }
    

    Please let me know if there is a better way I don't see to achieve my goal.

    Chris

     
  • Pavel Akimov

    Pavel Akimov - 2013-09-03

    Hello, Chris!
    I had seen your topic even before I commited 0.20 version. But I did`t include any concerned this problem features because of lack of time.

    The way you suggested seems to be fine.
    But I withdrew from this project a little and so I need time to revise and fix the most important issues (including this one) and make a normal release.

    Thanks for your concern.

     

Log in to post a comment.