Hello,

I have got problems with rounding errors in format-number(). This seems to
affect all versions > 6.5.5

Example:

1.234.567.890

1.234.567.890,1

1.234.567.890,1

1.234.567.890,12

1.234.567.890,12

1.234.567.890,12

1.234.567.890,12

1.234.567.890,12

1.234.567.890,120

1.234.567.890,12

1.234.567.890,1200

1.234.567.890,12

1.234.567.890,12000

1.234.567.890,1199999

1.234.567.890,1199999

1.234.567.890,11999989

1.234.567.890,11999989

1.234.567.890,119999886

1.234.567.890,119999886

1.234.567.890,1199998856

1.234.567.890,1199998856

1.234.567.890,1199998856

Regards, J.Mescheder

You haven't said how $testval was initialised, but I imagine you did something
like this:

in which case $testval will have a value that is the nearest xs:double value
to 1234567890.12. Because no xs:double is exactly equal to this number, the
nearest xs:double is probably something like 123456890.1199998856. The format-
number() function will then format this value to as many decimal places as you
request. So the rounding error is not coming from format-number(), it is
coming from the string-to-double conversion that happened when the variable
was initialized. If you need to avoid these problems, use xs:decimal rather
than xs:double.

Thanks for the quick reply!

Sorry, forgot to post the initialisation line:

1234567890.12

How should I alter the initialisation to avoid the problem?

Got it.

1234567890.12

works as expected.

Thanks for the hint!

I would write it as

It's generally better to use the select attribute on xsl:variable rather than
using child nodes, unless you actually want the value to be a tree structure.
The as="xs:decimal" ensures that the value is a decimal rather than a double.
In your code the value is actually a document node containing a text node;
when you supply that as input to format-number it is atomized, giving an
untypedAtomic value, and this is then converted to an xs:double.

Many thanks for your advice and explaination!