Hi,
Thanks for the info.
I took a look at the saxon source code, and it does seem to use =
DecimalFormat to do its work, and DecimalFormat uses halfeven rounding =
for formatting. I took a look at the xslt 2.0 spec to see what it =
specified for the behavior of formatnumber. It says that the rounding =
should follow the "XPath rounding procedure". I assume that this refers =
to the xpath round function, which is supposed to work as follows: =
"xf:round(x) produces the same result as xf:floor(x+0.5)". The example =
I posted is defintely inconsistent with this specification.
Regards,
David
Original Message
From: Clapham, Paul [mailto:pclapham@...]
Sent: Friday, October 18, 2002 8:18 AM
To: Trevor Nash; David Pirkle
Cc: saxonhelp@...
Subject: RE: [saxon] rounding inconsistency in formatnumber function
It is also possible that the rounding method being used here is the
"roundhalfeven" method in which numbers exactly halfway between two
possible roundings are rounded to the even one; the posted examples do =
that.
I have heard that this method is in fact what's used by some Java =
classes
that Saxon may use to implement formatnumber, but if that's the case it
isn't in the Java public documentation.
Regards
PC2
Original Message
From: Trevor Nash [mailto:tcn@...]
Sent: October 18, 2002 02:44
To: David Pirkle
Cc: saxonhelp@...
Subject: Re: [saxon] rounding inconsistency in formatnumber function
Hi David,
>I came across a bug in version 7.2 of Saxon, or at least an=20
>inconsistency. The formatnumber function sometimes rounds=20
>up and sometimes rounds down when the number is exactly=20
>halfway between the round up and round down values. =20
>If you run the following xsl:
>
>... you get the following results:
>
>formatnumber(227.55, "0.0") =3D 227.6
>formatnumber(227.65, "0.0") =3D 227.6
>
I have not actually done the sums, but I suspect this is a consequence
of XPath numbers being represented as floating point. In this system
many fractional numbers cannot be represented exactly, because they
end up as a recurring binary (just as you cannot represent 1/3 in
decimal notation).
So 227.55 might be 227.55000000001 and 227.65 > 227.64999999999999
So the numbers are not 'exactly halfway between'.
If this matters to you, for example in financial calculations, then
you have to use algorithms which avoid these rounding errors. Using
integers is one strategy: 22755 and 22765 rounded to the nearest ten
should give the result you expect. This may not be adequate if your
calculations involve division though.
This isn't specific to XSLT/XPath  the same problem crops up in many
other programming languages.
Regards,
Trevor Nash
Melvaig Software Engineering Limited
voice: +44 (0) 1445 771 271=20
email: tcn@... web: http://www.melvaig.co.uk
