Screenshot instructions:
Windows
Mac
Red Hat Linux
Ubuntu
Click URL instructions:
Rightclick on ad, choose "Copy Link", then paste here →
(This may not be possible with some types of ads)
From: Clapham, Paul <pclapham@co...>  20021018 15:12:35

It is also possible that the rounding method being used here is the "roundhalfeven" method in which numbers exactly halfway between two possible roundings are rounded to the even one; the posted examples do that. I have heard that this method is in fact what's used by some Java classes that Saxon may use to implement formatnumber, but if that's the case it isn't in the Java public documentation. Regards PC2 Original Message From: Trevor Nash [mailto:tcn@...] Sent: October 18, 2002 02:44 To: David Pirkle Cc: saxonhelp@... Subject: Re: [saxon] rounding inconsistency in formatnumber function Hi David, >I came across a bug in version 7.2 of Saxon, or at least an >inconsistency. The formatnumber function sometimes rounds >up and sometimes rounds down when the number is exactly >halfway between the round up and round down values. >If you run the following xsl: > >... you get the following results: > >formatnumber(227.55, "0.0") = 227.6 >formatnumber(227.65, "0.0") = 227.6 > I have not actually done the sums, but I suspect this is a consequence of XPath numbers being represented as floating point. In this system many fractional numbers cannot be represented exactly, because they end up as a recurring binary (just as you cannot represent 1/3 in decimal notation). So 227.55 might be 227.55000000001 and 227.65 > 227.64999999999999 So the numbers are not 'exactly halfway between'. If this matters to you, for example in financial calculations, then you have to use algorithms which avoid these rounding errors. Using integers is one strategy: 22755 and 22765 rounded to the nearest ten should give the result you expect. This may not be adequate if your calculations involve division though. This isn't specific to XSLT/XPath  the same problem crops up in many other programming languages. Regards, Trevor Nash Melvaig Software Engineering Limited voice: +44 (0) 1445 771 271 email: tcn@... web: http://www.melvaig.co.uk 
From: David Pirkle <dpirkle@sy...>  20021018 17:37:29

Hi, Thanks for the info. I took a look at the saxon source code, and it does seem to use = DecimalFormat to do its work, and DecimalFormat uses halfeven rounding = for formatting. I took a look at the xslt 2.0 spec to see what it = specified for the behavior of formatnumber. It says that the rounding = should follow the "XPath rounding procedure". I assume that this refers = to the xpath round function, which is supposed to work as follows: = "xf:round(x) produces the same result as xf:floor(x+0.5)". The example = I posted is defintely inconsistent with this specification. Regards, David Original Message From: Clapham, Paul [mailto:pclapham@...] Sent: Friday, October 18, 2002 8:18 AM To: Trevor Nash; David Pirkle Cc: saxonhelp@... Subject: RE: [saxon] rounding inconsistency in formatnumber function It is also possible that the rounding method being used here is the "roundhalfeven" method in which numbers exactly halfway between two possible roundings are rounded to the even one; the posted examples do = that. I have heard that this method is in fact what's used by some Java = classes that Saxon may use to implement formatnumber, but if that's the case it isn't in the Java public documentation. Regards PC2 Original Message From: Trevor Nash [mailto:tcn@...] Sent: October 18, 2002 02:44 To: David Pirkle Cc: saxonhelp@... Subject: Re: [saxon] rounding inconsistency in formatnumber function Hi David, >I came across a bug in version 7.2 of Saxon, or at least an=20 >inconsistency. The formatnumber function sometimes rounds=20 >up and sometimes rounds down when the number is exactly=20 >halfway between the round up and round down values. =20 >If you run the following xsl: > >... you get the following results: > >formatnumber(227.55, "0.0") =3D 227.6 >formatnumber(227.65, "0.0") =3D 227.6 > I have not actually done the sums, but I suspect this is a consequence of XPath numbers being represented as floating point. In this system many fractional numbers cannot be represented exactly, because they end up as a recurring binary (just as you cannot represent 1/3 in decimal notation). So 227.55 might be 227.55000000001 and 227.65 > 227.64999999999999 So the numbers are not 'exactly halfway between'. If this matters to you, for example in financial calculations, then you have to use algorithms which avoid these rounding errors. Using integers is one strategy: 22755 and 22765 rounded to the nearest ten should give the result you expect. This may not be adequate if your calculations involve division though. This isn't specific to XSLT/XPath  the same problem crops up in many other programming languages. Regards, Trevor Nash Melvaig Software Engineering Limited voice: +44 (0) 1445 771 271=20 email: tcn@... web: http://www.melvaig.co.uk 
From: Michael Kay <michael.h.kay@nt...>  20021019 19:11:51

> I took a look at the saxon source code, and it does seem to > use DecimalFormat to do its work, and DecimalFormat uses > halfeven rounding for formatting. I took a look at the xslt > 2.0 spec to see what it specified for the behavior of > formatnumber. It says that the rounding should follow the > "XPath rounding procedure". I assume that this refers to the > xpath round function, which is supposed to work as follows: > "xf:round(x) produces the same result as xf:floor(x+0.5)". > The example I posted is defintely inconsistent with this > specification. Yes, I noticed this too, and have noted it as an issue. I also need to check if there's an inadvertant difference between the XPath 1.0 and XPath 2.0 definitions of round(). Michael Kay Software AG home: Michael.H.Kay@... work: Michael.Kay@... 
Sign up for the SourceForge newsletter:
No, thanks