From: David Pirkle <dpirkle@sy...>  20021017 21:25:28

I came across a bug in version 7.2 of Saxon, or at least an = inconsistency. The formatnumber function sometimes rounds up and = sometimes rounds down when the number is exactly halfway between the = round up and round down values. If you run the following xsl: <?xml version=3D'1.0' encoding=3D"ISO88591"?> <xsl:stylesheet=20 version=3D"2.0" xmlns:xsl=3D"http://www.w3.org/1999/XSL/Transform"; xmlns:saxon=3D"http://saxon.sf.net/"=20 extensionelementprefixes=3D"saxon" > <xsl:template match=3D'/'> formatnumber(227.55, "0.0") =3D <xsl:valueof = select=3D'formatnumber(227.55, "0.0")'/><br/> formatnumber(227.65, "0.0") =3D <xsl:valueof = select=3D'formatnumber(227.65, "0.0")'/> </xsl:template> </xsl:stylesheet> ... you get the following results: formatnumber(227.55, "0.0") =3D 227.6 formatnumber(227.65, "0.0") =3D 227.6 David Pirkle Symyx Technologies, Inc. 
From: Trevor Nash <tcn@me...>  20021018 09:59:18

Hi David, >I came across a bug in version 7.2 of Saxon, or at least an=20 >inconsistency. The formatnumber function sometimes rounds=20 >up and sometimes rounds down when the number is exactly=20 >halfway between the round up and round down values. =20 >If you run the following xsl: > >... you get the following results: > >formatnumber(227.55, "0.0") =3D 227.6 >formatnumber(227.65, "0.0") =3D 227.6 > I have not actually done the sums, but I suspect this is a consequence of XPath numbers being represented as floating point. In this system many fractional numbers cannot be represented exactly, because they end up as a recurring binary (just as you cannot represent 1/3 in decimal notation). So 227.55 might be 227.55000000001 and 227.65 > 227.64999999999999 So the numbers are not 'exactly halfway between'. If this matters to you, for example in financial calculations, then you have to use algorithms which avoid these rounding errors. Using integers is one strategy: 22755 and 22765 rounded to the nearest ten should give the result you expect. This may not be adequate if your calculations involve division though. This isn't specific to XSLT/XPath  the same problem crops up in many other programming languages. Regards, Trevor Nash Melvaig Software Engineering Limited voice: +44 (0) 1445 771 271=20 email: tcn@... web: http://www.melvaig.co.uk 
From: Michael Kay <michael.kay@nt...>  20021019 15:06:57

The formatnumber() function in XSLT 1.0 is defined in terms of various classes in JDK 1.1. Go to the JDK specification, and see if you can decipher what they say about rounding  it's not easy! But Saxon conforms almost by definition, since it uses the same JDK classes in its implementation. It's not at all unreasonable that numbers ending in .5 should sometimes round up and sometimes down. Firstly, the actual value is in binary and is likely to be slightly more that or less than the decimal value as written. Secondly, rounding algorithms often deliberately have rules giving .5 an equal probability of rounding up or down, to eliminate statistical bias when averaging a large set of numbers. I haven't yet modified the formatnumber() function in Saxon 7.x to implement the rules given in the XSLT 2.0 specification  but the rules were intended to give the same answer as the XSLT 1.0 spec, only without a normative reference to the JDK. Michael Kay > Original Message > From: saxonhelpadmin@... > [mailto:saxonhelpadmin@...] On Behalf Of > David Pirkle > Sent: 17 October 2002 22:25 > To: saxonhelp@... > Subject: [saxon] rounding inconsistency in formatnumber function > > > I came across a bug in version 7.2 of Saxon, or at least an > inconsistency. The formatnumber function sometimes rounds > up and sometimes rounds down when the number is exactly > halfway between the round up and round down values. If you > run the following xsl: > > <?xml version='1.0' encoding="ISO88591"?> > <xsl:stylesheet > version="2.0" > xmlns:xsl="http://www.w3.org/1999/XSL/Transform"; > xmlns:saxon="http://saxon.sf.net/"; > extensionelementprefixes="saxon" > > > <xsl:template match='/'> > formatnumber(227.55, "0.0") = <xsl:valueof > select='formatnumber(227.55, "0.0")'/><br/> > formatnumber(227.65, "0.0") = <xsl:valueof > select='formatnumber(227.65, "0.0")'/> > </xsl:template> > > </xsl:stylesheet> > > ... you get the following results: > > formatnumber(227.55, "0.0") = 227.6 > formatnumber(227.65, "0.0") = 227.6 > > David Pirkle > Symyx Technologies, Inc. > > > >  > This sf.net email is sponsored by: viaVerio will pay you up > to $1,000 for every account that you consolidate with us. > http://ad.doubleclick.net/clk;4749864;7604308;>; v? > http://www.viaverio.com/consolidator/osdn.cfm _______________________________________________ saxonhelp mailing list saxonhelp@... https://lists.sourceforge.net/lists/listinfo/saxonhelp 