Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project!

## saxon-help

 [saxon] rounding inconsistency in format-number function From: David Pirkle - 2002-10-17 21:25:28 ```I came across a bug in version 7.2 of Saxon, or at least an = inconsistency. The format-number function sometimes rounds up and = sometimes rounds down when the number is exactly half-way between the = round up and round down values. If you run the following xsl: format-number(227.55, "0.0") =3D
format-number(227.65, "0.0") =3D
... you get the following results: format-number(227.55, "0.0") =3D 227.6 format-number(227.65, "0.0") =3D 227.6 David Pirkle Symyx Technologies, Inc. ```
 Re: [saxon] rounding inconsistency in format-number function From: Trevor Nash - 2002-10-18 09:59:18 ```Hi David, >I came across a bug in version 7.2 of Saxon, or at least an=20 >inconsistency. The format-number function sometimes rounds=20 >up and sometimes rounds down when the number is exactly=20 >half-way between the round up and round down values. =20 >If you run the following xsl: > >... you get the following results: > >format-number(227.55, "0.0") =3D 227.6 >format-number(227.65, "0.0") =3D 227.6 > I have not actually done the sums, but I suspect this is a consequence of XPath numbers being represented as floating point. In this system many fractional numbers cannot be represented exactly, because they end up as a recurring binary (just as you cannot represent 1/3 in decimal notation). So 227.55 might be 227.55000000001 and 227.65 -> 227.64999999999999 So the numbers are not 'exactly half-way between'. If this matters to you, for example in financial calculations, then you have to use algorithms which avoid these rounding errors. Using integers is one strategy: 22755 and 22765 rounded to the nearest ten should give the result you expect. This may not be adequate if your calculations involve division though. This isn't specific to XSLT/XPath - the same problem crops up in many other programming languages. Regards, Trevor Nash Melvaig Software Engineering Limited voice: +44 (0) 1445 771 271=20 email: tcn@... web: http://www.melvaig.co.uk ```
 RE: [saxon] rounding inconsistency in format-number function From: Michael Kay - 2002-10-19 15:06:57 ```The format-number() function in XSLT 1.0 is defined in terms of various classes in JDK 1.1. Go to the JDK specification, and see if you can decipher what they say about rounding - it's not easy! But Saxon conforms almost by definition, since it uses the same JDK classes in its implementation. It's not at all unreasonable that numbers ending in .5 should sometimes round up and sometimes down. Firstly, the actual value is in binary and is likely to be slightly more that or less than the decimal value as written. Secondly, rounding algorithms often deliberately have rules giving .5 an equal probability of rounding up or down, to eliminate statistical bias when averaging a large set of numbers. I haven't yet modified the format-number() function in Saxon 7.x to implement the rules given in the XSLT 2.0 specification - but the rules were intended to give the same answer as the XSLT 1.0 spec, only without a normative reference to the JDK. Michael Kay > -----Original Message----- > From: saxon-help-admin@... > [mailto:saxon-help-admin@...] On Behalf Of > David Pirkle > Sent: 17 October 2002 22:25 > To: saxon-help@... > Subject: [saxon] rounding inconsistency in format-number function > > > I came across a bug in version 7.2 of Saxon, or at least an > inconsistency. The format-number function sometimes rounds > up and sometimes rounds down when the number is exactly > half-way between the round up and round down values. If you > run the following xsl: > > > version="2.0" > xmlns:xsl="http://www.w3.org/1999/XSL/Transform"; > xmlns:saxon="http://saxon.sf.net/"; > extension-element-prefixes="saxon" > > > > format-number(227.55, "0.0") = select='format-number(227.55, "0.0")'/>
> format-number(227.65, "0.0") = select='format-number(227.65, "0.0")'/> >
> >
> > ... you get the following results: > > format-number(227.55, "0.0") = 227.6 > format-number(227.65, "0.0") = 227.6 > > David Pirkle > Symyx Technologies, Inc. > > > > ------------------------------------------------------- > This sf.net email is sponsored by: viaVerio will pay you up > to \$1,000 for every account that you consolidate with us. > http://ad.doubleclick.net/clk;4749864;7604308;>; v? > http://www.viaverio.com/consolidator/osdn.cfm _______________________________________________ saxon-help mailing list saxon-help@... https://lists.sourceforge.net/lists/listinfo/saxon-help ```