Re: [ojAlgo-user] Precision

[Andy T. <A.G...@le...>]

Thanks Anders :-)

In some of my code I provide a MathContext for calculations and a MathContext for returns to some methods and use these to guarantee that what is returned is precise to a scale.

I have a square root function which is like that. I have a use for it in precise distance calculations.

I was hoping to use other functions from jAlgo like that too, but at least I can use MathContext.DECIMAL128 to improve the precision in some things...

I'll have a look at the RationalNumber class...

Bye for now,

Andy


-----Original Message-----
From: Anders Peterson [mailto:an...@op...] 
Sent: 14 April 2010 19:36
To: oja...@li...
Subject: Re: [ojAlgo-user] Precision

You should look at precision rather than scale.

If you look at the BigFunction source code you'll se that most of the implementations either delegate to BigDecimal or to PrimitiveFunction. When it delegates to PrimitiveFunction the precision is given by double otherwise MathContext.DECIMAL128 is used. You cannot control this.

Did you notice the RationalNumber class?

http://ojalgo.org/generated/org/ojalgo/scalar/RationalNumber.html

/Anders


On 14 apr 2010, at 17.50, Andy Turner wrote:

> With functions how do I know what precision is returned and can a specific precision be requested?
> 
> Running the following code for example shows me that by default rootThousand0 is returned with a scale of 48 where as the rootTwo is returned with a scale of 52. Setting the scale of what is input seems not to have an effect.
> 
>        BigDecimal rootThousand0 = BigFunction.POW.invoke(
>                BigMath.THOUSAND,
>                BigMath.HALF);
>        System.out.println("" + BigMath.THOUSAND + " precision " + BigMath.THOUSAND.scale());
>        System.out.println("" + BigMath.HALF + " precision " + BigMath.HALF.scale());
>        System.out.println("" + rootThousand0 + " precision " + rootThousand0.scale());
>        System.out.println("Math.pow(1000.0d,0.5d) " + Math.pow(1000.0d,0.5d));
> 
>        BigDecimal thousand = BigMath.THOUSAND.setScale(100);
>        BigDecimal half = BigMath.HALF.setScale(100);
>        BigDecimal rootThousand1 = BigFunction.POW.invoke(
>                thousand,
>                half);
>        System.out.println("" + rootThousand1 + " precision " + rootThousand1.scale());
> 
>        BigDecimal rootTwo = BigFunction.POW.invoke(
>                BigMath.TWO,
>                BigMath.HALF);
>        System.out.println("" + rootTwo + " precision " + rootTwo.scale());
>        System.out.println("Math.pow(2.0d,0.5d) " + Math.pow(2.0d,0.5d));
> 
> 
> TIA for any help or guidance.
> 
> Andy
> http://www.geog.leeds.ac.uk/people/a.turner/
> 
> 
> 
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 


------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
ojAlgo-user mailing list
ojA...@li...
https://lists.sourceforge.net/lists/listinfo/ojalgo-user