Re: [SL] Rounding Error or Human Error?

[Luke <sl...@li...>]

More on this:

If I change the USD currency precision to 6, from its former setting of 2, 
the numbers come out right.  However, I am then left with totals like 
31.860000.  Changing the defaults precision setting seems to do nothing.

Is there any way to have calculations be done on a precision above 2, 
without displaying values with a precision above 2?

Or, better yet, have a final rounding be done for totals to a precision of 
2?

A calculation precision and a display precision, perhaps.

The problem seems to result from a calculation of the discount based upon 
the sellprice, not upon the (sellprice * quantity).

precision: 2
How I thought it was being done: 2.95 * 12 - 10% = 31.86
How it is being done: (2.95 - 10%) * 12 = 31.80

Luke


On Tue, 1 Jan 2008, Luke wrote:

> Hello
> 
> I have an invoice which appears to be calculating the discount 
> incorrectly.
> 
> The line item cost is $2.95 (sellprice--the list price is $2.99, and I 
> lowered this on the invoice to compensate for another seeming error).
> 
> Setting the discount to 10, with a quantity of 12, yields a cost of 
> $31.80, while my calculator seems to think that it should equal $31.86.
> 
> SL 2.8.10; PGSQL 8.2.
> 
> Am I missing something obvious here?
> 
> Luke
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> sql-ledger-users mailing list
> sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sql-ledger-users
>