Re[4]: [Firebird-devel] Computation problem in 1.5RC3

[Gabriel J. <gj...@co...>]

Hi Calin,

Monday, June 30, 2003, 12:48:43 PM, you wrote:

CIP> ----- Original Message ----- 
CIP> From: "Gabriel Juncu" 
>> So, can anybody tell me how can I get correct arithmetic in Firebird
>> on 18 digits without overflow errors?
>> 
CIP> Hmmm, yes.

CIP> Use numeric(18,0) as intermediate values then you only get a overflow 
CIP> if the result realy overflows.

CIP> Ex:
CIP> CREATE PROCEDURE W_NUMTST (Param1 numeric(18,4), param2 numeric(18,4))
CIP> RETURNS (
CIP>  Res NUMERIC(18,4))
CIP> AS
CIP>   declare variable n1 numeric(18,0);
CIP>   declare variable n2 numeric(18,0);
CIP>   declare variable n numeric(18,0);
CIP> begin
CIP>   n1 = param1*10000;
CIP>   n2 = param2*10000;
CIP>   n = n1*n2;
CIP>   res = n/10000;
CIP>   suspend;
CIP> end

You hurried a bit. My first example (and problem) was:

Param1 = 5655555;
Param2 = 32900;
Result = Param1 * Param2;

so (56555555 * 10000) * (32900 * 10000) gives 20 digits -> and the
result really overflows ;-)

That is exactly what Firebird does intenally, and again, if it is made
on 64 bits it gives numeric overflow (except this time is much easier
to understand it ;-)

All I am asking here is if the developers think that this is the
normal way that numeric multiply shoud behave, so I should abbandon
any hope of seeing this corrected (from my point of view) and start to
pollute my stored procedures with computing artifices so I can get
correct results for a simple multiply operation.

Thanks again for your answer,
Gabriel