Re: [Firebird-devel] Computation problem in 1.5RC3

[Gabriel J. <gj...@co...>]

Hi List,

Friday, June 27, 2003, 11:17:49 AM, I wrote:

GJ> Hi List!

GJ> I have a serious problem in computations in Firebird 1.5:

GJ> First my environment:

GJ> Firebird 1.5 Release Candidate 3 WI-T1.5.0.3481 Superserver on Windows XP Home Edition
GJ> Processor: Athlon XP 1700

GJ> I discovered the problem in a stored procedure but I can replicate the
GJ> problem in the following query:

GJ> select 5655555.0000 * 32900.0000 from RDB$DATABASE

GJ> Result:
GJ> Unsuccessful execution caused by system error that does not
GJ> preclude sccessful execution of subsequent statements.
GJ> Integer overflow. The result of an integer operation caused the most
GJ> significant bit of the result to carry.

GJ> The same result for queries:

GJ> select cast(5655555 as numeric(18,4)) * cast(32900 as numeric(18, 4)) from RDB$DATABASE
GJ> select cast(5655555.0000 * 32900.0000 as numeric(18, 4)) from RDB$DATABASE

GJ> But for queries:

GJ> select 5655555.000 * 32900.000 from RDB$DATABASE
GJ> select cast(5655555 as numeric(18,3)) * cast(32900 as numeric(18, 3)) from RDB$DATABASE

GJ> I get the result: 186,067,759,500.000032 !!! (obviously wrong)

Sorry that I misinformed you, but the above result is obtained in
IBExpert.
In isql the result is correct: 186,067,500.000000
Sorry once again!

Still I think that the bellow statement should work:

GJ> In the stored procedure where I noticed this behavior, I have 3 local
GJ> variables of type numeric(18,4) and I get the above error in a simple
GJ> multiply operation:

GJ> v2 = 5655555;
GJ> v3 = 32900;
GJ> v1 = v2 * v3;

GJ> What am I doing wrong? Or is it a bug in Firebird?

Thank you,
Gabriel Juncu