yorick

Tracker: Bugs

5 Casting Problems - ID: 2826222

Last Update: Settings changed ( dhmunro )

Details:

I apologize for not asking about this in the forum first but the forums
seem not to be working, and I am fairly certain that this is a bug.I also
do not know what this bug is actually a problem with: the typing or the
casting algorithm, (though they are probably closely related functions due
to Yorick's design).

When doing arithmetic on numbers that have been cast then casting them
again, the number does not cast properly. Below is a basic example of
this.
> x = 11.1
> ix = long(x)
> fx = x - ix
> ix
11
> fx
.1
> long(fx*10)
0

Where the output should be the same as:
> long(.1*10)
1

Submitted:

Ben Batha ( bbatha ) - 2009-07-23 21:21

Priority:

Status:

Closed

Resolution:

Invalid

Assigned:

Nobody/Anonymous

Category:

None

Group:

None

Visibility:

Public

Comments ( 2 )

Date: 2009-09-19 16:35 Sender: dhmunro This is a feature of floating point arithmetic, not a bug. You can try it directly in C code to prove it to yourself. The problem is that 11.1 cannot be precisely represented in binary floating point format, because 1/10 (or actually 1/5) has a non-terminating binary expansion. The number 11.0 is exactly representable. When you subtract 11.0 from 11.1, you lose a few bits of precision at the end of the expansion for 0.1, causing it to round downward. Thus (11.1-11.0)<0.1 (which you can verify by typing it directly to yorick). There are many references to this effect; try googling "floating point precision loss".
Date: 2009-07-23 22:47 Sender: bbatha Upon further inspection, this problem seems to be caused by a discrepancy between how Yorick internally handles floats, and how it displays them to the user. Looking at it, the write and implicit print function under stand that .1 held in a binary float is in fact .1, whereas the casting functions, or arithmetic functions can not see the difference. There maybe a way around this because the write functions can see the difference between the internal float and the actual value and then display it correctly. This check may be able to be applied to the cast or arithmetic.

Date: 2009-09-19 16:35
Sender: dhmunro

This is a feature of floating point arithmetic, not a bug. You can try it
directly in C code to prove it to yourself. The problem is that 11.1
cannot be precisely represented in binary floating point format, because
1/10 (or actually 1/5) has a non-terminating binary expansion. The number
11.0 is exactly representable. When you subtract 11.0 from 11.1, you lose
a few bits of precision at the end of the expansion for 0.1, causing it to
round downward. Thus (11.1-11.0)<0.1 (which you can verify by typing it
directly to yorick). There are many references to this effect; try
googling "floating point precision loss".

Date: 2009-07-23 22:47
Sender: bbatha

Upon further inspection, this problem seems to be caused by a discrepancy
between how Yorick internally handles floats, and how it displays them to
the user. Looking at it, the write and implicit print function under stand
that .1 held in a binary float is in fact .1, whereas the casting
functions, or arithmetic functions can not see the difference. There maybe
a way around this because the write functions can see the difference
between the internal float and the actual value and then display it
correctly. This check may be able to be applied to the cast or arithmetic.

Attached File

Changes ( 4 )

Field	Old Value	Date	By
status_id	Open	2009-09-19 16:35	dhmunro
resolution_id	None	2009-09-19 16:35	dhmunro
allow_comments	1	2009-09-19 16:35	dhmunro
close_date	-	2009-09-19 16:35	dhmunro

yorick

Tracker: Bugs

Comments ( 2 )

Attached File

No Files Currently Attached

Changes ( 4 )

About SourceForge

Find Software

Develop Software

Community

Help