From: Matthias Baas <matthias.baas@gm...>  20091004 18:46:49

Hi Simon, Simon Bunker wrote: > I have also noticed quite a significant precision issue with 3delight. > This shader (compiled to a sdl): > > class test ( > uniform float test_uniform_float = 23.6; > [...] > > returns this result from slparams: > >>>> from cgkit import slparams >>>> slparams.slparams("test.sdl") > [('surface', 'test', [('', 'uniform', 'float', None, 'test_uniform_float', None, > 23.600000381469727), > [...] > > To complicate things this is using 32 bit Python, 32 bit 3delight on > Windows Vista 64 bit. That doesn't matter in this case, floats/doubles are the same on 32bit and 64bit. > Is this a cgkit or a 3delight problem? It's neither a problem in cgkit, nor 3delight. It's just that 3delight represents the numbers using a float type which cannot represent the number 23.6 exactly. Apparently, 23.600000381469727 is the closest number to 23.6 that a float can represent. The problem is that we are used to writing numbers in the decimal system but floats use a basis of 2 which means a short decimal number doesn't fit neatly into a float and may require much more bits to be represented accurately. You can see for yourself by compiling the following C program: #include <stdio.h> int main() { float f = 23.6f; printf("%1.15f\n", f); return 0; } or in Python: >>> import ctypes >>> f = ctypes.c_float(23.6) >>> f c_float(23.600000381469727) The output is 23.600000381469727 (just what you got above) even though the number is supposed to be 23.6. I know, it's easy to forget that working with floats only yields approximate maths, in particular as printing numbers often just prints so many significant digits and you actually see a rounded number (which then often matches what you were typing in in the first place). Cheers,  Matthias  