PYTHON: numpy.std - different results for float and double
Status: Beta
Brought to you by:
m_schellens
Menu
▾
▴
#424 PYTHON: numpy.std - different results for float and double
GDL> print, python('numpy', 'std', [1,2,3,4,5,6.])
1.7078251
GDL> print, python('numpy', 'std', [1,2,3,4,5,6d])
1.1547005
and sometimes...:
GDL> print, python('numpy', 'std', [1,2,3,4,5,6d])
2.1622551
GDL> print, python('numpy', 'std', [1,2,3,4,5,6d])
Warning: overflow encountered in multiply
inf
works in 0.9.8