numpy-discussion

[Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Fernando P. <fpe...@gm...>]

I understand why this happens, but I wonder if it should be in any way
'fixed' (if that is even feasible without introducing other problems):

In [28]: x = 999999

In [29]: y = numpy.array([x])

In [30]: z = y[0]

In [31]: x==z
Out[31]: True

In [32]: x
Out[32]: 999999

In [33]: z
Out[33]: 999999

In [34]: x*x
Out[34]: 999998000001L

In [35]: z*z
Warning: overflow encountered in long_scalars
Out[35]: -729379967

I am sure it will be, to say the least, pretty surprising (and I can
imagine subtle bugs being caused by this).

Regards,

f

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Robert K. <rob...@gm...>]

Fernando Perez wrote:
> I understand why this happens, but I wonder if it should be in any way
> 'fixed' (if that is even feasible without introducing other problems):
> 
> In [28]: x = 999999
> 
> In [29]: y = numpy.array([x])
> 
> In [30]: z = y[0]
> 
> In [31]: x==z
> Out[31]: True
> 
> In [32]: x
> Out[32]: 999999
> 
> In [33]: z
> Out[33]: 999999
> 
> In [34]: x*x
> Out[34]: 999998000001L
> 
> In [35]: z*z
> Warning: overflow encountered in long_scalars
> Out[35]: -729379967
> 
> I am sure it will be, to say the least, pretty surprising (and I can
> imagine subtle bugs being caused by this).

I think we decided long ago that an int32 really is an array of 32-bit integers
and behaves like one. That's precisely why one uses int32 arrays rather than
object arrays. There are algorithms that do need the wraparound, and the Python
int behavior is always available through object arrays.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Fernando P. <fpe...@gm...>]

On 11/9/06, Robert Kern <rob...@gm...> wrote:
> Fernando Perez wrote:
> > I understand why this happens, but I wonder if it should be in any way
> > 'fixed' (if that is even feasible without introducing other problems):

[...]

> > I am sure it will be, to say the least, pretty surprising (and I can
> > imagine subtle bugs being caused by this).
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.

Mkay, fine by me.  Novus caveat emptor.

Cheers,

f

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Christopher B. <Chr...@no...>]

Robert Kern wrote:

> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one.

That would apply to y*y:

 >>> x = 999999
 >>> y = numpy.array([x])
 >>> x*x
999998000001L

So Python ints automatically convert to Python longs on overflow.

 >>> y*y
array([-729379967])

numpy int32 arrays wrap...

 >>> z = y[0]
 >>> type(z)
<type 'numpy.int32'>
 >>> z*z
Warning: overflow encountered in long_scalars
-729379967

int32 scalars wrap, but give a warning -- (why the warning here, but not 
with the array calculation?)

I'm a bit confused, because I thought that when you extracted a scalar 
from an array, you got regular python scalar for the datatypes that are 
supported. This made it clear that you always get a numpy Scalar, which, 
in a few situations, behaves differently than a seemingly equivalent 
Python scalar.

Have I got that right?

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Robert K. <rob...@gm...>]

Christopher Barker wrote:
> I'm a bit confused, because I thought that when you extracted a scalar 
> from an array, you got regular python scalar for the datatypes that are 
> supported. This made it clear that you always get a numpy Scalar, which, 
> in a few situations, behaves differently than a seemingly equivalent 
> Python scalar.

Part of the point of adding scalar types was to ensure that a[0], for example,
always exposes the array interface (.shape, .dtype, etc.) whether a.shape is
(10,) or (10, 10). This uniformity aids generic programming.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Tim H. <tim...@ie...>]

Robert Kern wrote:
> Fernando Perez wrote:
>   
>> I understand why this happens, but I wonder if it should be in any way
>> 'fixed' (if that is even feasible without introducing other problems):
>>
>> In [28]: x = 999999
>>
>> In [29]: y = numpy.array([x])
>>
>> In [30]: z = y[0]
>>
>> In [31]: x==z
>> Out[31]: True
>>
>> In [32]: x
>> Out[32]: 999999
>>
>> In [33]: z
>> Out[33]: 999999
>>
>> In [34]: x*x
>> Out[34]: 999998000001L
>>
>> In [35]: z*z
>> Warning: overflow encountered in long_scalars
>> Out[35]: -729379967
>>
>> I am sure it will be, to say the least, pretty surprising (and I can
>> imagine subtle bugs being caused by this).
>>     
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.
>
>   
Let me add that I can't imagine that the bugs will be all that subtle 
given that numpy now spits out a warning on overflow.
If you're really worried about this I suggest you crank up the error 
mode to make this an error - then you really won't be able to miss this 
kind of overflow.

-tim

Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs

[Fernando P. <fpe...@gm...>]

On 11/9/06, Tim Hochberg <tim...@ie...> wrote:

> Let me add that I can't imagine that the bugs will be all that subtle
> given that numpy now spits out a warning on overflow.
> If you're really worried about this I suggest you crank up the error
> mode to make this an error - then you really won't be able to miss this
> kind of overflow.

Not all computations are run interactively with a human in front of
the console to actually see the warnings go by.  If they get buried in
a log file on a cluster, there is a possibility you might miss them.

Cheers,

f