numpy-discussion

Re: [Numpy-discussion] Memory errors

[Francesc A. <fa...@ca...>]

A Divendres 06 Octubre 2006 15:30, Vikalpa Jetly va escriure:
> avoid the memory error. Related question is that I need to test for
> multiple conditions on the same array and set values to 1 or 0. I see that
> statements like b[a>200 or a<50] =3D 1 do not work. So is the way to do t=
his
> simply to run a separate statement in the form b[condition]=3D 1 for each
> test?

=46or this, you have to use the binary operators instead of logical ones. F=
or=20
example:

In [43]: a=3Dnumpy.arange(10)

In [44]: b=3Dnumpy.arange(10)

In [45]: b[(a < 2) | (a > 8)] =3D 1

In [46]: b
Out[46]: array([1, 1, 2, 3, 4, 5, 6, 7, 8, 1])

[be sure to use parentesizes appropriately]

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Memory errors

[Robert K. <rob...@gm...>]

If you are going to start a new topic, please start a new thread as well, 
instead of replying to a message. Thanks.

Vikalpa Jetly wrote:
> I am reading a very large array (~9000,11000) of 1 byte image values. I need
> to change values in the array that meet a certain condition so I am running
> something like:
> 
> b = numpy.where(a>200,0,1)
> 
> to create a new array with the changed values. However, I get a
> "MemoryError" everytime I try this. I have over 3gb of RAM on my machine
> (most of which is available). The process runs fine on smaller datasets. Is
> there a maximum array size that numpy handles? Any alternatives/workarounds?

There is no predefined limit on the array size, just your memory.

However, note that what you are doing here is creating an int32 (or int64 if you 
are on a 64-bit machine) array since you are using Python integers in your 
where() function. You can save quite a bit of memory by using the array from 
(a>200) and simply casting it to the uint8 type. That way, you only have two 
arrays in memory at any time, a and b.

   b = (a > 200).astype(numpy.uint8)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco