matplotlib-users

[Matplotlib-users] memory problem

[Humufr <hu...@ya...>]

                    Hi,

on a pc with linux and 512M of RAM, I have a problem of memory when I'm 
using matplotlib.

from pylab import *
imshow(zeros((2000,2000)))
show()

is working but:

from pylab import *
imshow(zeros((4000,4000)))
show()

is not.


That slow down the computer, it close to be freeze and I obtain this 
error (the two error message for show() and savefig('test.png') are at 
the bottom of the mail. (matshow give exactly the same result)

It's a big problem, for me at least, because I'm working with some image 
with 4000x4000 pixels.

Thanks,

       N.

Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/local/lib/python2.4/lib-tk/Tkinter.py", line 1345, in __call__
    return self.func(*args)
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_tkagg.py", 
line 140, in resize
    self.show()
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_tkagg.py", 
line 143, in draw
    FigureCanvasAgg.draw(self)
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py", 
line 312, in draw
    self.figure.draw(renderer)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py", 
line 395, in draw
    for a in self.axes: a.draw(renderer)
  File 
"/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/axes.py", 
line 1339, in draw
    im.draw(renderer)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py", 
line 182, in draw
    im = self.make_image(isUpper)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py", 
line 112, in make_image
    x = self.to_rgba(self._A, self._alpha)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/cm.py", line 
418, in to_rgba
    return self.cmap(x, alpha)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/colors.py", 
line 526, in __call__
    rgba = zeros(xa.shape+(4,), Float)
  File 
"/usr/local/lib/python2.4/site-packages/numarray/numarraycore.py", line 
1407, in zeros
    retarr = NumArray(shape=shape, type=type)
MemoryError: Couldn't allocate requested memory


I did the same with savefig

Traceback (most recent call last):
  File "test.py", line 12, in ?
    savefig('test.png')
  File 
"/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/pylab.py", 
line 719, in savefig
    return fig.savefig(*args, **kwargs)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py", 
line 512, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_tkagg.py", 
line 161, in print_figure
    agg.print_figure(filename, dpi, facecolor, edgecolor, orientation)
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py", 
line 375, in print_figure
    self.draw()
  File 
"/usr/local/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py", 
line 312, in draw
    self.figure.draw(renderer)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py", 
line 395, in draw
    for a in self.axes: a.draw(renderer)
  File 
"/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/axes.py", 
line 1339, in draw
    im.draw(renderer)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py", 
line 182, in draw
    im = self.make_image(isUpper)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py", 
line 112, in make_image
    x = self.to_rgba(self._A, self._alpha)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/cm.py", line 
418, in to_rgba
    return self.cmap(x, alpha)
  File "/usr/local/lib/python2.4/site-packages/matplotlib/colors.py", 
line 526, in __call__
    rgba = zeros(xa.shape+(4,), Float)
  File 
"/usr/local/lib/python2.4/site-packages/numarray/numarraycore.py", line 
1407, in zeros
    retarr = NumArray(shape=shape, type=type)
MemoryError: Couldn't allocate requested memory

[Matplotlib-users] memory problem

[Marco F. <fu...@la...>]

Hi, I have a memory problem.

I'm using: matplotlib-0.85, python 2.3.5, gtk 2.8.6 and  pygtk 2.8.2.

I create a FigureCanvasGTKAgg with a subplot and I add one Line2D to this 
subplot.

Then I add this FigureCanvas in a gtk.Window

When I destroy the gtk.Window (by the gtk method destroy),
the memory seem to be not released.

Even if I clear the subplot before destroy the gtk.window only a part of 
memory is released.

Am I missing something obvious?

Can somebody help me?

Thanks
	marco

Re: [Matplotlib-users] memory problem

[John H. <jdh...@ac...>]

>>>>> "Humufr" == Humufr  <hu...@ya...> writes:

    Humufr>                     Hi, on a pc with linux and 512M of
    Humufr> RAM, I have a problem of memory when I'm using matplotlib.

    Humufr> from pylab import * imshow(zeros((2000,2000))) show()

    Humufr> is working but:

    Humufr> from pylab import * imshow(zeros((4000,4000))) show()

Yes this is a problem.  The image module turns everything into an rgba
matrix under the hood.  This was an early design decision to conserve
programmer resources (my time) over memory and CPU.  We made it with
the knowledge that this couldn't last forever, since someone (you
apparently) would eventually need a grayscale image w/o this overhead.
Note if you want to colormap this image, then the problem is going to
be there regardless.

Several releases ago I spent some time hammering on the image module
looking for CPU performance gains, with some success.  Looks like I'll
have to do the same for memory.....

Note that agg has a built-in limit of 4096x4096 buffers, and your
display device is likely to be much smaller still.  I suggest you
consider down-sampling your image in numarray before processing
passing it to imshow.  There are a number of people working on
algorithms to down-sample images (I think Maxim, the agg author is one
of them, and I think the numarray/stsci people are too).  If there is
a good agg algorithm to do it, it would be nice to expose is in mpl.

JDH

Re: [Matplotlib-users] memory problem

[Humufr <hu...@ya...>]

Ok I didn't know that it was a well known problem, thanks to point me a 
solution. The image is grayscale because it's a fits file. I'm doing 
some publicity for matplotlib in my department and someone ask me to 
plot this peculiar image but we became to have some big CCD (or ccd 
mosaic) or create some simulation with big CCD.

It's good idea if that will be possible to have inside matplotlib the 
down-sampling :)

I saw in the same time a small problem with matshow. I thought that this 
command preserve the aspect of the figure. It's true but not if you're 
using the colorbar() function at the same time.
 
thanks,

N.

a=zeros((20,20))
a[3,4] = 4             # if the image is totally flat colorbar give an error
matshow(a)
colorbar()

John Hunter wrote:

>>>>>>"Humufr" == Humufr  <hu...@ya...> writes:
>>>>>>            
>>>>>>
>
>    Humufr>                     Hi, on a pc with linux and 512M of
>    Humufr> RAM, I have a problem of memory when I'm using matplotlib.
>
>    Humufr> from pylab import * imshow(zeros((2000,2000))) show()
>
>    Humufr> is working but:
>
>    Humufr> from pylab import * imshow(zeros((4000,4000))) show()
>
>Yes this is a problem.  The image module turns everything into an rgba
>matrix under the hood.  This was an early design decision to conserve
>programmer resources (my time) over memory and CPU.  We made it with
>the knowledge that this couldn't last forever, since someone (you
>apparently) would eventually need a grayscale image w/o this overhead.
>Note if you want to colormap this image, then the problem is going to
>be there regardless.
>
>Several releases ago I spent some time hammering on the image module
>looking for CPU performance gains, with some success.  Looks like I'll
>have to do the same for memory.....
>
>Note that agg has a built-in limit of 4096x4096 buffers, and your
>display device is likely to be much smaller still.  I suggest you
>consider down-sampling your image in numarray before processing
>passing it to imshow.  There are a number of people working on
>algorithms to down-sample images (I think Maxim, the agg author is one
>of them, and I think the numarray/stsci people are too).  If there is
>a good agg algorithm to do it, it would be nice to expose is in mpl.
>
>JDH
>
>  
>

Re: [Matplotlib-users] memory problem

[Perry G. <pe...@st...>]

It's clear you are running out of memory. The image you are displaying  
is on the order of 64MB in size. While the single array itself isn't  
enough to consume all your memory, it isn't clear what other memory is  
in use, or what your page size is, etc. Matplotlib will create some  
temporaries in the process of scaling arrays for display, so I wouldn't  
be surprised to see this image require a few more times that amount of  
memory.

Perry

On Apr 1, 2005, at 9:51 AM, Humufr wrote:

>                    Hi,
>
> on a pc with linux and 512M of RAM, I have a problem of memory when  
> I'm using matplotlib.
>
> from pylab import *
> imshow(zeros((2000,2000)))
> show()
>
> is working but:
>
> from pylab import *
> imshow(zeros((4000,4000)))
> show()
>
> is not.
>
>
> That slow down the computer, it close to be freeze and I obtain this  
> error (the two error message for show() and savefig('test.png') are at  
> the bottom of the mail. (matshow give exactly the same result)
>
> It's a big problem, for me at least, because I'm working with some  
> image with 4000x4000 pixels.
>
> Thanks,
>
>       N.
>
> Exception in Tkinter callback
> Traceback (most recent call last):
>  File "/usr/local/lib/python2.4/lib-tk/Tkinter.py", line 1345, in  
> __call__
>    return self.func(*args)
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_tkagg.py", line 140, in resize
>    self.show()
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_tkagg.py", line 143, in draw
>    FigureCanvasAgg.draw(self)
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_agg.py", line 312, in draw
>    self.figure.draw(renderer)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py",  
> line 395, in draw
>    for a in self.axes: a.draw(renderer)
>  File  
> "/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/ 
> axes.py", line 1339, in draw
>    im.draw(renderer)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py",  
> line 182, in draw
>    im = self.make_image(isUpper)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py",  
> line 112, in make_image
>    x = self.to_rgba(self._A, self._alpha)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/cm.py", line  
> 418, in to_rgba
>    return self.cmap(x, alpha)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/colors.py",  
> line 526, in __call__
>    rgba = zeros(xa.shape+(4,), Float)
>  File  
> "/usr/local/lib/python2.4/site-packages/numarray/numarraycore.py",  
> line 1407, in zeros
>    retarr = NumArray(shape=shape, type=type)
> MemoryError: Couldn't allocate requested memory
>
>
> I did the same with savefig
>
> Traceback (most recent call last):
>  File "test.py", line 12, in ?
>    savefig('test.png')
>  File  
> "/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/ 
> pylab.py", line 719, in savefig
>    return fig.savefig(*args, **kwargs)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py",  
> line 512, in savefig
>    self.canvas.print_figure(*args, **kwargs)
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_tkagg.py", line 161, in print_figure
>    agg.print_figure(filename, dpi, facecolor, edgecolor, orientation)
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_agg.py", line 375, in print_figure
>    self.draw()
>  File  
> "/usr/local/lib/python2.4/site-packages/matplotlib/backends/ 
> backend_agg.py", line 312, in draw
>    self.figure.draw(renderer)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/figure.py",  
> line 395, in draw
>    for a in self.axes: a.draw(renderer)
>  File  
> "/scratch/gruel/usr/local/lib/python2.4/site-packages/matplotlib/ 
> axes.py", line 1339, in draw
>    im.draw(renderer)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py",  
> line 182, in draw
>    im = self.make_image(isUpper)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/image.py",  
> line 112, in make_image
>    x = self.to_rgba(self._A, self._alpha)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/cm.py", line  
> 418, in to_rgba
>    return self.cmap(x, alpha)
>  File "/usr/local/lib/python2.4/site-packages/matplotlib/colors.py",  
> line 526, in __call__
>    rgba = zeros(xa.shape+(4,), Float)
>  File  
> "/usr/local/lib/python2.4/site-packages/numarray/numarraycore.py",  
> line 1407, in zeros
>    retarr = NumArray(shape=shape, type=type)
> MemoryError: Couldn't allocate requested memory
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by Demarc:
> A global provider of Threat Management Solutions.
> Download our HomeAdmin security software for free today!
> http://www.demarc.com/info/Sentarus/hamr30
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users