Hi,

I think I've figured out what's going on.  It's a combination of things:

1) iPython is ignorant of the problems associated with caching massive data output
2) iPython doesn't seem to have a good way to clear data from memory reliably (https://bugs.launchpad.net/ipython/+bug/412350)
3) matplotlib/Python seems to be insufficiently aggressive in its garbage collection (??)
4) For obvious reasons, JPGs are much bigger when stored as arrays (though they still seem to take up more memory than they should)

Problems 1-3 seem problematic enough that they will get fixed eventually.

... but (4) is a design issue.  Assuming it's possible, it looks like there could be benefits to making an array-like wrapper around PIL image objects (perhaps similar in principle to a sparse matrix).  Given PIL.ImageMath, ImagePath, etc., it seems actually fairly doable.  Wouldn't something like this be of major benefit to people using SciPy for anything image-related?

Leo

On Fri, Oct 2, 2009 at 7:45 AM, Michael Droettboom <mdroe@stsci.edu> wrote:
If you assign each figure to a new number, it will keep all of those figures around in memory (because pyplot thinks you may want to use it again.)  The best route is to call close('all') or fig.close() with each loop iteration.

40MB per image doesn't sound way out of reason to me.  How big are your images?

Mike


On 10/01/2009 10:25 PM, Leo Trottier wrote:
I have a friend who's having strange memory issues when opening and displaying images (using Matplotlib).

Here's what he says:
#######################################

pylab seems really inefficient: Opening a few images and displaying them eats up tons of memory, and the memory doesn't get freed.

Starting python, and run

In [5]: from glob import *;

In [6]: from pylab import *

python has 33MB of memory.


Run

In [7]: i = 1

In [8]: for imname in glob("*.JPG"):
  ...:     im = imread(imname)
  ...:     figure(i); i = i+1
  ...:     imshow(im)
  ...:

This opens 10 figures and displays them. Python takes 480MB of memory. This is crazy, for 10 images -- 40+MB of memory for each!

In [14]: close("all")

In [15]: i = 1

In [16]: for imname in glob("*.JPG"):
   im = imread(imname)
   figure(i); i = i+1
   imshow(im)
  ....:
  ....:

This closes all figures and opens them again. Python takes up 837MB of memory.

and so on... Something is really wrong with memory management.

##### System info: ##############

(using macosx backend)

2.4GHz MacBook Pro Intel Core 2 Duo

4GB 667MHz DDR2 SDRAM

In [5]: sys.version
Out[5]: '2.6.2 (r262:71600, Oct  1 2009, 16:44:23) \n[GCC 4.2.1 (Apple Inc. build 5646)]'

In [6]: numpy.__version__
Out[6]: '1.3.0'

In [7]: matplotlib.__version__
Out[7]: '0.99.1.1'

In [8]: scipy.__version__
Out[8]: '0.7.1'

In [9]:



------------------------------------------------------------------------------ Come build with us! The BlackBerry&reg; Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33; http://p.sf.net/sfu/devconf

_______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users