matplotlib-users

[Matplotlib-users] Garbage collection?

[Sarah M. <mou...@gm...>]

Hi all,

is there some way to clean up all the memory used by matplotlib? I
currently have a script which creates a bunch of plots and it seems to
use an increasing amount of memory until my m/c runs out of memory and
swap space. I'm forcing the gc to collect after every plot is
generated (and saved to disc) and that doesn't help. Looking through
the list of types that the gc is holding it seems like the vast
majority of them are objects generated by matplotlib. I'm pretty sure
now that it's not an issue with the data structures I'm creating, so
... any ideas?

If it helps, I'm using matplotlib 0.87 on Fedora Core 4 with numpy
0.9.5. I've run the memory test code in
http://matplotlib.sourceforge.net/faq.html#LEAKS and got the
following:

Average memory consumed per loop: -0.1083k bytes

Many thanks!

Sarah

--
http://www.mis.coventry.ac.uk/research/imd/
http://varspool.blogspot.com

Re: [Matplotlib-users] Garbage collection?

[John H. <jdh...@ac...>]

>>>>> "Sarah" == Sarah Mount <mou...@gm...> writes:

    Sarah> Hi all, is there some way to clean up all the memory used
    Sarah> by matplotlib? I currently have a script which creates a
    Sarah> bunch of plots and it seems to use an increasing amount of
    Sarah> memory until my m/c runs out of memory and swap space. I'm
    Sarah> forcing the gc to collect after every plot is generated
    Sarah> (and saved to disc) and that doesn't help. Looking through
    Sarah> the list of types that the gc is holding it seems like the
    Sarah> vast majority of them are objects generated by
    Sarah> matplotlib. I'm pretty sure now that it's not an issue with
    Sarah> the data structures I'm creating, so ... any ideas?

Are you using pylab?  If so, most likely you are creating but not
closing figures.  You need to pair every use of pylab figure with a
pylab close.  Feel free to post an example script.

    Sarah> If it helps, I'm using matplotlib 0.87 on Fedora Core 4
    Sarah> with numpy 0.9.5. I've run the memory test code in
    Sarah> http://matplotlib.sourceforge.net/faq.html#LEAKS and got
    Sarah> the following:

    Sarah> Average memory consumed per loop: -0.1083k bytes

Good -- this indicated matplotlib proper is not leaking, so it is a
figure management problem.  You may want to inspect _pylab_helpers to
see if it is storing references you don't want it to.

Another alternative is to not use pylab for figure management.  The
best approach here depends on what kinds of batch figures you are
trying to create (GUI vs hardcopy).

JDH

Re: [Matplotlib-users] Garbage collection?

[John H. <jdh...@ac...>]

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> Good -- this indicated matplotlib proper is not leaking, so
    John> it is a figure management problem.  You may want to inspect
    John> _pylab_helpers to see if it is storing references you don't
    John> want it to.

Here is a little demo to help you if you want to poke into the pylab
helpers data structures.  Note you should not manipulate these
structures directly, but through the pylab API

In [2]: import matplotlib._pylab_helpers as ph

In [3]: ph.Gcf.figs
Out[3]: {}

In [4]: figure()
Out[4]: <matplotlib.figure.Figure instance at 0xb5b91e4c>

In [5]: ph.Gcf.figs
Out[5]: {1: <matplotlib.backends.backend_gtkagg.FigureManagerGTKAgg
instance at 0xb5b91f4c>}

In [6]: figure()
Out[6]: <matplotlib.figure.Figure instance at 0xb5b9180c>

In [7]: ph.Gcf.figs
Out[7]:
{1: <matplotlib.backends.backend_gtkagg.FigureManagerGTKAgg instance
at 0xb5b91f4c>,
 2: <matplotlib.backends.backend_gtkagg.FigureManagerGTKAgg instance
 at 0xb5b91b4c>}

In [8]: close(1)

In [9]: ph.Gcf.figs
Out[9]: {2: <matplotlib.backends.backend_gtkagg.FigureManagerGTKAgg
instance at 0xb5b91b4c>}

JDH

Re: [Matplotlib-users] Garbage collection?

[Sarah M. <mou...@gm...>]

On 07/03/06, John Hunter <jdh...@ac...> wrote:
> >>>>> "Sarah" =3D=3D Sarah Mount <mou...@gm...> writes:
>
>     Sarah> Hi all, is there some way to clean up all the memory used
>     Sarah> by matplotlib? I currently have a script which creates a
>     Sarah> bunch of plots and it seems to use an increasing amount of
>     Sarah> memory until my m/c runs out of memory and swap space. I'm
>     Sarah> forcing the gc to collect after every plot is generated
>     Sarah> (and saved to disc) and that doesn't help. Looking through
>     Sarah> the list of types that the gc is holding it seems like the
>     Sarah> vast majority of them are objects generated by
>     Sarah> matplotlib. I'm pretty sure now that it's not an issue with
>     Sarah> the data structures I'm creating, so ... any ideas?
>
> Are you using pylab?  If so, most likely you are creating but not
> closing figures.  You need to pair every use of pylab figure with a
> pylab close.  Feel free to post an example script.

Ah! Well, I guess I just didn't read the doc's properly. pylab.close()
seems to do the trick. Interestingly, the amount of garbage still
seems to grow monotonically, but much, much slower and everything
seems to be kept in memory and not swap.

Thanks for such a speedy response!

Sarah

--
http://www.mis.coventry.ac.uk/research/imd/
http://varspool.blogspot.com

Re: [Matplotlib-users] Garbage collection?

[John H. <jdh...@ac...>]

>>>>> "Sarah" == Sarah Mount <mou...@gm...> writes:
    Sarah> Ah! Well, I guess I just didn't read the doc's
    Sarah> properly. pylab.close() seems to do the
    Sarah> trick. Interestingly, the amount of garbage still seems to
    Sarah> grow monotonically, but much, much slower and everything
    Sarah> seems to be kept in memory and not swap.

Or perhaps the docs just weren't explicit enough!

I added the following to the pylab.figure docstring:

    If you are creating many figures, make sure you explicitly call "close"
    on the figures you are not using, because this will enable pylab
    to properly clean up the memory.

JDH