Re: [Matplotlib-users] Possible memory leak?

[Michael D. <md...@st...>]

On 04/20/2011 11:27 AM, Caleb Constantine wrote:
> On Wed, Apr 20, 2011 at 9:29 AM, Michael Droettboom<md...@st...>  wrote:
>> On 04/20/2011 07:48 AM, Caleb Constantine wrote:
>>> On Tue, Apr 19, 2011 at 2:25 PM, Michael Droettboom<md...@st...>    wrote:
>>>> Ok.  I have a RHEL5 Linux box with Python 2.7.1.
>>>>
>>>> With Numpy 1.4.1 and 1.5.1 I don't see any leaks.  With Numpy git HEAD,
>>>> I did see a leak -- I submitted a pull request to Numpy here:
>>>>
>>>>     https://github.com/numpy/numpy/pull/76
>>>>
>>>> I get the same results (no leaks) running your wx, tk and agg scripts
>>>> (with the Windows-specific stuff removed).
>>>>
>>>> FWIW, I have wxPython 2.8.11.0 and Tkinter rev 81008.
>>>>
>>>> So the variables are the platform and the version of Python.  Perhaps
>>>> it's one of those two things?
>>>>
>>>> Mike
>>> Consider the following:
>>>
>>>        matplotlib 1.0.1, numpy 1.5.1, python 2.7.1, wxPython 2.8.11.0,
>>> Windows XP SP3
>>>
>>>        - 1 hour
>>>        - Plotted 3601 times, about 1Hz
>>>        - Memory usage increased by about 1.16MB (41.39 - 40.23), or
>>> about 0.33K per redraw
>>>
>>> It seems the same memory leak exists. Given you don't have this issue
>>> on Linux with the same Python configuration, I can only assume it is
>>> related to some Windows specific code somewhere. I'll run for a longer
>>> period of time just in case, but I don't expect the results to be
>>> different.
>> One way to rule out Windows-specific code may be to run with the Agg
>> backend only (without wx).  Have you plotted the memory growth?  This
>> amount of memory growth is well within the pool allocation sizes that
>> Python routinely uses.  Does the value of len(gc.get_objects()) grow
>> over time?
>>
> New results follows.
>
> matplotlib 1.0.1, numpy 1.5.1, python 2.7.1, wxPython 2.8.11.0, Windows XP SP3
>
> agg
> - 3601 redraws (1 hour), about 1Hz
> - Memory usage: 28.79 - 27.57 = 1.22 MB
> - len(gc.get_objects()): 23424 at beginning and end
> - Plot of memory growth: roughly linear, increasing with slope of 0.26KB
>
> tkagg
> - 3601 redraws (1 hour), about 1Hz
> - Memory usage: 33.22 - 33.32 = -0.1 MB
> - len(gc.get_objects()): 24182 at beginning and end
> - Plot of memory growth: very irregular (up and down), but a line fit
>    has a slope of about 0.025KB (I could run longer and see if slope
> approaches 0)
>
> wxagg
> - 3601 redraws (1 hour), about 1Hz
> - Memory usage: 43.28 - 41.80 = 1.5 MB
> - len(gc.get_objects()): 41473 at beginning and end
> - Plot of memory growth: roughly linear, increasing with slope of 0.32KB
Thanks.  These are very useful results.

The fact that gc.get_objects() remains constant suggests to me that this 
is not a simple case of holding on to a Python reference longer than we 
intend to.  Instead, this is either a C-side reference counting bug, or 
a genuine C malloc-and-never-free bug.  Puzzlingly, valgrind usually 
does a very good job of finding such bugs, but is turning up nothing for 
me.  Will have to scratch my head a little bit longer and see if I can 
come up with a proper experiment that will help me get to the bottom of 
this.

Cheers,
Mike