Re: [Matplotlib-users] hist() memory leak?

[Michael D. <md...@st...>]

There have been a number of memory leaks resolved since the 0.90.1 
release.  However, there are still known memory leaks in all of the GUI 
backends, some of which are unfortunately just beyond easy reach of 
matplotlib.  If this is an automated process and you only care about the 
file output, you could try using the "Agg", "Pdf", "Ps" or "Svg" 
backends, e.g.:

	import matplotlib
	matplotlib.use("Agg")	

I tried your script with both 0.90.1 and the latest svn, and I could 
reproduce your leak with the TkAgg backend, but not with the Agg backend.

If you need a GUI, you may want to try using the latest svn version if 
you can.  The leaks still exist there, but they are much smaller.

BTW -- you can get the version of matplotlib with:

 >>> import matplotlib
 >>> matplotlib.__version__
'0.90.1'

Cheers,
Mike


Luke Robison wrote:
> I'm writing a program that processes ~ 25,000 jobs and each iteration
> draws a histogram and writes out some of the output.  I let it run all
> night and when I came back, python was filling up all my memory
> (2Gigs) and was thrashing on and off of swap.  I narrowed the problem
> down to my calling of the hist() function, and was able to reproduce
> it in the following code I copied from the mpl website.
> 
> Am I not properly closing the figure somehow?
> Has this issue already been addressed?
> 
> I recently installed version 0.90.1 of matplotlib, although I don't
> see any easy way to verify that version number from within python.
> 
> -Luke Robison
> 
> 
> Code:
> -------------------
> import os,time,sys
> from pylab import *
> 
> def report_memory(i):
>     pid = os.getpid()
>     a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
>     print i, '  ', a2[1],
>     return int(a2[1].split()[1])
> 
> # take a memory snapshot on indStart and compare it with indEnd
> indStart, indEnd = 100, 150
> for i in range(indStart,indEnd):
>     ind = arange(100)
>     xx = rand(len(ind))
> 
>     figure(1)
>     hist(xx)
>     close(1)
> 
> # wait a few cycles for memory usage to stabilize
>     if i==indStart: start = val
>     if i>indStart:
>         end = val
>         print 'Average memory consumed per loop: %1.4fk bytes' % \
>               ((end-start)/float(indEnd-indStart))
> 
> -----------------
> 
> Output:
> 
> 
> python memtest.py
> 
> Average memory consumed per loop: 0.0000k bytes
> 102    39808 21991
> Average memory consumed per loop: 0.0000k bytes
> 103    39828 21991
> Average memory consumed per loop: 0.0000k bytes
> 104    39852 22024
> Average memory consumed per loop: 0.6600k bytes
> 105    39876 22024
> Average memory consumed per loop: 0.6600k bytes
> 106    39908 22024
> Average memory consumed per loop: 0.6600k bytes
> 107    39932 22024
> Average memory consumed per loop: 0.6600k bytes
> 108    39960 22024
> Average memory consumed per loop: 0.6600k bytes
> 109    39980 22057
> Average memory consumed per loop: 1.3200k bytes
> 110    40008 22057
> Average memory consumed per loop: 1.3200k bytes
> 111    40032 22057
> Average memory consumed per loop: 1.3200k bytes
> 112    40056 22057
> Average memory consumed per loop: 1.3200k bytes
> 113    40084 22057
> Average memory consumed per loop: 1.3200k bytes
> 114    40104 22090
> Average memory consumed per loop: 1.9800k bytes
> 115    40132 22090
> Average memory consumed per loop: 1.9800k bytes
> 116    40156 22090
> Average memory consumed per loop: 1.9800k bytes
> 117    40180 22090
> Average memory consumed per loop: 1.9800k bytes
> 118    40208 22090
> Average memory consumed per loop: 1.9800k bytes
> 119    40232 22123
> Average memory consumed per loop: 2.6400k bytes
> 120    40256 22123
> Average memory consumed per loop: 2.6400k bytes
> 121    40280 22123
> Average memory consumed per loop: 2.6400k bytes
> 122    40304 22123
> Average memory consumed per loop: 2.6400k bytes
> 123    40328 22123
> Average memory consumed per loop: 2.6400k bytes
> 124    40356 22123
> Average memory consumed per loop: 2.6400k bytes
> 125    40380 22156
> Average memory consumed per loop: 3.3000k bytes
> 126    40404 22156
> Average memory consumed per loop: 3.3000k bytes
> 127    40428 22156
> Average memory consumed per loop: 3.3000k bytes
> 128    40452 22156
> Average memory consumed per loop: 3.3000k bytes
> 129    40476 22156
> Average memory consumed per loop: 3.3000k bytes
> 130    40500 22189
> Average memory consumed per loop: 3.9600k bytes
> 131    40528 22189
> Average memory consumed per loop: 3.9600k bytes
> 132    40548 22189
> Average memory consumed per loop: 3.9600k bytes
> 133    40576 22189
> Average memory consumed per loop: 3.9600k bytes
> 134    40596 22189
> Average memory consumed per loop: 3.9600k bytes
> 135    40624 22222
> Average memory consumed per loop: 4.6200k bytes
> 136    40652 22222
> Average memory consumed per loop: 4.6200k bytes
> 137    40676 22222
> Average memory consumed per loop: 4.6200k bytes
> 138    40700 22222
> Average memory consumed per loop: 4.6200k bytes
> 139    40724 22222
> Average memory consumed per loop: 4.6200k bytes
> 140    40744 22222
> Average memory consumed per loop: 4.6200k bytes
> 141    40768 22255
> Average memory consumed per loop: 5.2800k bytes
> 142    40800 22255
> Average memory consumed per loop: 5.2800k bytes
> 143    40824 22255
> Average memory consumed per loop: 5.2800k bytes
> 144    40848 22255
> Average memory consumed per loop: 5.2800k bytes
> 145    40872 22255
> Average memory consumed per loop: 5.2800k bytes
> 146    40896 22288
> Average memory consumed per loop: 5.9400k bytes
> 147    40916 22288
> Average memory consumed per loop: 5.9400k bytes
> 148    40940 22288
> Average memory consumed per loop: 5.9400k bytes
> 149    40972 22288
> Average memory consumed per loop: 5.9400k bytes
> 
> 
> as you can see, the memory consumption is increasing each loop, and
> furthermore, and an increasing rate :-(
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users