From: Benjamin R. <ben...@ou...> - 2010-11-18 20:21:07
|
On Thu, Nov 18, 2010 at 1:11 PM, Caleb Constantine <cad...@gm...>wrote: > Matplotlib Users: > > It seems matplotlib plotting has a relatively small memory leak. My > experiments suggest it leaks between 5K and 8K bytes of RAM for ever plot > redraw. For example, in one experiment, plotting the same buffer (so as to > not > allocate new memory) every second for a period of about 12 hours resulted > in > memory usage (physical RAM) increasing by approximately 223MB, which is > about > 5.3K per replot. The plotting code is: > > class PlotPanel(wx.Panel): > def __init__(self, parent): > wx.Panel.__init__(self, parent, wx.ID_ANY, > style=wx.BORDER_THEME|wx.TAB_TRAVERSAL) > self._figure = MplFigure(dpi=None) > self._canvas = MplCanvas(self, -1, self._figure) > self._axes = self._figure.add_subplot(1,1,1) > > sizer = wx.BoxSizer(wx.VERTICAL) > sizer.Add(self._canvas, 1, wx.EXPAND|wx.TOP, 5) > self.SetSizer(sizer) > > def draw(self, channel, seconds): > self._axes.clear() > self._axes.plot(channel, seconds) > self._canvas.draw() > > > `draw()` is called every second with the same `channels` and `seconds` > numpy.array buffers. > > In my case, this leak, though relatively small, becomes a serious issue > since > my software often runs for long periods of time (days) plotting data > streamed > from a data acquisition unit. > > Any suggestions will help. Am I miss understanding something here? Maybe I > need to call some obscure function to free memory, or something? > > My testing environment: > > * Windws XP SP3, Intel Core 2 Duo @ 2.33GHz, 1.96 GB RAM > * Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit > (Intel)] on win32 > * matplotlib version 1.0.0 > * numpy 1.4.1 > * wxPython version 2.8.11.0 > > The complete test program follows. > > Thanks, > > Caleb > > > from random import random > from datetime import datetime > import os > import time > import win32api > import win32con > import win32process > > import wx > import numpy > > import matplotlib as mpl > from matplotlib.figure import Figure as MplFigure > from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as > MplCanvas > > def get_process_memory_info(process_id): > memory = {} > process = None > try: > process = win32api.OpenProcess( > win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ, > False, process_id); > if process is not None: > return win32process.GetProcessMemoryInfo(process) > finally: > if process: > win32api.CloseHandle(process) > return memory > > meg = 1024.0 * 1024.0 > > class PlotPanel(wx.Panel): > def __init__(self, parent): > wx.Panel.__init__(self, parent, wx.ID_ANY, > style=wx.BORDER_THEME|wx.TAB_TRAVERSAL) > self._figure = MplFigure(dpi=None) > self._canvas = MplCanvas(self, -1, self._figure) > self._axes = self._figure.add_subplot(1,1,1) > > sizer = wx.BoxSizer(wx.VERTICAL) > sizer.Add(self._canvas, 1, wx.EXPAND|wx.TOP, 5) > self.SetSizer(sizer) > > def draw(self, channel, seconds): > self._axes.clear() > self._axes.plot(channel, seconds) > self._canvas.draw() > > class TestFrame(wx.Frame): > def __init__(self, parent, id, title): > wx.Frame.__init__( > self, parent, id, title, wx.DefaultPosition, (600, 400)) > > self.testDuration = 60 * 60 * 24 > self.startTime = 0 > > self.channel = numpy.sin(numpy.arange(1000) * random()) > self.seconds = numpy.arange(len(self.channel)) > > self.plotPanel = PlotPanel(self) > > sizer = wx.BoxSizer(wx.VERTICAL) > sizer.Add(self.plotPanel, 1 ,wx.EXPAND) > self.SetSizer(sizer) > > self._timer = wx.Timer(self) > self.Bind(wx.EVT_TIMER, self._onTimer, self._timer) > self._timer.Start(1000) > print "starting memory: ",\ > get_process_memory_info(os.getpid())["WorkingSetSize"]/meg > > def _onTimer(self, evt): > if self.startTime == 0: > self.startTime = time.time() > > if (time.time() - self.startTime) >= self.testDuration: > self._timer.Stop() > > self.plotPanel.draw(self.channel, self.seconds) > > t = datetime.now() > memory = get_process_memory_info(os.getpid()) > print "time: {0}, working: {1:f}".format( > t, memory["WorkingSetSize"]/meg) > > class MyApp(wx.App): > def OnInit(self): > frame = TestFrame(None, wx.ID_ANY, "Memory Leak") > self.SetTopWindow(frame) > frame.Show(True) > return True > > if __name__ == '__main__': > app = MyApp(0) > app.MainLoop() > > > Caleb, Interesting analysis. One possible source of a leak would be some sort of dangling reference that still hangs around even though the plot objects have been cleared. By the time of the matplotlib 1.0.0 release, we did seem to clear out pretty much all of these, but it is possible there are still some lurking about. We should probably run your script against the latest svn to see how the results compare. Another possibility might be related to numpy. However this is the draw statement, so I don't know how much numpy is used in there. The latest refactor work in numpy has revealed some memory leaks that have existed, so who knows? Might be interesting to try making equivalent versions of this script using different backends, and different package versions to possibly isolate the source of the memory leak. Thanks for your observations, Ben Root |