matplotlib-users

[Matplotlib-users] memory leak with PyQt4 plus savefig or print_figure

[Ralf G. <ral...@go...>]

#!/usr/bin/env python

# embedding_in_qt4.py --- Simple Qt4 application embedding matplotlib canvases
#
# Copyright (C) 2005 Florent Rougon
#               2006 Darren Dale
#
# This file is an example program for matplotlib. It may be used and
# modified with no restriction; raw copies as well as modified versions
# may be distributed without limitation.

import sys, os, random
from PyQt4 import QtGui, QtCore

from numpy import arange, sin, pi
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.figure import Figure


class MyMplCanvas(FigureCanvas):
    """Ultimately, this is a QWidget (as well as a FigureCanvasAgg, etc.)."""
    def __init__(self, parent=None, width=5, height=4, dpi=100):
        fig = Figure(figsize=(width, height), dpi=dpi)
        self.axes = fig.add_subplot(111)
        # We want the axes cleared every time plot() is called
        self.axes.hold(False)

        self.compute_initial_figure()

        FigureCanvas.__init__(self, fig)
        self.setParent(parent)

        FigureCanvas.setSizePolicy(self,
                                   QtGui.QSizePolicy.Expanding,
                                   QtGui.QSizePolicy.Expanding)
        FigureCanvas.updateGeometry(self)

    def compute_initial_figure(self):
        pass


class MyStaticMplCanvas(MyMplCanvas):
    """Simple canvas with a sine plot."""
    def compute_initial_figure(self):
        t = arange(0.0, 3.0, 0.01)
        s = sin(2*pi*t)
        self.axes.plot(t, s)


def save_png():
    """Save an image as a png file"""

    pngpath = 'test_mplsave.png'

    fig = Figure()
    canvas = FigureCanvas(fig)
    ax = fig.add_subplot(111)
    x = arange(5e3)
    ax.plot(x, sin(x))
    #canvas.print_figure(pngpath)

    ## tried all things commented out below, all makes no difference ##
    #fig.savefig(pngpath)

    #del(fig)
    #del(canvas)
    #del(ax)

    #import matplotlib.pyplot as plt
    #plt.close(fig)

    #import gc
    #gc.collect()


class ApplicationWindow(QtGui.QMainWindow):
    def __init__(self):
        QtGui.QMainWindow.__init__(self)
        self.setAttribute(QtCore.Qt.WA_DeleteOnClose)
        self.setWindowTitle("application main window")

        self.file_menu = QtGui.QMenu('&File', self)
        self.file_menu.addAction('&Quit', self.fileQuit,
                                 QtCore.Qt.CTRL + QtCore.Qt.Key_Q)
        self.menuBar().addMenu(self.file_menu)

        self.help_menu = QtGui.QMenu('&Help', self)

        self.main_widget = QtGui.QWidget(self)

        l = QtGui.QVBoxLayout(self.main_widget)
        sc = MyStaticMplCanvas(self.main_widget, width=5, height=4, dpi=100)
        dc = QtGui.QPushButton('Save image')
        l.addWidget(sc)
        l.addWidget(dc)

        self.main_widget.setFocus()
        self.setCentralWidget(self.main_widget)

        self.statusBar().showMessage("All hail matplotlib!", 2000)
        self.connect(dc, QtCore.SIGNAL("clicked()"), save_png)

    def fileQuit(self):
        self.close()

    def closeEvent(self, ce):
        self.fileQuit()


qApp = QtGui.QApplication(sys.argv)

aw = ApplicationWindow()
aw.setWindowTitle("Try saving a simple png image")
aw.show()
sys.exit(qApp.exec_())

Re: [Matplotlib-users] memory leak with PyQt4 plus savefig or print_figure

[Michael D. <md...@st...>]

It is not surprising that memory usage is much lower without printing 
the plot.  Very little is actually done by the "plot" command other than 
setting up a tree of objects that is later rendered during the printing 
process (where most of the work happens).

The attached script based on what you provided doesn't leak memory for 
me with matplotlib 0.98.5.2.  It would appear that there is something 
else in your application triggering the leak.  Perhaps there is 
something holding a reference longer than it should?

You can see from the attached massif plots that memory usage never 
becomes unbounded.  It is normal for Python to delay deallocation for 
efficiency reasons.  Calling gc.collect (the second graph) does help 
keep memory usage more compact however, if that is a primary concern.

Cheers,
Mike

Ralf Gommers wrote:
> Hi,
>
> I am working on a PyQt4 application with some embedded MPL figures, 
> and am also trying to save some figures as png's without displaying 
> them. I am observing huge memory increases (10s or 100s of Mb) the 
> moment I try to save a png. I reproduced the issue by combining two 
> mpl examples, 
> http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_qt4.html 
> and http://matplotlib.sourceforge.net/examples/api/agg_oo.html. Full 
> code is attached. When pressing the "save figure" button, memory usage 
> shoots up, multiple clicks keep sending it higher (although not 
> monotonically).
>
> I tested on two different platforms
> - Matplotlib 98.5.2 and Python 2.6.2 on Ubuntu.
> - latest Enthought Python Distribution on Windows XP.
>
> The function that does the png saving is:
>
> def save_png():
>     """Save an image as a png file"""
>
>     pngpath = 'test_mplsave.png'
>
>     fig = Figure()
>     canvas = FigureCanvas(fig)
>     ax = fig.add_subplot(111)
>     x = arange(5e3)
>     ax.plot(x, sin(x))
>     canvas.print_figure(pngpath)
>
>     ## tried all things commented out below, all makes no difference ##
>     #fig.savefig(pngpath)
>
>     #del(fig)
>     #del(canvas)
>     #del(ax)
>
>     #import matplotlib.pyplot as plt
>     #plt.close(fig)
>
>     #import gc
>     #gc.collect()
>
> Commenting out the canvas.print_figure line fixes the issue.
>
> Am I doing something obviously wrong, or mixing two incompatible ways 
> of doing things?
>
> Cheers,
> Ralf
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Re: [Matplotlib-users] memory leak with PyQt4 plus savefig or print_figure

[Ralf G. <ral...@go...>]

sorry, i replied to Mike and not to the list. see below.


On Thu, Jul 2, 2009 at 2:57 PM, Ralf Gommers <ral...@go...>wrote:

> Thanks for looking into this Mike.
>
> On Thu, Jul 2, 2009 at 10:39 AM, Michael Droettboom <md...@st...>wrote:
>
>> It is not surprising that memory usage is much lower without printing the
>> plot.  Very little is actually done by the "plot" command other than setting
>> up a tree of objects that is later rendered during the printing process
>> (where most of the work happens).
>>
>> The attached script based on what you provided doesn't leak memory for me
>> with matplotlib 0.98.5.2.  It would appear that there is something else in
>> your application triggering the leak.  Perhaps there is something holding a
>> reference longer than it should?
>>
>
> Your attached script memleak2.py is indeed fine, it never takes up more
> than 60Mb of RAM. But did you try to run the PyQt4  GUI I attached? There
> the same save_png() function does increase memory usage for each call.
>
> I'm not sure how to exactly measure memory usage for a GUI program, but it
> is so large I can look at "System Activity" (or Task Manager on XP) and get
> the approximate number:
>
> Loading the GUI: 30.5Mb
> 1st call to save_png: 82Mb
> 2nd call: 116Mb
> 10th call: 380Mb
>
> I can see the memory usage drop after some calls, so I guess it is a
> problem of references being held and sometimes being released as you said.
> But memory use does seem to be unbounded. Waiting for minutes, or
> interacting with the GUI, never releases any memory. It is strange, the
> PyQt4 demo program is fine by itself, save_png() is fine by itself, but
> combine them and there's a memory problem.
>
> Cheers,
> Ralf
>
>
>
>> You can see from the attached massif plots that memory usage never becomes
>> unbounded.  It is normal for Python to delay deallocation for efficiency
>> reasons.  Calling gc.collect (the second graph) does help keep memory usage
>> more compact however, if that is a primary concern.
>>
>> Cheers,
>> Mike
>>
>> Ralf Gommers wrote:
>>
>>> Hi,
>>>
>>> I am working on a PyQt4 application with some embedded MPL figures, and
>>> am also trying to save some figures as png's without displaying them. I am
>>> observing huge memory increases (10s or 100s of Mb) the moment I try to save
>>> a png. I reproduced the issue by combining two mpl examples,
>>> http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_qt4.htmland
>>> http://matplotlib.sourceforge.net/examples/api/agg_oo.html. Full code is
>>> attached. When pressing the "save figure" button, memory usage shoots up,
>>> multiple clicks keep sending it higher (although not monotonically).
>>>
>>> I tested on two different platforms
>>> - Matplotlib 98.5.2 and Python 2.6.2 on Ubuntu.
>>> - latest Enthought Python Distribution on Windows XP.
>>>
>>> The function that does the png saving is:
>>>
>>> def save_png():
>>>    """Save an image as a png file"""
>>>
>>>    pngpath = 'test_mplsave.png'
>>>
>>>    fig = Figure()
>>>    canvas = FigureCanvas(fig)
>>>    ax = fig.add_subplot(111)
>>>    x = arange(5e3)
>>>    ax.plot(x, sin(x))
>>>    canvas.print_figure(pngpath)
>>>
>>>    ## tried all things commented out below, all makes no difference ##
>>>    #fig.savefig(pngpath)
>>>
>>>    #del(fig)
>>>    #del(canvas)
>>>    #del(ax)
>>>
>>>    #import matplotlib.pyplot as plt
>>>    #plt.close(fig)
>>>
>>>    #import gc
>>>    #gc.collect()
>>>
>>> Commenting out the canvas.print_figure line fixes the issue.
>>>
>>> Am I doing something obviously wrong, or mixing two incompatible ways of
>>> doing things?
>>>
>>> Cheers,
>>> Ralf
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-users mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>
>>
>> --
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>> from matplotlib.backends.backend_agg import FigureCanvasAgg as
>> FigureCanvas
>> from matplotlib.figure import Figure
>>
>> from numpy import arange, sin
>>
>> import gc
>>
>> def save_png():
>>    """Save an image as a png file"""
>>
>>    pngpath = 'test_mplsave.png'
>>
>>    fig = Figure()
>>    canvas = FigureCanvas(fig)
>>    ax = fig.add_subplot(111)
>>    x = arange(5e3)
>>    ax.plot(x, sin(x))
>>    canvas.print_figure(pngpath)
>>
>>    # The below is not necessary to prevent a leak, but it does make
>>    # memory usage more compact
>>    gc.collect()
>>
>> for i in range(100):
>>    save_png()
>>
>>
>>
>>
>

Re: [Matplotlib-users] memory leak with PyQt4 plus savefig or print_figure

[Alexander B. <bak...@gm...>]

Ralph,

Perhaps time to migrate to Chaco API from Enthought? not sure if there is
Ubuntu support yet however.

Alex Baker

http://code.enthought.com/chaco/


Ralf Gommers-2 wrote:
> 
> Hi,
> 
> I am working on a PyQt4 application with some embedded MPL figures, and am
> also trying to save some figures as png's without displaying them. I am
> observing huge memory increases (10s or 100s of Mb) the moment I try to
> save
> a png. I reproduced the issue by combining two mpl examples,
> http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_qt4.htmland
> http://matplotlib.sourceforge.net/examples/api/agg_oo.html. Full code is
> attached. When pressing the "save figure" button, memory usage shoots up,
> multiple clicks keep sending it higher (although not monotonically).
> 
> I tested on two different platforms
> - Matplotlib 98.5.2 and Python 2.6.2 on Ubuntu.
> - latest Enthought Python Distribution on Windows XP.
> 
> The function that does the png saving is:
> 
> def save_png():
>     """Save an image as a png file"""
> 
>     pngpath = 'test_mplsave.png'
> 
>     fig = Figure()
>     canvas = FigureCanvas(fig)
>     ax = fig.add_subplot(111)
>     x = arange(5e3)
>     ax.plot(x, sin(x))
>     canvas.print_figure(pngpath)
> 
>     ## tried all things commented out below, all makes no difference ##
>     #fig.savefig(pngpath)
> 
>     #del(fig)
>     #del(canvas)
>     #del(ax)
> 
>     #import matplotlib.pyplot as plt
>     #plt.close(fig)
> 
>     #import gc
>     #gc.collect()
> 
> Commenting out the canvas.print_figure line fixes the issue.
> 
> Am I doing something obviously wrong, or mixing two incompatible ways of
> doing things?
> 
> Cheers,
> Ralf
> 
> #!/usr/bin/env python
> 
> # embedding_in_qt4.py --- Simple Qt4 application embedding matplotlib
> canvases
> #
> # Copyright (C) 2005 Florent Rougon
> #               2006 Darren Dale
> #
> # This file is an example program for matplotlib. It may be used and
> # modified with no restriction; raw copies as well as modified versions
> # may be distributed without limitation.
> 
> import sys, os, random
> from PyQt4 import QtGui, QtCore
> 
> from numpy import arange, sin, pi
> from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as
> FigureCanvas
> from matplotlib.figure import Figure
> 
> 
> class MyMplCanvas(FigureCanvas):
>     """Ultimately, this is a QWidget (as well as a FigureCanvasAgg,
> etc.)."""
>     def __init__(self, parent=None, width=5, height=4, dpi=100):
>         fig = Figure(figsize=(width, height), dpi=dpi)
>         self.axes = fig.add_subplot(111)
>         # We want the axes cleared every time plot() is called
>         self.axes.hold(False)
> 
>         self.compute_initial_figure()
> 
>         FigureCanvas.__init__(self, fig)
>         self.setParent(parent)
> 
>         FigureCanvas.setSizePolicy(self,
>                                    QtGui.QSizePolicy.Expanding,
>                                    QtGui.QSizePolicy.Expanding)
>         FigureCanvas.updateGeometry(self)
> 
>     def compute_initial_figure(self):
>         pass
> 
> 
> class MyStaticMplCanvas(MyMplCanvas):
>     """Simple canvas with a sine plot."""
>     def compute_initial_figure(self):
>         t = arange(0.0, 3.0, 0.01)
>         s = sin(2*pi*t)
>         self.axes.plot(t, s)
> 
> 
> def save_png():
>     """Save an image as a png file"""
> 
>     pngpath = 'test_mplsave.png'
> 
>     fig = Figure()
>     canvas = FigureCanvas(fig)
>     ax = fig.add_subplot(111)
>     x = arange(5e3)
>     ax.plot(x, sin(x))
>     #canvas.print_figure(pngpath)
> 
>     ## tried all things commented out below, all makes no difference ##
>     #fig.savefig(pngpath)
> 
>     #del(fig)
>     #del(canvas)
>     #del(ax)
> 
>     #import matplotlib.pyplot as plt
>     #plt.close(fig)
> 
>     #import gc
>     #gc.collect()
> 
> 
> class ApplicationWindow(QtGui.QMainWindow):
>     def __init__(self):
>         QtGui.QMainWindow.__init__(self)
>         self.setAttribute(QtCore.Qt.WA_DeleteOnClose)
>         self.setWindowTitle("application main window")
> 
>         self.file_menu = QtGui.QMenu('&File', self)
>         self.file_menu.addAction('&Quit', self.fileQuit,
>                                  QtCore.Qt.CTRL + QtCore.Qt.Key_Q)
>         self.menuBar().addMenu(self.file_menu)
> 
>         self.help_menu = QtGui.QMenu('&Help', self)
> 
>         self.main_widget = QtGui.QWidget(self)
> 
>         l = QtGui.QVBoxLayout(self.main_widget)
>         sc = MyStaticMplCanvas(self.main_widget, width=5, height=4,
> dpi=100)
>         dc = QtGui.QPushButton('Save image')
>         l.addWidget(sc)
>         l.addWidget(dc)
> 
>         self.main_widget.setFocus()
>         self.setCentralWidget(self.main_widget)
> 
>         self.statusBar().showMessage("All hail matplotlib!", 2000)
>         self.connect(dc, QtCore.SIGNAL("clicked()"), save_png)
> 
>     def fileQuit(self):
>         self.close()
> 
>     def closeEvent(self, ce):
>         self.fileQuit()
> 
> 
> qApp = QtGui.QApplication(sys.argv)
> 
> aw = ApplicationWindow()
> aw.setWindowTitle("Try saving a simple png image")
> aw.show()
> sys.exit(qApp.exec_())
> 
> ------------------------------------------------------------------------------
> 
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
> 
> 

-- 
View this message in context: http://www.nabble.com/memory-leak-with-PyQt4-plus-savefig-or-print_figure-tp24298299p24603899.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Re: [Matplotlib-users] memory leak with PyQt4 plus savefig or print_figure

[Ralf G. <ral...@go...>]

On Wed, Jul 22, 2009 at 6:36 AM, Alexander Baker
<bak...@gm...>wrote:

>
>
> Ralph,
>
> Perhaps time to migrate to Chaco API from Enthought? not sure if there is
> Ubuntu support yet however.


Alex, I assume you did not mean to send this to the list...

Anyway, matplotlib is a central part of my application and is not going
anywhere. The next step to try to fix this issue is compiling from trunk and
seeing if anything changes. Unfortunately I do not have too much time at the
moment so it may take a while to find the problem.

Cheers,
Ralf


>
> Alex Baker
>
> http://code.enthought.com/chaco/
>
>
> Ralf Gommers-2 wrote:
> >
> > Hi,
> >
> > I am working on a PyQt4 application with some embedded MPL figures, and
> am
> > also trying to save some figures as png's without displaying them. I am
> > observing huge memory increases (10s or 100s of Mb) the moment I try to
> > save
> > a png. I reproduced the issue by combining two mpl examples,
> >
> http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_qt4.htmland
> > http://matplotlib.sourceforge.net/examples/api/agg_oo.html. Full code is
> > attached. When pressing the "save figure" button, memory usage shoots up,
> > multiple clicks keep sending it higher (although not monotonically).
>
>