matplotlib-devel — matplotlib developers

Re: [matplotlib-devel] Re: Routine to show matrices?

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Anyway, many thanks for including this.  Let me know
    Fernando> when you fix the current bugs and I'll test it again.

Thanks for the comments - I incorporated them into my local tree.  The
axes thing was just a typo.  In matshow, it should read


    ax = fig.add_axes([0.1, 0.05, 0.8, 0.8])
                         ^

JDH

Re: [matplotlib-devel] Re: Routine to show matrices?

[Fernando P. <Fer...@co...>]

John Hunter wrote:

>  * you also need to be sure that the axes have the same width and
>    height.  Since the width and height are expressed in fractions of
>    the figure size, you can use any width and height you want and get
>    the right answer as long as they are the same, since the figure
>    width and height have the right aspect ration.  The default axes --
>    subplot(111) -- have slightly different values for w and h.  This
>    is fixed in CVS with 
> 
> 
>     w,h = figaspect(arr)
>     fig = figure(figsize=(w,h))
>     ax = fig.add_axes([0.0, 0.05, 0.8, 0.8])

Well, but this breaks things badly.  Have a run at my supplied examples with 
your version, and you'll see the arrays get banged against the left side. 
(make sure to rename the matshow routine in my example to matshow2, so you are 
actually testing matplotlib and not my code)

I played with these numbers a bit but couldn't get something that looked OK 
for all the examples.  The original looked perfect, so it's a matter of 
reusing the same defaults that I was ending up with via my imshow() call, I 
guess.  But I did some grepping for add_axes and couldn't find it elsewhere, 
so I don't know how to fix it.

>  * To get the labels on top, you do
> 
>    ax.xaxis.tick_top()  # this turns off tick bottom and turns on tick  top
> 
>    likewise, there are yaxis functions tick_left and tick_right.  This
>    is also in CVS

This is nice, thanks.

>   * I return a (fig, ax, im) tuple 

OK.

Minor docstring nits:

     matshow() calls imshow() with Aargs and **kwargs, but by default

should be:

     matshow() calls imshow() with *args and **kwargs, but by default

And why do you use enumerate here?

     for i, d in enumerate(dimlist):
         fig, ax, im = matshow(samplemat(d))
     show()

This example would be clearer with just

     for d dimlist:
         fig, ax, im = matshow(samplemat(d))
     show()

since you don't really use the index for anything.  Less noise to read through.

Anyway, many thanks for including this.  Let me know when you fix the current 
bugs and I'll test it again.

Cheers,

f

Re: [matplotlib-devel] Re: Routine to show matrices?

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> I just tested this code on a second box with different
    Fernando> mpl defaults and found a few minor issues.  They are now
    Fernando> fixed in this new attached version.

Hi Fernando, this is a very useful submission that solves a recurrent
problem people have.  So useful, that I factored out the hard part
(getting the figure size right) into a separate function figaspect
which lives in matplotlib.figure so it could be reused in other
contexts.  The rest is easy, and lives in pylab.matshow, which calls
figaspect to do the heavy lifting.

A few comments 

 * you also need to be sure that the axes have the same width and
   height.  Since the width and height are expressed in fractions of
   the figure size, you can use any width and height you want and get
   the right answer as long as they are the same, since the figure
   width and height have the right aspect ration.  The default axes --
   subplot(111) -- have slightly different values for w and h.  This
   is fixed in CVS with 


    w,h = figaspect(arr)
    fig = figure(figsize=(w,h))
    ax = fig.add_axes([0.0, 0.05, 0.8, 0.8])


 * To get the labels on top, you do

   ax.xaxis.tick_top()  # this turns off tick bottom and turns on tick  top

   likewise, there are yaxis functions tick_left and tick_right.  This
   is also in CVS

  * I return a (fig, ax, im) tuple 

Give it a test drive and let me know what you think (pylab revision
1.34 or later in CVS)

JDH

[matplotlib-devel] Re: Routine to show matrices?

[<Fer...@co...>]

Quoting Fernando Perez <Fer...@co...>:

> a better name.  If you run the attached script via %run in a pylab session,
> it will display various arrays.

I just tested this code on a second box with different mpl defaults and found a
few minor issues.  They are now fixed in this new attached version.

Regards,

f

[matplotlib-devel] Routine to show matrices?

[Fernando P. <Fer...@co...>]

Hi all,

one of the things which has been bugging me in my recent conversion to mpl has 
been the lack of a good matrix display routine, which would plot the arrays 
with 'upper' origin but row-descending numbers, and which would satisfy the 
colliding constraints of no stretching, a window with the aspect ratio of the 
original array, all within reasonable window sizes.

imshow, figimage and pcolor all fail to do this in different ways.  So I'm 
attaching here a new routine, which roughly does what I want (with current CVS 
code, since it needs my new figure() patch).  I called it matshow, for lack of 
a better name.  If you run the attached script via %run in a pylab session, it 
will display various arrays.

I think the functionality is a good improvement over the existing 
image-related display routines (I need something like this from day 1 if I'm 
to use matplotlib), but I have some doubts about my implementation.  I poked 
around inside matplotlib for a while, trying to finesse this functionality 
into imshow() itself, but I got hopelessly lost.  Perhaps a better coder than 
me can make it work, if y'all think the functionality is worth having.

If people don't mind the implementation, I guess I'd like to see it included 
(perhaps with a better name, and after a good review by the experts).  The 
docstring contains some notes about improvements I don't know how to make, and 
the code is heavily commented.

Regards,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     Fernando> Finally, is this really a close bug?
> 
>     Fernando> In [20]: fig=plot(range(10))
> 
>     Fernando> In [21]: close(fig) ERROR: Unrecognized argument type to
>     Fernando> close
> 
> No, its a Fernando bug :-)
> 
> plot returns a list of Line2D instances.  Perhaps you mean
> 
>     fig = figure()
>     plot(range(10))
>     close(fig)

No, it's a John bug :)  I also had tried that, and this is what I get with TkAgg:

In [4]: fig=figure()

In [5]: plot(range(10))
Out[5]: [<matplotlib.lines.Line2D instance at 0x4110494c>]

In [6]: close(fig)
---------------------------------------------------------------------------
exceptions.AttributeError                            Traceback (most recent 
call last)

/home/fperez/code/python/pylab/<console>

/usr/lib/python2.3/site-packages/matplotlib/pylab.py in close(*args)
     611         elif isinstance(arg, Figure):
     612             for manager in _pylab_helpers.Gcf.get_all_fig_managers():
--> 613                 if manager.figure==arg:
     614                     _pylab_helpers.Gcf.destroy(manager.num)
     615         else:

AttributeError: FigureManagerTkAgg instance has no attribute 'figure'

Which is why I thought it might be the other way around.  So this may be a 
backend-specifig bug.

But still a bug :)

Cheers,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Finally, is this really a close bug?

    Fernando> In [20]: fig=plot(range(10))

    Fernando> In [21]: close(fig) ERROR: Unrecognized argument type to
    Fernando> close

No, its a Fernando bug :-)

plot returns a list of Line2D instances.  Perhaps you mean

    fig = figure()
    plot(range(10))
    close(fig)

It's OK, you can stop hitting yourself on the head now.  Actually, the
error message would have been more helpful if it reported the type,
which it will do in the next release.

JDH

Re: [matplotlib-devel] savefig problem in interactive mode

[Fernando P. <Fer...@co...>]

John Hunter wrote:

>And your figure num patch is already in....

Atop the docstring:

     figure(num = 1, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')

should read

     figure(num = None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')

since that is the real new default.

Cheers,

f

Re: [matplotlib-devel] savefig problem in interactive mode

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     Fernando> You have to carefully wire reload() calls into the
>     Fernando> proper backends.  It's doable, but doing it generically
>     Fernando> and with all the complex mpl backend machinery would
>     Fernando> likely take a bit of effort.
> 
> I have something in CVS that appears to work.  You can interactively
> switch backends in the pylab interface.  The caveat is that you lose
> all current figures when doing the switch (close('all') is called).
> With some work I could probably patch the current figure manager to
> work with multiple simultaneous backends but have no real interest in
> doing this now.  But it could serve as a basis for an ipython patch
> that allowed you to run as ps, eg something like the following
> 
> 
> def runps(fname):
> 
>     curr = pylab.rcParams['backend']
>     pylab.switch_backend('PS')
>     reload(pylab)
>     run(fname)
>     pylab.switch_backend(curr)
>     reload(pylab)
>        
> 
> or modify "run" to take a kwarg for the backend.

Modifying run is defitely easy.  How would you like to have this done?

run -backend=BACKEND foo.py

?

I can't use '-d' because that's already used by %run for something else.  Or 
we can have a different run altogether

runb BACKEND foo.py

which would be specific to backend switching, modeled on your runps() above.

Please note that I'll only add this if you really see it as an issue, I 
brought it up mostly for discussion, because I'm just getting my bearings 
around mpl.  If you feel it's best to leave things as they are, I'll go along.

>     Fernando> But for now, I'd rather see the tk close bug fixed and
>     Fernando> the figure() improvements I referred to in my other mail
>     Fernando> ;)
> 
> Well, the tk close bug should be a non-issue with CVS matplotlib.
> Just make sure you use the rc param (which is the default in CVS)
> 
> tk.pythoninspect  : False    # tk sets PYTHONINSEPCT

Sorry, but no.  I just updated CVS (and I did get my new patch, so it seems to 
be pretty up to date), set this variable in my .matplolibrc, and I still get this:


In [1]: run tkbug.py
*** I'm about to close figure 1, this will crash VTK!!! ***

Generic Warning: In 
/usr/local/installers/src/vtk/VTK/Rendering/vtkTkRenderWidget.cxx, line 633
A TkRenderWidget is being destroyed before it associated vtkRenderWindow is 
destroyed. This is very bad and usually due to the order in which objects are 
being destroyed. Always destroy the vtkRenderWindow before destroying the user 
interface components.

So the Tk window destruction bug remains...  Let me know if you make updates, 
I'll gladly test.  Or if you have TkAgg/Mayavi, this is again  the simple test 
case for reference:

planck[pylab]> cat tkbug.py
# This script crashes vtk, when run in an ipython -pylab session, with TkAgg
# as the default backend.

# The key is that the pylab.plot() call is made BEFORE the imv.surf call.  If
# I call imv.surf() first, it works fine.  Something in matplotlib is
# destroying windows it shouldn't.

from matplotlib import pylab
from mayavi.tools import imv

x= y = pylab.arange(256)
z= pylab.rand(256,256)

pylab.plot(range(10))
pylab.show()

imv.surf(x,y,z)

print "*** I'm about to close figure 1, this will crash VTK!!! *** \n"
pylab.close(1)
########################## EOF


> And your figure num patch is already in....

I saw that, great.  Many thanks, I really like it better this way.

Best,

f

Re: [matplotlib-devel] savefig problem in interactive mode

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> You have to carefully wire reload() calls into the
    Fernando> proper backends.  It's doable, but doing it generically
    Fernando> and with all the complex mpl backend machinery would
    Fernando> likely take a bit of effort.

I have something in CVS that appears to work.  You can interactively
switch backends in the pylab interface.  The caveat is that you lose
all current figures when doing the switch (close('all') is called).
With some work I could probably patch the current figure manager to
work with multiple simultaneous backends but have no real interest in
doing this now.  But it could serve as a basis for an ipython patch
that allowed you to run as ps, eg something like the following


def runps(fname):

    curr = pylab.rcParams['backend']
    pylab.switch_backend('PS')
    reload(pylab)
    run(fname)
    pylab.switch_backend(curr)
    reload(pylab)
       

or modify "run" to take a kwarg for the backend.

    Fernando> But for now, I'd rather see the tk close bug fixed and
    Fernando> the figure() improvements I referred to in my other mail
    Fernando> ;)

Well, the tk close bug should be a non-issue with CVS matplotlib.
Just make sure you use the rc param (which is the default in CVS)

tk.pythoninspect  : False    # tk sets PYTHONINSEPCT


And your figure num patch is already in....

JDH

Re: [matplotlib-devel] savefig problem in interactive mode

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     Fernando> Hi all, I have a question: in interactive (ipython
>     Fernando> -pylab) mode, a call to savefig('foo.eps') (either via
>     Fernando> %run or straight at the prompt) still pops up a GUI plot
>     Fernando> window.  If I run the same script from a system command
>     Fernando> line, the eps is made, but no GUI opens (what I consider
>     Fernando> the correct behavior).
> 
>     Fernando> Is this something that should be fixed in ipython or in
>     Fernando> matplotlib?
> 
> For the sake of clarity, let's consider the canonical script
> 
>     import matplotlib
>     matplotlib.use('PS')
>     from pylab import *
>     plot([1,2,3])
>     savefig('test.ps')
>     show()
> 
> When run from the shell, it does what you want -- makes a PS with no
> popup.  It fails in  ipython (pops up a window) because you have
> already selected a backend and all pylab commands are directed to that
> backend.
> 
> How to fix it?
> 
>  * ipython invokes an external python process to run each script.  Of
>    course you pay a performance hit here, and this would likely change
>    the meaning of the way run is meant to work (eg, are local ipython
>    shell vars available in a "run" script.

Not good.  If users want this, they can always just call !python foo.py and be 
done.  The whole point of %run is to benefit from enhanced tracebacks, debug 
hooks, variable access, etc.  Doing that across processes is basically impossible.

>  * provide better support for backend switching in matplotlib.  Eg,
>    allowing you at any time to call matplotlib.use.  Currently, this
>    only works before the import of pylab.  It may be possible to write
>    a pylab.use that simply rebinds the 4 backend functions:
>    new_figure_manager, error_msg, draw_if_interactive, show.  At the
>    end of a "run", you could simply do a
>    matplotlib.pylab.use(defaultBackend) to rebind.  run could be
>    enhanced to support backend switching
> 
>      run somescript.py -dPS
> 
>    much like one can do from the shell.
> 
>   You know more about python module reloading than I do.  How does one
>   force a module to reload, eg if I wanted to set the rc 'backend'
>   param and then do, eg
> 
>  
>     rcParams['backend'] = 'PS'
>     from backends import new_figure_manager, error_msg, draw_if_interactive, show
> 
> 
>   to get new symbols?

You have to carefully wire reload() calls into the proper backends.  It's 
doable, but doing it generically and with all the complex mpl backend 
machinery would likely take a bit of effort.

> There may be another way, but those two come to mind.  I'll mull it
> over.

It's not a big deal anyway, just something to think about.  My concern is that 
if I run from ipython (for testing/debugging) some big script which is wired 
to dump hundreds of EPS figures to a plots directory (yes, I do stuff like 
that with Gnuplot), my screen is going to be instantly flooded with hundreds 
of windows.


But for now, I'd rather see the tk close bug fixed and the figure() 
improvements I referred to in my other mail ;)

Cheers,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
>     Fernando> planck[pylab]> cat tkbug.py from matplotlib import pylab
> 
>     Fernando> pylab.plot(range(10)) pylab.show()
> 
>     Fernando> # Now I try to run this with plain python, no ipython in
>     Fernando> sight:
> 
> This is the result of adding
> 
>     if rcParams['tk.pythoninspect']:
>         os.environ['PYTHONINSPECT'] = '1'
> 
> to tkagg.  Comment out the pythoninspect line (or set the rc param
> accordingly) and see if makes a difference.  The pythoninspect thing
> appears to be required to make idle work in interactive mode, though
> it was introduced for other reasons I won't go into now.

OK, to summarize things with respect to this bug.  Indeed, commenting out the 
pythoninspect line solves the spurious prompt problem.  But it does nothing to 
the close('all') bug.  I managed to find a small test case to replicate the bug:

planck[pylab]> pylab

In [1]: cat tkbug.py
# This script crashes vtk, when run in an ipython -pylab session, with TkAgg
# as the default backend.

# The key is that the pylab.plot() call is made BEFORE the imv.surf call.  If
# I call imv.surf() first, it works fine.  Something in matplotlib is
# destroying windows it shouldn't.

from matplotlib import pylab
from mayavi.tools import imv

x = y = pylab.arange(256)
z = pylab.rand(256,256)

pylab.plot(range(10))
pylab.show()

imv.surf(x,y,z)

print "*** I'm about to close figure 1, this will crash VTK!!! *** \n"
pylab.close(1)
########################## EOF

In [2]: run tkbug
*** I'm about to close figure 1, this will crash VTK!!! ***

Generic Warning: In 
/usr/local/installers/src/vtk/VTK/Rendering/vtkTkRenderWidget.cxx, line 633
A TkRenderWidget is being destroyed before it associated vtkRenderWindow is 
destroyed. This is very bad and usually due to the order in which objects are 
being destroyed. Always destroy the vtkRenderWindow before destroying the user 
interface components.


Somehow, it seems that the Tk figure manager is messing with windows it 
shouldn't touch.

On a related note, I've been playing with some things in matplotlib which 
require looping through figure lists, making new figures with guaranteed new 
numbers, etc.  I'd like to propose a change.

I'd like figure() to allow calling wit None as the num argument.  If num is 
None, it would produce a guaranteed new figure window, with a number equal to 
the currently highest + 1.  This would make it easy, amongst other things, to 
write code which displays arrays with a proper aspect ratio by wrapping 
imshow() or figimage().  I can currently make such a guaranteed new number by 
using the following as a patch to pylab.py:


     if num==0:
         error_msg('Figure number can not be 0.\n' + \
                   'Hey, give me a break, this is matlab(TM) compatability')

     # NEW CODE
     if num is None:
         allnums = [f.num for f in _pylab_helpers.Gcf.get_all_fig_managers()]
         if allnums:
             num = max(allnums) + 1
         else:
             num = 1
     # /NEW CODE

I also think the default value for num should be None instead of 1.  This 
would allow you to make the following simple, clean use for building new plots:

planck[~]> pylab

In [1]: plot(range(10))
Out[1]: [<matplotlib.lines.Line2D instance at 0x412ccd8c>]

In [2]: figure()
Out[2]: <matplotlib.figure.Figure instance at 0x412ccd2c>

In [3]: plot(range(20))
Out[3]: [<matplotlib.lines.Line2D instance at 0x410d320c>]

In [4]: figure()
Out[4]: <matplotlib.figure.Figure instance at 0x410d32ac>

In [5]: plot(range(30))
Out[5]: [<matplotlib.lines.Line2D instance at 0x4115b02c>]


without never having to worry about manually managing figure number lists 
(unless you explicitly want to, which you still can).


Finally, is this really a close bug?

In [20]: fig=plot(range(10))

In [21]: close(fig)
ERROR: Unrecognized argument type to close

It sounds to me from reading the docstring like this should work, but maybe I 
just misunderstood things...

Regards,

f

Re: [matplotlib-devel] savefig problem in interactive mode

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Hi all, I have a question: in interactive (ipython
    Fernando> -pylab) mode, a call to savefig('foo.eps') (either via
    Fernando> %run or straight at the prompt) still pops up a GUI plot
    Fernando> window.  If I run the same script from a system command
    Fernando> line, the eps is made, but no GUI opens (what I consider
    Fernando> the correct behavior).

    Fernando> Is this something that should be fixed in ipython or in
    Fernando> matplotlib?

For the sake of clarity, let's consider the canonical script

    import matplotlib
    matplotlib.use('PS')
    from pylab import *
    plot([1,2,3])
    savefig('test.ps')
    show()

When run from the shell, it does what you want -- makes a PS with no
popup.  It fails in  ipython (pops up a window) because you have
already selected a backend and all pylab commands are directed to that
backend.

How to fix it?

 * ipython invokes an external python process to run each script.  Of
   course you pay a performance hit here, and this would likely change
   the meaning of the way run is meant to work (eg, are local ipython
   shell vars available in a "run" script.

 * provide better support for backend switching in matplotlib.  Eg,
   allowing you at any time to call matplotlib.use.  Currently, this
   only works before the import of pylab.  It may be possible to write
   a pylab.use that simply rebinds the 4 backend functions:
   new_figure_manager, error_msg, draw_if_interactive, show.  At the
   end of a "run", you could simply do a
   matplotlib.pylab.use(defaultBackend) to rebind.  run could be
   enhanced to support backend switching

     run somescript.py -dPS

   much like one can do from the shell.

  You know more about python module reloading than I do.  How does one
  force a module to reload, eg if I wanted to set the rc 'backend'
  param and then do, eg

 
    rcParams['backend'] = 'PS'
    from backends import new_figure_manager, error_msg, draw_if_interactive, show


  to get new symbols?

There may be another way, but those two come to mind.  I'll mull it
over.

    Fernando> ps. Yes, John, I've finally started to use matplotlib
    Fernando> for my own work.  Brace yourself, I'm compiling a pretty
    Fernando> hefty list of things to do.  I hope you don't plan on
    Fernando> sleeping much in the coming months ;)

Well, I knew it was coming....  Stress tests are usually a good
thing.  Plus, I'm sure you can't do anything to interrupt my sleep
that my 3 kids haven't already mastered!

JDH

[matplotlib-devel] savefig problem in interactive mode

[Fernando P. <Fer...@co...>]

Hi all,

I have a question:  in interactive (ipython -pylab) mode, a call to 
savefig('foo.eps') (either via %run or straight at the prompt) still pops up a 
GUI plot window.  If I run the same script from a system command line, the eps 
is made, but no GUI opens (what I consider the correct behavior).

Is this something that should be fixed in ipython or in matplotlib?

Cheers,

f.

ps. Yes, John, I've finally started to use matplotlib for my own work.  Brace 
yourself, I'm compiling a pretty hefty list of things to do.  I hope you don't 
plan on sleeping much in the coming months ;)

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
>     Fernando> planck[pylab]> cat tkbug.py from matplotlib import pylab
> 
>     Fernando> pylab.plot(range(10)) pylab.show()
> 
>     Fernando> # Now I try to run this with plain python, no ipython in
>     Fernando> sight:
> 
> This is the result of adding
> 
>     if rcParams['tk.pythoninspect']:
>         os.environ['PYTHONINSPECT'] = '1'
> 
> to tkagg.  Comment out the pythoninspect line (or set the rc param
> accordingly) and see if makes a difference.  The pythoninspect thing
> appears to be required to make idle work in interactive mode, though
> it was introduced for other reasons I won't go into now.

OK, my report from this afternoon was written in the office, where I have mpl 
0.71.  I'm now home, and my laptop runs 0.70.  Here, I don't see the problem 
at all.  I can't test here the big code where I saw the massive mayavi 
breakdown, because that was written by my officemate.  But at least I can 
confirm that 0.70, whose show() looks like:

def show():
     """
     Show all the figures and enter the gtk mainloop

     This should be the last line of your script
     """
     for manager in Gcf.get_all_fig_managers():
         manager.show()
     import matplotlib
     matplotlib.interactive(True)
     #os.environ['PYTHONINSPECT'] = '1'
     if show._needmain:
         Tk.mainloop()
         show._needmain = False


does not show the spurious prompt thingie (note the PYTHONINSPECT thingie is 
commented out).  I'll try to make this change to 0.71 and test the larger code 
with the mayavi problems, but it may be a few days before I can do that.

HTH,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:
    Fernando> planck[pylab]> cat tkbug.py from matplotlib import pylab

    Fernando> pylab.plot(range(10)) pylab.show()

    Fernando> # Now I try to run this with plain python, no ipython in
    Fernando> sight:

This is the result of adding

    if rcParams['tk.pythoninspect']:
        os.environ['PYTHONINSPECT'] = '1'

to tkagg.  Comment out the pythoninspect line (or set the rc param
accordingly) and see if makes a difference.  The pythoninspect thing
appears to be required to make idle work in interactive mode, though
it was introduced for other reasons I won't go into now.

The rc param is only in CVS -- you may just want to comment out the
whole bit and see if it helps.

Let me know...

JDH

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

John Hunter wrote:

> Just a guess,
> 
> The problem may be arising when the backend tries to quit when the
> total figure count reaches zero.  The relevant backend_tkagg section
> is
> 
>     def destroy(self, *args):
>         if Gcf.get_num_fig_managers()==0 and not matplotlib.is_interactive():
>             if self.window is not None:
>                 self.window.quit()
>         if self.window is not None:
>             #print 'calling window destroy'
>             self.window.destroy()
>         self.window = None
> 
> Try playing with this function and see if you can deduce where the
> problem is.

Sorry, but no luck.  I tried a few simple things and made no progress, and I 
really can't spend more time on this right now.  But I did find something 
bizarre.  Consider:

planck[pylab]> cat tkbug.py
from matplotlib import pylab

pylab.plot(range(10))
pylab.show()

#
Now I try to run this with plain python, no ipython in sight:

planck[pylab]> python tkbug.py
 >>>


The '>>>' prompt came in after I closed the plot window, and this is exactly 
the same thing I see with ipython: I get tossed into a naked python prompt, 
which is half-broken.  With ipython, you get all sorts of weird errors related 
to ipython having been torn down already.  With plain python, the only obvious 
sign of trouble is that readline is broken (^]]A instead of up-arrow, etc.)

So something fishy is going on there, but I really don't have time to track it 
down.  For now, I'll live with my hackish 'window sentinel' approach, ugly as 
it may be.

Cheers,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Fernando Perez wrote:
    >> OK, a bit more info.  It turns out that the crash happens
    >> whenever the _first_ figure window is deleted with a
    >> close(fignum) command.  So in order to really block the
    >> problem, I had to create a dummy 'sentinel' window, numbered
    >> -666, for which close() is never called.  It's OK to close this
    >> window via the window manager by clicking on its close button,
    >> but pylab.close() must NEVER be called on it.  The current code
    >> looks like this:

    Fernando> More info, sorry about the noise.  It's NOT OK to close
    Fernando> the sentinel in any way whatsoever, even via the window
    Fernando> manager.  If this window is closed, through any
    Fernando> mechanism, Tk/VTK is hosed.

Just a guess,

The problem may be arising when the backend tries to quit when the
total figure count reaches zero.  The relevant backend_tkagg section
is

    def destroy(self, *args):
        if Gcf.get_num_fig_managers()==0 and not matplotlib.is_interactive():
            if self.window is not None:
                self.window.quit()
        if self.window is not None:
            #print 'calling window destroy'
            self.window.destroy()
        self.window = None

Try playing with this function and see if you can deduce where the
problem is.

JDH

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

Fernando Perez wrote:

> OK, a bit more info.  It turns out that the crash happens whenever the _first_ 
> figure window is deleted with a close(fignum) command.  So in order to really 
> block the problem, I had to create a dummy 'sentinel' window, numbered -666, 
> for which close() is never called.  It's OK to close this window via the 
> window manager by clicking on its close button, but pylab.close() must NEVER 
> be called on it.  The current code looks like this:

More info, sorry about the noise.  It's NOT OK to close the sentinel in any 
way whatsoever, even via the window manager.  If this window is closed, 
through any mechanism, Tk/VTK is hosed.

This is what you get on screen if you close the very first Tk figure window, 
once MayaVi has run and made a figure:

In [8]: Generic Warning: In
/usr/local/installers/src/vtk/VTK/Rendering/vtkTkRenderWidget.cxx, line 633
A TkRenderWidget is being destroyed before it associated vtkRenderWindow is
destroyed. This is very bad and usually due to the order in which objects are
being destroyed. Always destroy the vtkRenderWindow before destroying the user
interface components.


Any attempt to use mayavi afterwards produces this traceback:

---------------------------------------------------------------------------
_tkinter.TclError                                  Traceback (most recent call
last)

/home/sandberg/WavePropagation3D/Codes/Test/wave1DTEST.py
     269 mm.title('Non-projected D2 wrt prolates')
     270
--> 271 imv.surf(range(nnod),range(nnod),255*improc.mat2gray(D2p))
     272
     273

/usr/lib/python2.3/site-packages/mayavi/tools/imv.py in surf(x, y, z, warp,
scale, viewer, f_args, f_keyw)
     267     # do the mayavi stuff.
     268     if not viewer:
--> 269         v = mayavi.mayavi()
     270     else:
     271         v = viewer

/usr/lib/python2.3/site-packages/mayavi/Main.py in mayavi(geometry)
    1826     t = Tkinter.Toplevel (r)
    1827     t.withdraw ()
-> 1828     app = MayaViTkGUI (t, geometry)
    1829     return app
    1830

/usr/lib/python2.3/site-packages/mayavi/Main.py in __init__(self, master,
geometry)
     918         self.renwin_frame = Tkinter.Frame (master_f)
     919         self.renwin_frame.pack (side='left', fill='both', expand=1)
--> 920         self.renwin = Misc.RenderWindow.RenderWindow (self.renwin_frame)
     921         self.renwin.Render ()
     922

/usr/lib/python2.3/site-packages/mayavi/Misc/RenderWindow.py in __init__(self,
master)
      86         else:
      87             tkw = vtkRenderWidget.vtkTkRenderWidget (self.frame,
width=600,
---> 88                                                      height=505)
      89         self.tkwidget = tkw
      90         self.tkwidget.pack (expand='true',fill='both')

/usr/local/lib/python2.3/site-packages/vtk_python/vtk/tk/vtkTkRenderWidget.py in
__init__(self, master, cnf, **kw)
     104
     105         kw['rw'] = renderWindow.GetAddressAsString("vtkRenderWindow")
--> 106         Tkinter.Widget.__init__(self, master, 'vtkTkRenderWidget', cnf,
kw)
     107
     108         self._CurrentRenderer = None

/usr/src/build/475206-i386/install/usr/lib/python2.3/lib-tk/Tkinter.py in
__init__(self, master, widgetName, cnf, kw, extra)
    1833                 classes.append((k, cnf[k]))
    1834                 del cnf[k]
-> 1835         self.tk.call(
    1836             (widgetName, self._w) + extra + self._options(cnf))
    1837         for k, v in classes:

TclError: invalid command name "vtkTkRenderWidget"


More interestingly, this leaves python in some very strange state.  If you 
close ipyhton, instead of a system prompt you get the '>>>' python prompt, but 
you can't execute _anything_ there.  Even a simple '1+1' fails, all you can do 
is quit.  So basically the closing of that first matplotlib figure window is 
destroying enough in the python internals to render it completely unusable.

Regards,

f

Re: [matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

Fernando Perez wrote:
> Hi all,
> 
> we've just run into a nasty problem with the TkAgg backend if close('all') is 
> called.  In our setup, we use ipython+pylab with TkAgg because we also need 
> MayaVi to be active, and the GTK/WX backends block Tk windows.  But if a call 
> is made to close('all'), matplotlib closes not only all of its own windows, 
> but also it destroys the MayaVi window in some very nasty way.  The VTK 
> wrapper complains loudly about improper deletions, and afterwards running any 
> mayavi code is impossible (Tcl/Tk errors come from inside python itself).
> 
> The problem is that matplotlib should not be touching any windows that don't 
> belong to it.  I quickly wrote the following wrapper code to use here to work 
> around this bug:

OK, a bit more info.  It turns out that the crash happens whenever the _first_ 
figure window is deleted with a close(fignum) command.  So in order to really 
block the problem, I had to create a dummy 'sentinel' window, numbered -666, 
for which close() is never called.  It's OK to close this window via the 
window manager by clicking on its close button, but pylab.close() must NEVER 
be called on it.  The current code looks like this:

# Temporary hack around a matplotlib figure closing bug
import matplotlib.pylab as mm

try:
     mm.all_figures
except AttributeError:
     mm.all_figures = []

# hack: sentinel to prevent pylab from destroying tk windows.  NEVER make a 
-666 figure!
mm.figure(-666)

def figure(num=1,*args,**kw):
     """Wrapper around mm.figure which updates a global list of held figures."""
     if num == -666:
         raise ValueError,'-666 is an internal sentinel, do not use for your 
figures'
     mm.figure(num,*args,**kw)
     mm.all_figures.append(num)

def close(*args):
     """Close all open figures managed by our figure() wrapper."""

     if len(args)==1 and args[0]=='all':
         print 'Closing figures:',mm.all_figures
         map(mm.close,mm.all_figures)
         mm.all_figures = []
     else:
         mm.close(*args)


Regards,

f

[matplotlib-devel] Bugs in close('all') with the TkAgg backend

[Fernando P. <Fer...@co...>]

Hi all,

we've just run into a nasty problem with the TkAgg backend if close('all') is 
called.  In our setup, we use ipython+pylab with TkAgg because we also need 
MayaVi to be active, and the GTK/WX backends block Tk windows.  But if a call 
is made to close('all'), matplotlib closes not only all of its own windows, 
but also it destroys the MayaVi window in some very nasty way.  The VTK 
wrapper complains loudly about improper deletions, and afterwards running any 
mayavi code is impossible (Tcl/Tk errors come from inside python itself).

The problem is that matplotlib should not be touching any windows that don't 
belong to it.  I quickly wrote the following wrapper code to use here to work 
around this bug:

try:
     mm.all_figures
except AttributeError:
     mm.all_figures = []

def figure(num=1):
     """Wrapper around mm.figure which updates a global list of held figures."""
     mm.figure(num)
     mm.all_figures.append(num)

def close_all():
     """Close all open figures managed by our figure() wrapper."""

     print 'Closing figures:',mm.all_figures
     map(mm.close,mm.all_figures)
     mm.all_figures = []


but it would be nice to see the TkAgg backend do the right thing.

I should also mention that when I first wrote my wrapper code, I used in 
figure() the following:

mm.all_figures.append(mm.figure(num))

to append the actual figure handles.  This didn't work because the close() 
call to a handle seems to be also broken.  I worked around this other bug by 
using figure numbers instead of handles.

While we're at it, I think it would be nice to extend the close() syntax to 
allow a sequence of integers or figure handles to be passed to it, so that one 
could simply say

close([1,3,5,21,101])

to only close those windows, or the same with their respective figure handles 
(nicely stored by ipython in the _NN variables).

Regards,

f

Re: [matplotlib-devel] First cut on a QtAgg backend

[John H. <jdh...@ac...>]

>>>>> "Sigve" == Sigve Tjora <si...@tj...> writes:

    Ted> All I really interested in is seeing the QtAgg front end in
    Ted> matplotlib as a supported component (which we'd be happy to
    Ted> help maintain) so whatever gets us there the quickest sounds
    Ted> good to me.

    Sigve> I am in the same position as you; I just want a Qt-backend
    Sigve> for Matplotlib.

Well, actually it looks like Ted wants a front end and Sigve wants a
backend.  Let's hope these differences are not irreconcilable!

<wink>
JDH

Re: [matplotlib-devel] First cut on a QtAgg backend

[Sigve T. <si...@tj...>]

Hi again Ted,
As far as I can tell, we are using the same method to connect between 
Agg and Qt. You are doing it from C++, while I am doing it with python 
only. Another difference is that I subclass QWidget instead of QLabel. 
Can you please elaborate on why have you choosen QLabel over QWidget? I 
don’t see what QLabel gives you that you don’t have in QWidget. Perhaps 
you get double buffering? I am just curious and trying to learn… I don’t 
think there should be any major speed penalties by doing it all from 
python, but I haven’t benchmarked anything yet.

Please do incorporate whatever you need from my code into your version. 
I am not the right person to decide what should go into Matplotlib, any 
solution is fine by me. I am in the same position as you; I just want a 
Qt-backend for Matplotlib.

Good luck!

Best regards,
Sigve I


Ted Drain skrev:

> We've also been working on a Qt front end. It's been going slow 
> because we've only been able to spend about 1 oerson-day per week on 
> it. But, starting this week I have someone full time on it. He's 
> gotten a basic widget working using the same organization that the GTK 
> code uses. Sigve's code has a lot more functionality in the tool bars 
> than ours right now.
>
> Here's where we're at:
> - Finished C++ routine to convert AGG -> QPixmap (in /src/_qtagg.cpp)
> - Finished low level widget to display the AGG pixmap using a widget 
> derived from QLabel (in ./lib/matplotlib/backends/backend_qtagg.py)
> - Working on a higher level window widget to display the plot and the 
> tool bar. We implemented the tool bar as a Qt tool bar which allows 
> you drag it around, detach it, etc.
>
> The guy working on it doesn't think it would take more than a day or 
> two to incorporate Sigve's code for the tool bar handling into our 
> code. We're close enough to finishing that it probably doesn't make a 
> lot of sense to try and set up a collaboration right now but I'm open 
> to that possibility if that's what people want. All I really 
> interested in is seeing the QtAgg front end in matplotlib as a 
> supported component (which we'd be happy to help maintain) so whatever 
> gets us there the quickest sounds good to me.
>
> Ted
>
> PS: here's the code we used to do the AGG->QPixmap conversion. It may 
> not be the fastest way to do it but it does work which is all we were 
> after for the first cut.
>
> PyQObject* pyDrawable = static_cast< PyQObject* >( args[0].ptr() );
> QLabel* label = static_cast< QLabel* >( pyDrawable->obj );
>
> RendererAgg* aggRenderer = static_cast< RendererAgg* >( args[1].ptr() );
>
> unsigned int width = aggRenderer->get_width();
> unsigned int height = aggRenderer->get_height();
>
> QImage image( aggRenderer->pixBuffer, width, height, 32, 0, 256,
> QImage::LittleEndian );
> QPixmap pixmap;
> pixmap.convertFromImage( image, QPixmap::Color );
>
> label->setPixmap( pixmap );

Re: [matplotlib-devel] First cut on a QtAgg backend

[Sigve T. <si...@tj...>]

John Hunter wrote:

>    Sigve> That probably leaves us with the OSX problems, though. Ahh,
>    Sigve> fixed in CVS, ok then.
>
>I'm lost here -- as far as I know the endianess problem *was not*
>fixed in CVS.  I did add an agg method tostring_bgra to CVS which
>appears to fix the reversed color problem.  I haven't had a chance to
>test this on OSX since my powerbook is at home...
>
>  
>
Sorry about that, it came out wrong. Yes the inversed color could be 
fixed (I have not tested the CVS-version yet), the endiannes is not. You 
could try to change the endian in FigureCanvasQtAgg.draw()
        self.qimage = qt.QImage(self.stringBuffer,
                                self.renderer.width,
                                self.renderer.height,
                                32,
                                None,
                                0,
                                qt.QImage.IgnoreEndian,
                                )

to

        self.qimage = qt.QImage(self.stringBuffer,
                                self.renderer.width,
                                self.renderer.height,
                                32,
                                None,
                                0,
                                qt.QImage.BigEndian,
                                )

Or perhaps qt.QImage.LittleEndian if that is the way the conversion 
should go.

Good luck!

Regards,
Sigve

Re: [matplotlib-devel] First cut on a QtAgg backend

[John H. <jdh...@ac...>]

>>>>> "Ted" == Ted Drain <ted...@jp...> writes:

    Ted> We've also been working on a Qt front end.  It's been going
    Ted> slow because we've only been able to spend about 1 oerson-day
    Ted> per week on it.  But, starting this week I have someone full
    Ted> time on it.  He's gotten a basic widget working using the
    Ted> same organization that the GTK code uses.  Sigve's code has a
    Ted> lot more functionality in the tool bars than ours right now.

Great -- it looks like there will be a good opportunity to merge the
best features from both backends -- two functional qt backends is
definitely an improvement over what we had a week ago!  Particularly
useful might be to retain the current implementation which uses string
methods to transfer the agg image to the qt canvas as a fallback for
your extension code method.  That way, people who have trouble getting
the extension code version working can always fallback to something
slower that still works.

    Ted> The guy working on it doesn't think it would take more than a
    Ted> day or two to incorporate Sigve's code for the tool bar
    Ted> handling into our code.  We're close enough to finishing that
    Ted> it probably doesn't make a lot of sense to try and set up a
    Ted> collaboration right now but I'm open to that possibility if
    Ted> that's what people want.  All I really interested in is
    Ted> seeing the QtAgg front end in matplotlib as a supported
    Ted> component (which we'd be happy to help maintain) so whatever
    Ted> gets us there the quickest sounds good to me.

Whatever works for you and Sigve works for me.  My guess is that it
will be easy to cross-pollinate from one implementation to the other.

Thanks!
JDH