matplotlib-users

[Matplotlib-users] Interactive gtk console. Fixed (I think)

[Fernando P. <Fer...@co...>]

Hi all,

thanks to Antoon Pardon from the pygtk mailing list, I now have what appears 
to be a fully working interactive python console for gtk/matplotlib usage, 
which does NOT lock up when things like os.system('gv foo.eps &') are emitted.

I've attached it here, and it would be great if people could take it for a 
spin and let me know how it works.  Just make it executable and run

./pyint-gtk.py -mplot

and it preloads matplotlib for you, honoring your .matplotlibrc settings 
(except that it forces matplotlib to interactive mode).

If this resists a few days of testing, I'll port the mods Anton made to the 
real ipython.

Best regards,

f.

ps. John, I think we almost have this nailed :)

Re: [Matplotlib-users] Interactive gtk console. Fixed (I think)

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> thanks to Antoon Pardon from the pygtk mailing list, 

I told you those guys were brainy :-)

    Fernando> ps. John, I think we almost have this nailed :)

Don't tempt the fates -- I got an X11 freeze.  But I think I fixed it.

    def pre_interact(self):
        """Initialize matplotlib before user interaction begins"""

--->    import matplotlib.matlab

I don't think you want this, especially not here.  Remember that all
the matplotlib.use and matplotlib.interactive calls should be made
*before* importing matplotlib.matlab (which is when the actual backend
is imported)

        push = self.shell.push
        # Code to execute in user's namespace
        lines = ["import matplotlib",
--->             "matplotlib.use('GTKAgg')",
                 "matplotlib.interactive(1)",
                 "import matplotlib.matlab as matlab",
                 "from matplotlib.matlab import *"]
   
You might make the additional caveat in the docs about honoring rc
settings that the default backend is overridden.

        map(push,lines)

        # turn off rendering until end of script
--->     matplotlib.matlab.interactive = 0

I'm pretty sure you mean matplotlib.interactive(0) -- ie, it's a
function, not a variable, and it isn't in the matlab interface module.
And don't you want to turn off interaction only if they load a file,
and then turn it back on?  Ie,

        # Execute file if given.
        if len(sys.argv)>1:
--->        import matplotlib
--->        matplotlib.interactive(0) # turn off interaction
            fname = sys.argv[1]
            try:
                inFile = file(fname, 'r')
            except IOError:
                print '*** ERROR *** Could not read file <%s>' % fname
            else:
                print '*** Executing file <%s>:' % fname
                for line in inFile:
                    if line.lstrip().find('show()')==0: continue
                    print '>>', line,
                    push(line)
                inFile.close()
--->        matplotlib.interactive(1)   # turn on interaction


But with these changes, it works great!  Here is my modified version

#!/usr/bin/env python
"""Multithreaded interactive interpreter with GTK and Matplotlib support.

Usage:

  pyint-gtk.py -> starts shell with gtk thread running separately

  pyint-gtk.py -mplot [filename] -> initializes matplotlib, optionally running
  the named file.  The shell starts after the file is executed.

Threading code inspired by:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109, by Brian
McErlean and John Finlay.

Matplotlib support taken from interactive.py in the matplotlib distribution.

Also borrows liberally from code.py in the Python standard library."""

__author__ = "Fernando Perez <Fer...@co...>"

import sys
import code
import threading

import pygtk
pygtk.require("2.0")
import gtk

try:
    import readline
except ImportError:
    has_readline = False
else:
    has_readline = True

class MTConsole(code.InteractiveConsole):
    """Simple multi-threaded shell"""

    def __init__(self,on_kill=None,*args,**kw):
        code.InteractiveConsole.__init__(self,*args,**kw)
        self.code_to_run = None
        self.ready = threading.Condition()
        self._kill = False
        if on_kill is None:
            on_kill = []
        # Check that all things to kill are callable:
        for _ in on_kill:
            if not callable(_):
                raise TypeError,'on_kill must be a list of callables'
        self.on_kill = on_kill
        # Set up tab-completer
        if has_readline:
            import rlcompleter
            try:  # this form only works with python 2.3
                self.completer = rlcompleter.Completer(self.locals)
            except: # simpler for py2.2
                self.completer = rlcompleter.Completer()
                
            readline.set_completer(self.completer.complete)
            # Use tab for completions
            readline.parse_and_bind('tab: complete')
            # This forces readline to automatically print the above list when tab
            # completion is set to 'complete'.
            readline.parse_and_bind('set show-all-if-ambiguous on')
            # Bindings for incremental searches in the history. These searches
            # use the string typed so far on the command line and search
            # anything in the previous input history containing them.
            readline.parse_and_bind('"\C-r": reverse-search-history')
            readline.parse_and_bind('"\C-s": forward-search-history')

    def runsource(self, source, filename="<input>", symbol="single"):
        """Compile and run some source in the interpreter.

        Arguments are as for compile_command().

        One several things can happen:

        1) The input is incorrect; compile_command() raised an
        exception (SyntaxError or OverflowError).  A syntax traceback
        will be printed by calling the showsyntaxerror() method.

        2) The input is incomplete, and more input is required;
        compile_command() returned None.  Nothing happens.

        3) The input is complete; compile_command() returned a code
        object.  The code is executed by calling self.runcode() (which
        also handles run-time exceptions, except for SystemExit).

        The return value is True in case 2, False in the other cases (unless
        an exception is raised).  The return value can be used to
        decide whether to use sys.ps1 or sys.ps2 to prompt the next
        line.
        """
        try:
            code = self.compile(source, filename, symbol)
        except (OverflowError, SyntaxError, ValueError):
            # Case 1
            self.showsyntaxerror(filename)
            return False

        if code is None:
            # Case 2
            return True

        # Case 3
        # Store code in self, so the execution thread can handle it
        self.ready.acquire()
        self.code_to_run = code
        self.ready.wait()  # Wait until processed in timeout interval
        self.ready.release()

        return False

    def runcode(self):
        """Execute a code object.

        When an exception occurs, self.showtraceback() is called to display a
        traceback."""

        self.ready.acquire()
        if self._kill:
            print 'Closing threads...',
            sys.stdout.flush()
            for tokill in self.on_kill:
                tokill()
            print 'Done.'

        if self.code_to_run is not None:
            self.ready.notify()
            code.InteractiveConsole.runcode(self,self.code_to_run)

        self.code_to_run = None
        self.ready.release()
        return True

    def kill (self):
        """Kill the thread, returning when it has been shut down."""
        self.ready.acquire()
        self._kill = True
        self.ready.release()

class GTKInterpreter(threading.Thread):
    """Run a gtk mainloop() in a separate thread.
    Python commands can be passed to the thread where they will be executed.
    This is implemented by periodically checking for passed code using a
    GTK timeout callback.
    """
    TIMEOUT = 100 # Milisecond interval between timeouts.
    
    def __init__(self,banner=None):
        threading.Thread.__init__(self)
        self.banner = banner
        self.shell = MTConsole(on_kill=[gtk.mainquit])

    def run(self):
        self.pre_interact()
        self.shell.interact(self.banner)
        self.shell.kill()

    def mainloop(self):
        self.start()
        gtk.timeout_add(self.TIMEOUT, self.shell.runcode)
        try:
            if gtk.gtk_version[0] >= 2:
                gtk.threads_init()
        except AttributeError:
            pass
        gtk.mainloop()
        self.join()

    def pre_interact(self):
        """This method should be overridden by subclasses.

        It gets called right before interact(), but after the thread starts.
        Typically used to push initialization code into the interpreter"""
        
        pass

class MatplotLibInterpreter(GTKInterpreter):
    """Threaded interpreter with matplotlib support."""

    def __init__(self,banner=None):
        banner = """\nWelcome to matplotlib, a matlab-like python environment.
    help(matlab)   -> help on matlab compatible commands from matplotlib.
    help(plotting) -> help on plotting commands.
    """
        GTKInterpreter.__init__(self,banner)
        
    def pre_interact(self):
        """Initialize matplotlib before user interaction begins"""

        push = self.shell.push
        # Code to execute in user's namespace
        lines = ["import matplotlib",
                 "matplotlib.use('GTKAgg')",
                 "matplotlib.interactive(1)",
                 "import matplotlib.matlab as matlab",
                 "from matplotlib.matlab import *"]

        map(push,lines)
        
        

        # Execute file if given.
        if len(sys.argv)>1:
            import matplotlib
            matplotlib.interactive(0) # turn off interaction
            fname = sys.argv[1]
            try:
                inFile = file(fname, 'r')
            except IOError:
                print '*** ERROR *** Could not read file <%s>' % fname
            else:
                print '*** Executing file <%s>:' % fname
                for line in inFile:
                    if line.lstrip().find('show()')==0: continue
                    print '>>', line,
                    push(line)
                inFile.close()
            matplotlib.interactive(1)   # turn on interaction

if __name__ == '__main__':
    # Quick sys.argv hack to extract the option and leave filenames in sys.argv.
    # For real option handling, use optparse or getopt.
    if len(sys.argv) > 1 and sys.argv[1]=='-mplot':
        sys.argv = [sys.argv[0]]+sys.argv[2:]
        MatplotLibInterpreter().mainloop()
    else:
        GTKInterpreter().mainloop()

Re: [Matplotlib-users] Interactive gtk console. Fixed (I think)

[Fernando P. <Fer...@co...>]

John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
>     Fernando> ps. John, I think we almost have this nailed :)
> 
> Don't tempt the fates -- I got an X11 freeze.  But I think I fixed it.

Hey, I said _almost_ :)

>     def pre_interact(self):
>         """Initialize matplotlib before user interaction begins"""
> 
> --->    import matplotlib.matlab
> 
> I don't think you want this, especially not here.  Remember that all
> the matplotlib.use and matplotlib.interactive calls should be made
> *before* importing matplotlib.matlab (which is when the actual backend
> is imported)

Noted.

>         push = self.shell.push
>         # Code to execute in user's namespace
>         lines = ["import matplotlib",
> --->             "matplotlib.use('GTKAgg')",
>                  "matplotlib.interactive(1)",
>                  "import matplotlib.matlab as matlab",
>                  "from matplotlib.matlab import *"]
>    
> You might make the additional caveat in the docs about honoring rc
> settings that the default backend is overridden.

Well, actually I think we should honor the user's rc choice, don't you?  In 
fact, the ipython version does this, so the equivalent here would be just to 
comment out the line you highlighted.

> 
>         map(push,lines)
> 
>         # turn off rendering until end of script
> --->     matplotlib.matlab.interactive = 0
> 
> I'm pretty sure you mean matplotlib.interactive(0) -- ie, it's a
> function, not a variable, and it isn't in the matlab interface module.
> And don't you want to turn off interaction only if they load a file,
> and then turn it back on?  Ie,
[...]

Ok. In fact, in the ipython-based version you and I have floated, this was 
alreadey fixed (by you).  But I'll double-check as soon as I can get that done 
(probably this evening).  I asked the pygtk crowd for help with the simpler 
pyint-gtk, which did NOT have our recent work, because it made it much easier 
for them to test without asking them to download CVS ipython.


> But with these changes, it works great!  Here is my modified version

Cool.  I'll integrate Antoon's and your fixes into the 'real' (read 
ipython-based) code, and we'll play with that a bit.  Do you foresee any 
problems with the WX backend?  Right now I have global pygtk calls, but I 
suppose those should be done only by the matplotlib shell class IF Gtk is the 
chosen backend.  That part of the design needs a bit of polishing, and I may 
have to leave it til we meet at scipy'04.  I still have to put my talks 
together for the meeting, so my hacking time will be very limited for the next 
two weeks.

But we _are_ close :)

Best,

f

Re: [Matplotlib-users] Interactive gtk console. Fixed (I think)

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:


    >> push = self.shell.push # Code to execute in user's namespace
    >> lines = ["import matplotlib",
    ---> "matplotlib.use('GTKAgg')",
    >> "matplotlib.interactive(1)", "import matplotlib.matlab as
    >> matlab", "from matplotlib.matlab import *"] You might make the
    >> additional caveat in the docs about honoring rc settings that
    >> the default backend is overridden.

    Fernando> Well, actually I think we should honor the user's rc
    Fernando> choice, don't you?  In fact, the ipython version does
    Fernando> this, so the equivalent here would be just to comment
    Fernando> out the line you highlighted.

Yes, and along those lines for the GTK shell, we should use
rcParams['backend'] to detect GTK versus GTKAgg and use whichever is
in the rc file in the call to matplotlib.use.


    Fernando> Cool.  I'll integrate Antoon's and your fixes into the
    Fernando> 'real' (read ipython-based) code, and we'll play with
    Fernando> that a bit.  Do you foresee any problems with the WX
    Fernando> backend?  Right now I have global pygtk calls, but I
    Fernando> suppose those should be done only by the matplotlib
    Fernando> shell class IF Gtk is the chosen backend.  

Relocating the pygtk calls into the backend dependent section sounds
good.  As for problems with wx, I can't really say until we try.

Thanks!
JDH

Re: [Matplotlib-users] Interactive gtk console. Fixed (I think)

[Fernando P. <Fer...@co...>]

John Hunter wrote:

> Yes, and along those lines for the GTK shell, we should use
> rcParams['backend'] to detect GTK versus GTKAgg and use whichever is
> in the rc file in the call to matplotlib.use.

Ok.  I may take a first stab at this in a day or two, but no promises.

> good.  As for problems with wx, I can't really say until we try.

I can :(  I just tried, and the new code (which doesn't lock up gv), will 
never start if the backend chosen is WX.  But it may be a matter of thread 
conflicts or something, since pygtk calls are still being made.  Let's get 
Tk*/Gtk* working first, with a well-localized set of thread calls only when 
needed.  With a clean enough base, putting the WX code in the right place 
should be reasonably easy.  Perhaps Andrew (who wanted to hack matplotlib a 
bit) might help along in LA.

Cheers,

f