Menu
▾
▴
matplotlib-devel
|
From: Baptiste C. <bap...@al...> - 2005-01-23 23:29:07
Attachments:
gui.diff
|
Hello, here is a proposition of a new strategy for interactive plotting in GTK (probably also WX), that doesn't need IPython, and runs the mainloop in a separate thread from the calculations. It has no impact on the user unless he chooses to make use of it, and a small impact on the backend developper. It works in two parts: matplotlib/__init__.py imports a gtk_protect function that returns a wrapper around "dangerous" functions or methods. The one proposed here (protect_mutex.py) encloses the actual function call in a gtk.threads_enter() / gtk.threads_leave() pair in order to acquire the GTK mutex. The same strategy seems possible in WX, though I encountered a lot of annoying little problems with WX 2.4. If someone is interested in taking over that part, please ask me (I usually don't use WX). For the non-interactive case, matplotlib defines a dummy gtk_protect function which returns its own input, i.e. nothing is changed in that case. Then all the backend as to do is call gtk_protect on dangerous functions or methods in order to replace them with a wrapper. The wrapper has little overhead, so I may have protected more than needed. How to use: after applying the patch, you have to change your rc params to actually use it. Change your .matplotlibrc to contain the following: backend : GTKAgg # or GTK or GTKCairo interactive : True gui_protect : Mutex Afterwards, you can run most of the examples with python -i, and modify the plots to your liking :-) Cheers, BC |
|
From: Baptiste C. <bap...@al...> - 2005-01-24 21:43:54
Attachments:
gui.diff
|
Hello, here is an imporved version of the patch (minor bug corrected + diffed against latest CVS). By the way, it looks like the web interface is cutting off the patch. Probably the right thing given its size. Providing a link to the attached file would be nicer, though... Cheers, BC |
|
From: Fernando P. <Fer...@co...> - 2005-01-24 21:48:57
|
Baptiste Carvello wrote: > Hello, > > here is an imporved version of the patch (minor bug corrected + diffed > against latest CVS). Just a note: are you aware of the gui_thread module in scipy? It was written exactly to do this (though it's currently WX-specific, I think), and it has seen a pretty extensive amount of work. I would much rather see a generic, external solution be tested and improved than a matplotlib-specific rather intrusive patch. Don't take it badly, I can tell you've put a lot of work into this, but this kind of problem is IMHO best addressed by a more generic mechanism. The kind of attribute access which python allows opens the door for such a solution, as gui_thread has shown. gui_thread currently needs testing and improvements, but I think in the long run it is a cleaner, better alternative. Just my 2 cents. Cheers, f |
|
From: Baptiste C. <bap...@al...> - 2005-01-24 22:42:35
|
Hello, I'm just making a proposal. As long as the documentation points to=20 IPython for interactive use, you can't say we have a perfect solution on=20 our own :-) (this is not meant to criticize your work on IPython, it's=20 just that I feel more confortable with the usual python interface) I know the patch is intrusive, though maybe not as much as it looks=20 (everything except protect_mutex.py is pretty trivial). I didn't=20 expected it to be accepted right away, I see it more as a contribution=20 to the whole gui thread discution. As for gui_thread, I looked at it last year, I just remember I didn't=20 like that code that much. Maybe I should get another look... Cheers, BC Fernando Perez a =E9crit : > Just a note: are you aware of the gui_thread module in scipy? It was=20 > written exactly to do this (though it's currently WX-specific, I think)= ,=20 > and it has seen a pretty extensive amount of work. I would much rather= =20 > see a generic, external solution be tested and improved than a=20 > matplotlib-specific rather intrusive patch. Don't take it badly, I can= =20 > tell you've put a lot of work into this, but this kind of problem is=20 > IMHO best addressed by a more generic mechanism. The kind of attribute= =20 > access which python allows opens the door for such a solution, as=20 > gui_thread has shown. >=20 > gui_thread currently needs testing and improvements, but I think in the= =20 > long run it is a cleaner, better alternative. >=20 > Just my 2 cents. >=20 > Cheers, >=20 > f >=20 |
|
From: Fernando P. <Fer...@co...> - 2005-01-24 22:49:09
|
Baptiste Carvello wrote: > Hello, > > I'm just making a proposal. As long as the documentation points to > IPython for interactive use, you can't say we have a perfect solution on > our own :-) (this is not meant to criticize your work on IPython, it's > just that I feel more confortable with the usual python interface) > > I know the patch is intrusive, though maybe not as much as it looks > (everything except protect_mutex.py is pretty trivial). I didn't > expected it to be accepted right away, I see it more as a contribution > to the whole gui thread discution. > > As for gui_thread, I looked at it last year, I just remember I didn't > like that code that much. Maybe I should get another look... I actually didn't have ipython in mind at all in my comment. I agree that a generic, shell-independent solution is a good thing to have. I just think that gui_thread provides a cleaner approach (if not clean code :) to the overall problem. gui_thread is messy, but it allows general (meaning, beyond matplotlib) access to gui elements from a command line. I quickly tested it with matplotlib and it crashed, so there's work to be done... I was just trying to point out an alternate direction, ultimately if John is happy with this approach, it's his call (he gets to maintain it :) Cheers, f |
|
From: John H. <jdh...@ac...> - 2005-01-24 23:05:55
|
>>>>> "Baptiste" == Baptiste Carvello <bap...@al...> writes:
Baptiste> I know the patch is intrusive, though maybe not as much
Baptiste> as it looks (everything except protect_mutex.py is
Baptiste> pretty trivial). I didn't expected it to be accepted
Baptiste> right away, I see it more as a contribution to the whole
Baptiste> gui thread discution.
Baptiste> As for gui_thread, I looked at it last year, I just
Baptiste> remember I didn't like that code that much. Maybe I
Baptiste> should get another look...
Most of what I've heard of gui thread (caveat, the situation may be
different now but this was the last I heard) was that the code was
hairy, required scipy distutils to build, was wx specific and had
problems that required significant work to fix. In that context, a
patch, even an intrusive one, that allowed gtk* and wx* to work from
an arbitrary python shell would be most welcome. While I think
ipython is the one true shell, there are many other shells in
existence and that won't change. If for example, this patch allowed
gtk* to be used from within IDLE or pycrust in addition to the
standard python shell, that would be a big win.
The problem we need to solve is substantially easier than the one
gui_thread wants to solve, if I understand the issues correctly. gui
thread enables you to interact with general wx apps from a general
python shell. We want gtk* and wx* backends to work from a general
python shell. If this patch addresses that, it's worth a close look.
Baptiste, have you have a chance to test with other shells? Would
your patch break gtk* or wx* under ipython? Does it work in IDLE
and/or pycrust? Do you have any ideas on how this would affect
performance? Obviously it adds some function call overhead -- with
collections the cases where there are large numbers of calls to the
backend renderers is reduced.
Another thought spoken as a threading newbie: could this be done more
simply by, eg subclassing RendererGTK and automating the protect
wrapping. This latter approach would satisfy a couple of points: it
would not be intrusive in the base class, could be made optional via
interactive setting or rc, and would not adversely impact performance
when it was not needed (on my quick read of the patch, it looks like
the last point is already handled by the call)
if is_interactive():
import_protect()
I'll give your patches a test-drive. Perhaps Steve could also comment
on the gtk changes.
JDH
|
|
From: Fernando P. <Fer...@co...> - 2005-01-24 23:57:17
|
John Hunter wrote: >>>>>>"Baptiste" == Baptiste Carvello <bap...@al...> writes: > > > Baptiste> I know the patch is intrusive, though maybe not as much > Baptiste> as it looks (everything except protect_mutex.py is > Baptiste> pretty trivial). I didn't expected it to be accepted > Baptiste> right away, I see it more as a contribution to the whole > Baptiste> gui thread discution. > > Baptiste> As for gui_thread, I looked at it last year, I just > Baptiste> remember I didn't like that code that much. Maybe I > Baptiste> should get another look... > > Most of what I've heard of gui thread (caveat, the situation may be > different now but this was the last I heard) was that the code was > hairy, required scipy distutils to build, was wx specific and had > problems that required significant work to fix. In that context, a > patch, even an intrusive one, that allowed gtk* and wx* to work from > an arbitrary python shell would be most welcome. While I think > ipython is the one true shell, there are many other shells in > existence and that won't change. If for example, this patch allowed > gtk* to be used from within IDLE or pycrust in addition to the > standard python shell, that would be a big win. I certainly have never been a fan of digging too deep into gui_thread myself, I admit that much :) Last time I looked at the code, I got a bit scared. On the other hand, Prabhu recently did a bunch of work on it, so it may be cleaner/better, I just don't know. And as I said, I really was not speaking from the ipython side of things at all, I don't know why Baptiste took it that way (I didn't even mention ipython in my post). My opinion was strictly based on the generic approach offered by gui_thread, and the potential benefit of doing things that way. > The problem we need to solve is substantially easier than the one > gui_thread wants to solve, if I understand the issues correctly. gui > thread enables you to interact with general wx apps from a general > python shell. We want gtk* and wx* backends to work from a general > python shell. If this patch addresses that, it's worth a close look. One thing that I like about gui_thread is that it works in a generic manner. What happens with this patch if someone uses matplotlib but adds new widgets? Or if they embed a matplotlib window into something else? Are such usage cases covered automatically, or do they require special handling by third-party code? There certainly are benefits to a matplotlib-based self-contained solution, and gui_thread comes with its own baggage, so I'll let you guys judge where the balance rests. Cheers, f |
|
From: Steve C. <ste...@ya...> - 2005-01-26 04:16:53
Attachments:
mt.py
|
Regarding Baptiste's patch - I'm not too keen on the idea of wrapping gtk.gdk.threads_enter() / gtk.gdk.threads_leave() around every GDK function call. - I share Fernando's concern with possible problems when adding new widgets or embedding a matplotlib window into something else. My understanding is that wrapping gtk.gdk.threads_enter() / gtk.gdk.threads_leave() around GDK functions is not necessary if all the GDK calls occur in the same thread. So I think a simpler solution is to arrange for all gui calls to go to a single thread and do away with enter()/leave(). I created a module that allows interactive matplotlib access from a python shell, its a modified version of http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109 This method seems to work for me (on Linux) and is not very intrusive to matplotlib code - it just requires that 'gui_cmd()' is used to execute all gui commands when in interactive mode. Any comments? Regards Steve |
|
From: Fernando P. <Fer...@co...> - 2005-01-26 05:25:26
Attachments:
pyint-gtk.py
|
Steve Chaplin wrote: > I created a module that allows interactive matplotlib access from a > python shell, its a modified version of > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109 > > This method seems to work for me (on Linux) and is not very intrusive to > matplotlib code - it just requires that 'gui_cmd()' is used to execute > all gui commands when in interactive mode. > > Any comments? In case anyone is interested, here's a lightweight, standalone (non-ipython) GTK threaded interactive interpreter with matplotlib support. This code is also based on the same ASPN recipe, it's basically the prototype I used to understand that recipe when I was adding the matplotlib support to IPython. The threading tricks are the gold in that recipe, but the rest of the code is rather messy. I pretty much rewrote it when studying it (while keeping the threading core). Perhaps this may be of use to others. The ASPN recipe duplicates a lot of the functionality in the code.py module from the standard library, which is completely unnecessary (it's guaranteed to be there in any python installation). By using code.py, the whole thing can be rewritten much more cleanly (and you benefit from a good implementation in the stdlib for all the interpreter emulation parts). I think I sent this to John a while back so he could replace the interactive.py shipped with matplotlib. I fully acknowledge in the header the ASPN recipe as the origin of the threading solution. The pylab support in current ipython is basically this same idea, but generalized to work with either GTK or WX, and blended into the IPython code structure (that wasn't too bad, since ipython itself is based on code.py). Cheers, f |
|
From: Steve C. <ste...@ya...> - 2005-01-27 16:56:34
|
On Tue, 2005-01-25 at 22:25 -0700, Fernando Perez wrote: > In case anyone is interested, here's a lightweight, standalone (non-ipython) > GTK threaded interactive interpreter with matplotlib support. This code is > also based on the same ASPN recipe, it's basically the prototype I used to > understand that recipe when I was adding the matplotlib support to IPython. Thanks, its a good example of code.InteractiveConsole, when I saw codeop in the ASPN recipe I wondered why code.py was not used. I noticed a few small points: - GTKInterpreter says "Run a gtk mainloop() in a separate thread.", yet gtk.main() runs in the main thread, and the interpreter/console runs in a separate thread. - gtk.timeout_add() is now deprecated in favour of gobject.timeout_add() - runsource() and runcode() are synchronised using wait()/notify() so they deal with one code object at a time. If self.code_to_run is changed to a code queue (using list.append() and list.pop(0)) then the wait ()/notify() could be removed and the interpreter thread would return immediately it adds the code to self.code_to_run and not when the main thread has executed the code. > I think I sent this to John a while back so he could replace the > interactive.py shipped with matplotlib. I fully acknowledge in the header the > ASPN recipe as the origin of the threading solution. John, this looks like an much improved interactive.py, is it OK to use it to replace interactive.py in cvs, and if so should the '-pylab' option be left as it is or removed to be compatible with the current interactive.py? Steve |
|
From: Fernando P. <Fer...@co...> - 2005-01-27 17:12:12
|
Steve Chaplin wrote: > On Tue, 2005-01-25 at 22:25 -0700, Fernando Perez wrote: > >>In case anyone is interested, here's a lightweight, standalone (non-ipython) >>GTK threaded interactive interpreter with matplotlib support. This code is >>also based on the same ASPN recipe, it's basically the prototype I used to >>understand that recipe when I was adding the matplotlib support to IPython. > > Thanks, its a good example of code.InteractiveConsole, when I saw codeop in > the ASPN recipe I wondered why code.py was not used. > > I noticed a few small points: > - GTKInterpreter says "Run a gtk mainloop() in a separate thread.", yet > gtk.main() runs in the main thread, and the interpreter/console runs in > a separate thread. > - gtk.timeout_add() is now deprecated in favour of gobject.timeout_add() > > - runsource() and runcode() are synchronised using wait()/notify() so > they deal with one code object at a time. If self.code_to_run is changed > to a code queue (using list.append() and list.pop(0)) then the wait > ()/notify() could be removed and the interpreter thread would return > immediately it adds the code to self.code_to_run and not when the main > thread has executed the code. I am _really_ ignorant of thread issues, and so I have avoided touching the threading part of this code at all, though I've always suspected it could be cleaner. Would you mind applying these fixes and sending it back my way (including the GTK stuff)? I could then port your improvements over to the code in ipython proper which implements this same functionality. Cheers, f |
|
From: Fernando P. <Fer...@co...> - 2005-01-28 00:14:57
|
Steve Chaplin wrote: > On Tue, 2005-01-25 at 22:25 -0700, Fernando Perez wrote: > >>In case anyone is interested, here's a lightweight, standalone (non-ipython) >>GTK threaded interactive interpreter with matplotlib support. This code is >>also based on the same ASPN recipe, it's basically the prototype I used to >>understand that recipe when I was adding the matplotlib support to IPython. > > Thanks, its a good example of code.InteractiveConsole, when I saw codeop in > the ASPN recipe I wondered why code.py was not used. > > I noticed a few small points: > - GTKInterpreter says "Run a gtk mainloop() in a separate thread.", yet > gtk.main() runs in the main thread, and the interpreter/console runs in > a separate thread. You are correct. > - gtk.timeout_add() is now deprecated in favour of gobject.timeout_add() Feel free to fix it, I didn't know better. > - runsource() and runcode() are synchronised using wait()/notify() so > they deal with one code object at a time. If self.code_to_run is changed > to a code queue (using list.append() and list.pop(0)) then the wait > ()/notify() could be removed and the interpreter thread would return > immediately it adds the code to self.code_to_run and not when the main > thread has executed the code. On second thought, I don't think this is a good idea in general. The problem I see is the serialization of output. Imagine you start a long running job and you get the prompt back right away. If your job prints to stdout along the way (as is very common with scientific codes), your input line will get clobbered by this output. If you could run in a gui window where you could associate the stdout for each input 'cell' with its own output one, it would be different. This is exactly how Mathematica notebooks work, they stack a queue of cells for running sequentially, you get your typing prompt back immediately, but the output of each command is collected in an Out[] cell next to the corresponding In[] one. In a plain terminal, I don't see how we can make this work (in fact, in a terminal Mathematica also serializes output for this very reason). I have been toying with the idea of adding a way to 'background' processes automatically to ipython, by creating a list of worker threads for long running jobs. But the problem of handling stdout has me stumped. I'm open to comments, though, or perhaps I'm missing something obvious here. Best, f |
|
From: Steve C. <ste...@ya...> - 2005-01-30 03:44:08
|
On Thu, 2005-01-27 at 17:14 -0700, Fernando Perez wrote: > > - runsource() and runcode() are synchronised using wait()/notify() so > > they deal with one code object at a time. If self.code_to_run is changed > > to a code queue (using list.append() and list.pop(0)) then the wait > > ()/notify() could be removed and the interpreter thread would return > > immediately it adds the code to self.code_to_run and not when the main > > thread has executed the code. > > On second thought, I don't think this is a good idea in general. The problem > I see is the serialization of output. Imagine you start a long running job > and you get the prompt back right away. If your job prints to stdout along > the way (as is very common with scientific codes), your input line will get > clobbered by this output. > > If you could run in a gui window where you could associate the stdout for each > input 'cell' with its own output one, it would be different. This is exactly > how Mathematica notebooks work, they stack a queue of cells for running > sequentially, you get your typing prompt back immediately, but the output of > each command is collected in an Out[] cell next to the corresponding In[] one. > In a plain terminal, I don't see how we can make this work (in fact, in a > terminal Mathematica also serializes output for this very reason). > > I have been toying with the idea of adding a way to 'background' processes > automatically to ipython, by creating a list of worker threads for long > running jobs. But the problem of handling stdout has me stumped. > > I'm open to comments, though, or perhaps I'm missing something obvious here. So removing wait()/notify() would not be such a good idea, it might be useful if you are only running gui code, but not when the command prints to stdout. To run a background process as a thread you would need to redirect its stdout somewhere (StringIO for example). The problem with using threads for this is that they share the stdout with the main thread, so if you redirect the background threads stdout you will redirect the python interpreters stdout too! You could run a background process as an actual process and not a thread (and Python 2.4 has a new subprocess module which unifies the previous solutions of os.system, os.spawn*, os.popen*, popen2.*, commands.*). It lets you redirect stdout, but is used for running executables, not calling python functions. It has me stumped too, how can you call a python function and redirect its stdout independent of the python interpreters stdout? Steve |
|
From: Fernando P. <Fer...@co...> - 2005-01-30 18:43:40
|
Steve Chaplin wrote: > To run a background process as a thread you would need to redirect its > stdout somewhere (StringIO for example). The problem with using threads > for this is that they share the stdout with the main thread, so if you > redirect the background threads stdout you will redirect the python > interpreters stdout too! > You could run a background process as an actual process and not a thread > (and Python 2.4 has a new subprocess module which unifies the previous > solutions of os.system, os.spawn*, os.popen*, popen2.*, commands.*). It > lets you redirect stdout, but is used for running executables, not > calling python functions. > > It has me stumped too, how can you call a python function and redirect > its stdout independent of the python interpreters stdout? Not as far as I know. That's why I think the only solution for the whole 'backgrounding' problem with stdout is to put everything into a gui window (a la mathematica notebooks). I've been thinking about it, and the proper way to write it is probably with _two_ python processes. One would be the gui environment, and the other would be just a 'kernel' executing commands. The stdout of the kernel would then be connected to an object capable of putting the results of each command next to the input cell which generated them. I bet that's pretty much how Mathematica is organized, simply because it looks like the only reasonable way to deal with the issue. Cheers, f |
|
From: Steve C. <ste...@ya...> - 2005-01-26 09:47:05
|
On Wed, 2005-01-26 at 12:17 +0800, Steve Chaplin wrote: > I created a module that allows interactive matplotlib access from a > python shell, its a modified version of > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109 > > This method seems to work for me (on Linux) and is not very intrusive to > matplotlib code - it just requires that 'gui_cmd()' is used to execute > all gui commands when in interactive mode. I may not have clearly explained my idea. I'm not proposing another Python shell/interpreter to run matplotlib. The mt.py module is to test an idea without needing to modify any matplotlib code. To use it as intended would involve moving it into matplotlib and getting matplotlib to call gui_cmd() to execute interactive gtk/gdk calls in their own thread. This is the part I'm not sure about - what is the minimal set of functions to wrap with gui_cmd() to ensure that it encloses all gtk/gdk functions? gui_cmd() could be called from draw_if_interactive() and maybe new_figure_manager(), I'm not sure where else it might be needed. Also it would need to be fed import commands. Steve |
|
From: Baptiste C. <bap...@al...> - 2005-01-26 21:19:35
|
Steve Chaplin a =E9crit : > Regarding Baptiste's patch > - I'm not too keen on the idea of wrapping gtk.gdk.threads_enter() / > gtk.gdk.threads_leave() around every GDK function call. > - I share Fernando's concern with possible problems when adding new > widgets or embedding a matplotlib window into something else. >=20 > My understanding is that wrapping gtk.gdk.threads_enter() / > gtk.gdk.threads_leave() around GDK functions is not necessary if all th= e > GDK calls occur in the same thread. So I think a simpler solution is to > arrange for all gui calls to go to a single thread and do away with > enter()/leave(). >=20 This is mostly a question of taste. A bug in the handling of threads_enter()/threads_leave() leads to a=20 deadlock, while a bug with a gui thread yields "Xlib async reply".=20 Choose which ugly death you prefer ! > I created a module that allows interactive matplotlib access from a > python shell, its a modified version of > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109 >=20 In fact your module fits well into the framework of my proposal: all you=20 have to do is rename your gui_cmd to gtk_protect, and change a few=20 things like passing the function directly, passing the arguments, and=20 returning the value. Then you will have a drop-in replacement for=20 protect_mutex.py. > This method seems to work for me (on Linux) and is not very intrusive t= o > matplotlib code - it just requires that 'gui_cmd()' is used to execute > all gui commands when in interactive mode. >=20 This is where the real beef is. There are several strategies: * wrap everything: this is the examples/interactive.py strategy, simple,=20 but the gui freezes on long calculations * wrap all gui calls: this is the gui_thread strategy: elegant, but=20 difficult * wrap only pylab functions (draw_if_interactive etc), as you seem to=20 suggest in your second email: imho this is not acceptable because it=20 breaks method calls of matplotlib objects * draw the line somewhere in the backend: this is what I proposed. In the end, the backends developpers have the final say, because they=20 will have to do the day-to-day maintenance. > Any comments? >=20 > Regards > Steve >=20 Cheers, BC PS: tell me if I need to CC you or not |
|
From: Steve C. <ste...@ya...> - 2005-01-30 08:37:08
|
On Wed, 2005-01-26 at 22:18 +0100, Baptiste Carvello wrote:
I've finally got round installing the interactive gui thread patch to
have a look at it. It worked quite well for a few simple plots I tried.
> A bug in the handling of threads_enter()/threads_leave() leads to a
> deadlock, while a bug with a gui thread yields "Xlib async reply".
> Choose which ugly death you prefer !
What situation would yield an "Xlib async reply"?
> There are several strategies:
...
> * wrap only pylab functions (draw_if_interactive etc), as you seem to
> suggest in your second email: imho this is not acceptable because it
> breaks method calls of matplotlib objects
How does it break method calls of matplotlib objects? In interactive
mode the gtk backend does not work (by itself) so there's nothing to
break, and in non-interactive mode the gui_cmd() function would not be
used.
When thinking about using threads for interactive mode I had been
assuming people using the interactive mode would be using the pylab
interface, does anyone use the matplotlib class interface in interactive
mode?
I have a few questions about patch itself:
In import_gtk() there is time.sleep(1) - what is that for?
Shouldn't widget creation be protected as well by wrapping things like
new_figure_manager() and FigureCanvasGTK(). I know __init__ is wrapped
but objects may have __new__ methods which call gdk/gtk functions before
__init__ is called.
How about if a widgets standard methods such as show(), present() etc
are used - they are not protected (unless called from a protected
method). I think this just applies to using the class interface in
interactive mode, so it may be rarely encountered but might be a
problem.
Why use gtk_protect(), exec() and GTK_TEMPLATE, when you could use:
def gtk_protect (fun):
def decorator (*args, **kwargs):
if IN_GTK or threading.currentThread() is Thread:
return fun(*args, **kwargs)
IN_GTK.append (fun)
gtk.threads_enter()
try:
res = fun(*args, **kwargs)
finally:
gtk.threads_leave()
IN_GTK.pop()
return res
return decorator
Steve
|
|
From: Baptiste C. <bap...@al...> - 2005-01-30 22:47:31
|
Hi,
Steve Chaplin a =E9crit :
> What situation would yield an "Xlib async reply"?
> =20
using your mt module:
>>> import mt
>>> mt.gui_cmd('print "OK"')
OK
>>> import matplotlib
>>> matplotlib.interactive(1)
>>> import pylab
Xlib: unexpected async reply (sequence 0x4b)!
This is on linux, I don't know what you get on other platforms, but I=20
guess it's just as bad.
> How does it break method calls of matplotlib objects? In interactive
> mode the gtk backend does not work (by itself) so there's nothing to
> break
Well, it depends on how ambitious you are :-)
> When thinking about using threads for interactive mode I had been
> assuming people using the interactive mode would be using the pylab
> interface, does anyone use the matplotlib class interface in interactiv=
e
> mode?
>=20
I sometimes do. However, I've painted the situation worse than it is. I=20
almost only use the setter/getter methods, which are not dangerous.
One annoying thing though: error_msg will hang the gui:
>>> import mt
>>> mt.gui_cmd('from pylab import *')
>>> mt.gui_cmd('plot([1,2])')
>>> mt.gui_cmd('plot([1,2],"0r")')
*** gui hangs here, you get the prompt back, but no gui commands will=20
work ***
> In import_gtk() there is time.sleep(1) - what is that for?
>=20
This is an attempt to avoid a race condition were a proxy function would=20
be called before the GTK thread is initialized. I don't know if this=20
would lead to a real problem. 1s wait not that bad anyway.
> Shouldn't widget creation be protected as well by wrapping things like
> new_figure_manager() and FigureCanvasGTK(). I know __init__ is wrapped
> but objects may have __new__ methods which call gdk/gtk functions befor=
e
> __init__ is called.
>=20
I didn't think of that. Most GTK object are not dangerous until they are=20
shown.
> How about if a widgets standard methods such as show(), present() etc
> are used - they are not protected (unless called from a protected
> method). I think this just applies to using the class interface in
> interactive mode, so it may be rarely encountered but might be a
> problem.
>=20
As I stated, I didn't try to protect methods inherited from GTK objects=20
(ie not defined in matplotlib). Which means people using them are on=20
their own. You have to draw the boundary somewhere, unless you want to=20
implement a full gui_thread.
> Why use gtk_protect(), exec() and GTK_TEMPLATE, when you could use:
>=20
> def gtk_protect (fun):
> def decorator (*args, **kwargs):
>
Well this was a try to set the proxy function name as the real function=20
name, in order to avoid weirdeness when reading object help. But I could=20
not set the argument spec correctly, so I ended up adding the 'proxy=20
from' line to the docstring. So I think all the GTK_TEMPLATE machinery=20
serves no purpose now.
As a final note, I have to say that the latest version of interactive.py=20
is good enough for my needs, so I'm no longer very motivated for pushing=20
this patch.
If you just need the pylab interface, you can also do it without=20
patching matplotlib. Just create a protected_pylab.py module, where you=20
would start the gui thread, define the gtk_protect function, import=20
pylab and define command=3Dgtk_protect(pylab.command) for every command i=
n=20
pylab. Then doing 'from protected_pylab import *' would give you a=20
working pylab interface, with just the error_msg problem.
Cheers,
Baptiste
|
|
From: Baptiste C. <bap...@al...> - 2005-01-25 01:20:45
|
Hi John and Fernando, John, here is the quick answer to your questions: * the patch does almost nothing (a few calls to a dummy function) except when the gui_protect rcParam is enabled * when enabled the overhead is acceptable (on my Pentium III 500MHz), but import time of pylab is a little longer, because most of the work happens there. Also, only the functions and methods defined by matplotlib are protected. The idea is that if you call functions or methods from the gtk module, you ought to know what you're doing anyway (for that matter, I am not as ambitious as gui_thread). * the patch currently doesn't work with wx because I've not finished the work, but it is doable. The GTK or WX mainloop is launched at the time the backend is imported, and only the one for the backend you are using will be launched. * if you use interactive.py, or the code in IPython, you'd better not enable gui_protect. If you do, 2 GTK mainloops will be launched. This is allowed in GTK, but doesn't make sense. * the GTK mainloop (probably same for WX) can run concurrently with tk. This means no problem plotting GTK from IDLE. You can even switch backends at runtime by reloading matplotlib.backends and matplotlib.pylab * the GTK and WX mainloops cannot run concurrently. This is because, at least on Linux, WX itself launches a GTK mainloop (it's called WxGTK for a reason :). It might be possible to use this very mainloop to also display GTK figures, but I've had no luck with this until now. Fernando, I've been looking again at gui_thread. I now understand the beauty and power of it, but it is also more brittle, which means a lot of work and testing. It might prove the best solution in the end, though. I took no offense at all about IPython, so neither should you :-) I just mentioned it because it is the advised solution in matplotlib doc. Have a nice day, BC PS: You don't have to CC me, I'm subscribed to the list. |
|
From: John H. <jdh...@ac...> - 2005-01-27 17:52:14
|
>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:
Steve> John, this looks like an much improved interactive.py, is
Steve> it OK to use it to replace interactive.py in cvs, and if so
Steve> should the '-pylab' option be left as it is or removed to
Steve> be compatible with the current interactive.py?
Hey Steve,
I probably just forgot to update interactive.py last time Fernando
sent it to me. Feel free to do update interactive.py and its
interface and command line args (eg, pylab or not) as you see fit.
Just update the changelog so I can include them in the next release
notes.
After you're done, you may want to take a look at interactive.html
and/or \section{Interactive} of the user's guide, and make updates as
you see fit.
Thanks!
JDH
|
|
From: Steve C. <ste...@ya...> - 2005-01-28 07:05:53
|
On Thu, 2005-01-27 at 11:43 -0600, John Hunter wrote:
> I probably just forgot to update interactive.py last time Fernando
> sent it to me. Feel free to do update interactive.py and its
> interface and command line args (eg, pylab or not) as you see fit.
> Just update the changelog so I can include them in the next release
> notes.
>
> After you're done, you may want to take a look at interactive.html
> and/or \section{Interactive} of the user's guide, and make updates as
> you see fit.
I replaced examples/interactive.py with Fernando's updated version
(adding a comment that gtk.main runs in the main thread and replacing
gtk.timeout_add with gobject.timeout_add).
I looked at interactive.html.template and user_guide.tex and saw a
reference to ipython, but no mention of interactive.py, so it does not
look like they need updating. I seem to recall interactive.py being
mentioned in the docs in the past, but they must have been changed since
then.
Steve
|
|
From: Fernando P. <Fer...@co...> - 2005-01-30 23:00:36
|
Baptiste Carvello wrote: > Hi, >=20 > Steve Chaplin a =E9crit : >>When thinking about using threads for interactive mode I had been >>assuming people using the interactive mode would be using the pylab >>interface, does anyone use the matplotlib class interface in interactiv= e >>mode? >> >=20 > I sometimes do. Agreed. One should also be able to execute interactively large scripts which may = have=20 been written to the OO interface. I know that I routinely test 10k LOC= =20 modules from within ipython (though not matplotlib based). It would be I= MHO=20 poor design to put all sorts of special-case warnings, where certain feat= ures=20 of matplotlib would only be available in certain ways of using it. Speci= al=20 cases lead to madness... Note that I haven't followed the details close enough to vouch for any=20 particular approach. I just would like to see any solution implemented t= o be=20 as clean and general as possible. As far as users are concerned, these=20 details should be, where possible, totally irrelevant (internal implement= ation=20 only). As a reference, ipython tries mightily hard to make sure that _an= y_=20 code which can be executed at a system command line as 'python foo.py'=20 produces identical behavior at an ipython prompt via 'run foo'. There ar= e=20 special options to 'run' for modifying this, but the default attempts to=20 emulate python very faitfully, just with better tracebacks, debug support= ,=20 etc. And I consider deviations from this bevavior as bugs. It's not 100= %=20 perfect, but it really works fairly well in most cases. Regards, f |
|
From: Steve C. <ste...@ya...> - 2005-01-31 09:56:27
|
The gdk.threads_init() entry in the pygtk manual says: http://www.pygtk.org/pygtk2reference/gdk-functions.html#function-gdk-- threads-init "PyGTK does not release the GIL when calling a GTK or GDK function. Also it does not acquire the GDK global lock (GGL). This means that, in effect, Python threads can use the GIL alone to serialize access to the GTK and GDK libraries. Of course, if there are non-Python threads calling GTK or GDK functions the GGL must be used." I read this to mean that Python programs with no non-Python threads, like matplotlib, can run gtk.main in its own thread and do not need to use gtk_protect (for gdk.threads_enter/leave GGL locking) since the Python GIL will serialise access to GTK/GDK libraries. That would be an easy solution to the interactive GTK problem - just start gtk.main in its own thread with no need to alter any other code! It sounds too simple to be true, have I missed something? Steve |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X