matplotlib-devel

[matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Brian G. <ell...@gm...>]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release.  In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI.  For background on these changes please see this
thread:

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic.  This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits.  We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code).  To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Cheers,

Brian

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Eric F. <ef...@ha...>]

On 09/03/2010 12:37 PM, Brian Granger wrote:
> Hello all,
>
> I would like to submit the following branch on github for review and
> merging into matplotlib trunk:
>
> http://github.com/ellisonbg/matplotlib/commits/guisupport
>
> This branch implements the logic needed for the qt4 and wx backends to
> fully work with the upcoming IPython 0.11 release.  In our testing we
> have run many of the mpl examples (including the new animation
> examples) in both qt4/wx in both the terminal based IPython and the
> new IPython Qt GUI.  For background on these changes please see this
> thread:
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>
> It is important to note that we have not updated the other matplotlib
> backends (gtk, tk, etc.) to have this logic.  This is mainly because
> we know almost nothing about these toolkits and could really use some
> help from folks who are experts at the respective toolkits.  We have
> done some minimal testing and these other backends do work for simple
> examples in the terminal IPython, but they won't work in all cases and
> will definitely not work in the new IPython Qt based GUI.
>
> We would love feedback and help testing as these changes are
> significant (even though only a few lines of code).  To test this
> stuff you will need to grab the following IPython development branch:
>
> http://github.com/ipython/ipython/tree/newkernel
>
> You should be able to run the examples in regular ipython:
>
> ipython --pylab qt4|wx
>
> Or the new GUI
>
> ipythonqt --pylab qt4|wx
>

Brian and others,

It's not quite that simple.  After some initial thrashing around, I 
installed zmq from source, and then pyzmq--but I can't import zmq:

In [1]: import zmq
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/home/efiring/<ipython console> in <module>()

/usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>()
      24 
#-----------------------------------------------------------------------------

      25
---> 26 from zmq import _zmq
      27 from zmq._zmq import *
      28

ImportError: libzmq.so.0: cannot open shared object file: No such file 
or directory

But it is right where it should be:

efiring@manini:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq*
-rw-r--r-- 1 root root 4736930 2010-09-03 14:35 /usr/local/lib/libzmq.a
-rwxr-xr-x 1 root root     825 2010-09-03 14:35 /usr/local/lib/libzmq.la
lrwxrwxrwx 1 root root      15 2010-09-03 14:35 /usr/local/lib/libzmq.so 
-> libzmq.so.0.0.0
lrwxrwxrwx 1 root root      15 2010-09-03 14:35 
/usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0
-rwxr-xr-x 1 root root 1993116 2010-09-03 14:35 
/usr/local/lib/libzmq.so.0.0.0

And it looks like the extension was built OK, complete with linking:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall 
-Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6 
-c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o
zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’ 
defined but not used
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions 
build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o 
build/lib.linux-x86_64-2.6/zmq/_zmq.so


I'm stumped.

Eric

> Cheers,
>
> Brian

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Brian G. <ell...@gm...>]

Eric,

Looks like you are on Linux.  If so, you will have to make sure you do
two things before doing python setup.py install on pyzmq:

1.  Edit the setup.cfg (copy the template) to point to your zeromq install.
2.  Use the --rpath option:

python setup.py build_ext --rpath=/opt/zeromq-dev/lib --inplace

Alternatively, you can set the LD_LIBRARY_PATH to point to your zeromq lib path.

Also, for now, we suggest using the stable 2.0.7 releases of both
pyzmq and zeromq.

Sorry about the hitches, parts of this are definitely still rough and
we hope to iron out these things.  I should mention that zeromq/pyzmq
are only required for the new Qt gui, the old terminal ipython doesn't
require them.
Let us know if you run into any other issues.

Cheers,

Brian

On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <ef...@ha...> wrote:
> On 09/03/2010 12:37 PM, Brian Granger wrote:
>> Hello all,
>>
>> I would like to submit the following branch on github for review and
>> merging into matplotlib trunk:
>>
>> http://github.com/ellisonbg/matplotlib/commits/guisupport
>>
>> This branch implements the logic needed for the qt4 and wx backends to
>> fully work with the upcoming IPython 0.11 release.  In our testing we
>> have run many of the mpl examples (including the new animation
>> examples) in both qt4/wx in both the terminal based IPython and the
>> new IPython Qt GUI.  For background on these changes please see this
>> thread:
>>
>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>>
>> It is important to note that we have not updated the other matplotlib
>> backends (gtk, tk, etc.) to have this logic.  This is mainly because
>> we know almost nothing about these toolkits and could really use some
>> help from folks who are experts at the respective toolkits.  We have
>> done some minimal testing and these other backends do work for simple
>> examples in the terminal IPython, but they won't work in all cases and
>> will definitely not work in the new IPython Qt based GUI.
>>
>> We would love feedback and help testing as these changes are
>> significant (even though only a few lines of code).  To test this
>> stuff you will need to grab the following IPython development branch:
>>
>> http://github.com/ipython/ipython/tree/newkernel
>>
>> You should be able to run the examples in regular ipython:
>>
>> ipython --pylab qt4|wx
>>
>> Or the new GUI
>>
>> ipythonqt --pylab qt4|wx
>>
>
> Brian and others,
>
> It's not quite that simple.  After some initial thrashing around, I
> installed zmq from source, and then pyzmq--but I can't import zmq:
>
> In [1]: import zmq
> ---------------------------------------------------------------------------
> ImportError                               Traceback (most recent call last)
> /home/efiring/<ipython console> in <module>()
>
> /usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>()
>      24
> #-----------------------------------------------------------------------------
>
>      25
> ---> 26 from zmq import _zmq
>      27 from zmq._zmq import *
>      28
>
> ImportError: libzmq.so.0: cannot open shared object file: No such file
> or directory
>
> But it is right where it should be:
>
> efiring@manini:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq*
> -rw-r--r-- 1 root root 4736930 2010-09-03 14:35 /usr/local/lib/libzmq.a
> -rwxr-xr-x 1 root root     825 2010-09-03 14:35 /usr/local/lib/libzmq.la
> lrwxrwxrwx 1 root root      15 2010-09-03 14:35 /usr/local/lib/libzmq.so
> -> libzmq.so.0.0.0
> lrwxrwxrwx 1 root root      15 2010-09-03 14:35
> /usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0
> -rwxr-xr-x 1 root root 1993116 2010-09-03 14:35
> /usr/local/lib/libzmq.so.0.0.0
>
> And it looks like the extension was built OK, complete with linking:
>
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6
> -c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o
> zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’
> defined but not used
> gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
> build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o
> build/lib.linux-x86_64-2.6/zmq/_zmq.so
>
>
> I'm stumped.
>
> Eric
>
>> Cheers,
>>
>> Brian
>
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgr...@ca...
ell...@gm...

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Fernando P. <fpe...@gm...>]

On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <ef...@ha...> wrote:
> It's not quite that simple.  After some initial thrashing around, I
> installed zmq from source, and then pyzmq--but I can't import zmq:
>

Mhh, sorry to see you burn up on this, Eric.  Brian is the zmq expert,
not me, but it *may* be a version compatibility issue, perhaps?  I am
using these versions of zmq/pyzmq:

- zmq 2.0.8: http://www.zeromq.org/local--files/area:download/zeromq-2.0.8.tar.gz
- pyzmq 2.0.7: http://github.com/downloads/zeromq/pyzmq/pyzmq-2.0.7.tar.gz

These two worked fine for me, with:

- for zmq:

./configure --prefix=$HOME/usr/opt
make && make install

note that $HOME/usr/opt/lib is in my LD_LIBRARY_PATH, and the include/
dir in my include path, etc.

- for pyzmq:

python setup.py install --prefix=$HOME/usr/opt

And at that point it just worked.  A few questions:

- is it possible that you did a build from the head of either
zmq/pyzmq git tree?  that might cause an incompatibility.
- could it be that /usr/local/lib isn in your LD_LIBRARY_PATH?

I'll be available over the weekend and will be more than happy to try
and help sort this out.

Regards, and sorry for the hassle

f

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Eric F. <ef...@ha...>]

On 09/03/2010 12:37 PM, Brian Granger wrote:
> Hello all,
>
> I would like to submit the following branch on github for review and
> merging into matplotlib trunk:
>
> http://github.com/ellisonbg/matplotlib/commits/guisupport
>
> This branch implements the logic needed for the qt4 and wx backends to
> fully work with the upcoming IPython 0.11 release.  In our testing we
> have run many of the mpl examples (including the new animation
> examples) in both qt4/wx in both the terminal based IPython and the
> new IPython Qt GUI.  For background on these changes please see this
> thread:
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>
> It is important to note that we have not updated the other matplotlib
> backends (gtk, tk, etc.) to have this logic.  This is mainly because
> we know almost nothing about these toolkits and could really use some
> help from folks who are experts at the respective toolkits.  We have
> done some minimal testing and these other backends do work for simple
> examples in the terminal IPython, but they won't work in all cases and
> will definitely not work in the new IPython Qt based GUI.
>
> We would love feedback and help testing as these changes are
> significant (even though only a few lines of code).  To test this
> stuff you will need to grab the following IPython development branch:
>
> http://github.com/ipython/ipython/tree/newkernel
>
> You should be able to run the examples in regular ipython:
>
> ipython --pylab qt4|wx
>
> Or the new GUI
>
> ipythonqt --pylab qt4|wx

Brian, Fernando,

I have been doing a little testing with ipython 0.10 versus 
ipython-newkernel, both modes, and with mpl svn versus your guisupport. 
There are so many possible modes of operation and combinations of 
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when 
you use mpl *svn* with ipython-newkernel, in either or both of the 
console or gui modes, but *does* work with your guisupport version?

Thanks.

Eric

>
> Cheers,
>
> Brian
>

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Fernando P. <fpe...@gm...>]

Hi Eric,

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <ef...@ha...> wrote:
>
> I have been doing a little testing with ipython 0.10 versus
> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
> There are so many possible modes of operation and combinations of
> versions and backends that all this will take some time to sort out.
>
> Can you give me simple examples of what does *not* work correctly when
> you use mpl *svn* with ipython-newkernel, in either or both of the
> console or gui modes, but *does* work with your guisupport version?

Thanks for your testing, Eric.

With matplotlib *alone*, I can't find a way to crash/lock/whatever the
combo of matplotlib(svn)+ipython-newkernel.

The reason, i believe, is that guisupport.py is available in ipython
itself, and it goes out of its way to avoid creating a second main qt
app, letting matplotlib create it. Since that main app is alive all
the time, there's only one app and one event loop and life is good.
But if I were to open another library that uses Qt and makes a new
main qApp unconditionally, we'd have problems.

Brian and Evan have recently just added the guisupport.py patch to
Enthought's ETS, so that now it probably will be pretty hard to
actually see the problem: if both ipython and ets go out of their way
to avoid the nested main app issue, mpl can get away with making one
unconditionally and things will probably work OK.

But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
collaborative protocol with a simple api: check for one special
'_in_event_loop' flag in the main toolkit before making one.  That
will make it easier to have interactive code that uses Wx or Qt from
more than one library coexisting in the same process.

I'll let Brian fill in with more details when he has some
availability, but I think that's the gist of it.

Regards,

f

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Brian G. <ell...@gm...>]

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <ef...@ha...> wrote:
> On 09/03/2010 12:37 PM, Brian Granger wrote:
>> Hello all,
>>
>> I would like to submit the following branch on github for review and
>> merging into matplotlib trunk:
>>
>> http://github.com/ellisonbg/matplotlib/commits/guisupport
>>
>> This branch implements the logic needed for the qt4 and wx backends to
>> fully work with the upcoming IPython 0.11 release.  In our testing we
>> have run many of the mpl examples (including the new animation
>> examples) in both qt4/wx in both the terminal based IPython and the
>> new IPython Qt GUI.  For background on these changes please see this
>> thread:
>>
>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>>
>> It is important to note that we have not updated the other matplotlib
>> backends (gtk, tk, etc.) to have this logic.  This is mainly because
>> we know almost nothing about these toolkits and could really use some
>> help from folks who are experts at the respective toolkits.  We have
>> done some minimal testing and these other backends do work for simple
>> examples in the terminal IPython, but they won't work in all cases and
>> will definitely not work in the new IPython Qt based GUI.
>>
>> We would love feedback and help testing as these changes are
>> significant (even though only a few lines of code).  To test this
>> stuff you will need to grab the following IPython development branch:
>>
>> http://github.com/ipython/ipython/tree/newkernel
>>
>> You should be able to run the examples in regular ipython:
>>
>> ipython --pylab qt4|wx
>>
>> Or the new GUI
>>
>> ipythonqt --pylab qt4|wx
>
> Brian, Fernando,
>
> I have been doing a little testing with ipython 0.10 versus
> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
> There are so many possible modes of operation and combinations of
> versions and backends that all this will take some time to sort out.
>
> Can you give me simple examples of what does *not* work correctly when
> you use mpl *svn* with ipython-newkernel, in either or both of the
> console or gui modes, but *does* work with your guisupport version?

The problems are when matplotlib and enthought.traits/pyface/mayavi
stuff are used together.  There are two types of problems:

* Multiple apps are created.  The enthought stuff used to not check
for existing apps, but that has been fixed.
* Event loop is started multiple times.  This one is more subtle and
on some toolkits the error is not fatal.  This problem shows up when
IPython is run in terminal mode and event loop integration is handled
through PyOS_InputHook.  In these situations, if matplotlib or ets
start the event loop themselves, IPython will hang.

Unfortunately, I am having trouble getting an install of both
matplotlib svn and ets svn on the same machine, so I can't reproduce
any of the failures ATM.  I am trying to get things installed so I can
reproduce the problems.

Cheers,

Brian

> Thanks.
>
> Eric
>
>>
>> Cheers,
>>
>> Brian
>>
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgr...@ca...
ell...@gm...

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Eric F. <ef...@ha...>]

On 09/07/2010 11:07 AM, Fernando Perez wrote:
> Hi Eric,
>
> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...>  wrote:
>>
>> I have been doing a little testing with ipython 0.10 versus
>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>> There are so many possible modes of operation and combinations of
>> versions and backends that all this will take some time to sort out.
>>
>> Can you give me simple examples of what does *not* work correctly when
>> you use mpl *svn* with ipython-newkernel, in either or both of the
>> console or gui modes, but *does* work with your guisupport version?
>
> Thanks for your testing, Eric.
>
> With matplotlib *alone*, I can't find a way to crash/lock/whatever the
> combo of matplotlib(svn)+ipython-newkernel.
>
> The reason, i believe, is that guisupport.py is available in ipython
> itself, and it goes out of its way to avoid creating a second main qt
> app, letting matplotlib create it. Since that main app is alive all
> the time, there's only one app and one event loop and life is good.
> But if I were to open another library that uses Qt and makes a new
> main qApp unconditionally, we'd have problems.
>
> Brian and Evan have recently just added the guisupport.py patch to
> Enthought's ETS, so that now it probably will be pretty hard to
> actually see the problem: if both ipython and ets go out of their way
> to avoid the nested main app issue, mpl can get away with making one
> unconditionally and things will probably work OK.
>
> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
> collaborative protocol with a simple api: check for one special
> '_in_event_loop' flag in the main toolkit before making one.  That
> will make it easier to have interactive code that uses Wx or Qt from
> more than one library coexisting in the same process.

Fernando,

There are two parts to guisupport: ensure a single main app, and ensure 
no more than one call to the mainloop.  The first makes perfect sense, 
and cannot cause any problems that I can see.  The second one is the one 
that I think may be both unnecessary and undesirable.  The reason is 
that the gui toolkit mainloop functions or methods are designed for 
nested calls.  This permits blocking within a running mainloop, and 
allows show() to block when pyplot is not in interactive mode.  This is 
what is lost with the guisupport mods.  Some changes to mainloop logic 
may well be needed, but I don't think that prohibiting nested calls to 
the underlying toolkit mainloop function is necessary or desirable.

Eric

>
> I'll let Brian fill in with more details when he has some
> availability, but I think that's the gist of it.
>
> Regards,
>
> f

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Brian G. <ell...@gm...>]

Eric,

Sorry about the delay, I was on vacation last week...comments inline below...

On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing <ef...@ha...> wrote:
> On 09/07/2010 11:07 AM, Fernando Perez wrote:
>> Hi Eric,
>>
>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...>  wrote:
>>>
>>> I have been doing a little testing with ipython 0.10 versus
>>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>>> There are so many possible modes of operation and combinations of
>>> versions and backends that all this will take some time to sort out.
>>>
>>> Can you give me simple examples of what does *not* work correctly when
>>> you use mpl *svn* with ipython-newkernel, in either or both of the
>>> console or gui modes, but *does* work with your guisupport version?
>>
>> Thanks for your testing, Eric.
>>
>> With matplotlib *alone*, I can't find a way to crash/lock/whatever the
>> combo of matplotlib(svn)+ipython-newkernel.
>>
>> The reason, i believe, is that guisupport.py is available in ipython
>> itself, and it goes out of its way to avoid creating a second main qt
>> app, letting matplotlib create it. Since that main app is alive all
>> the time, there's only one app and one event loop and life is good.
>> But if I were to open another library that uses Qt and makes a new
>> main qApp unconditionally, we'd have problems.
>>
>> Brian and Evan have recently just added the guisupport.py patch to
>> Enthought's ETS, so that now it probably will be pretty hard to
>> actually see the problem: if both ipython and ets go out of their way
>> to avoid the nested main app issue, mpl can get away with making one
>> unconditionally and things will probably work OK.
>>
>> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
>> collaborative protocol with a simple api: check for one special
>> '_in_event_loop' flag in the main toolkit before making one.  That
>> will make it easier to have interactive code that uses Wx or Qt from
>> more than one library coexisting in the same process.
>
> Fernando,


> There are two parts to guisupport: ensure a single main app, and ensure
> no more than one call to the mainloop.

Yes, that is a good summary.

> The first makes perfect sense,
> and cannot cause any problems that I can see.  The second one is the one
> that I think may be both unnecessary and undesirable.  The reason is
> that the gui toolkit mainloop functions or methods are designed for
> nested calls.  This permits blocking within a running mainloop, and
> allows show() to block when pyplot is not in interactive mode.  This is
> what is lost with the guisupport mods.  Some changes to mainloop logic
> may well be needed, but I don't think that prohibiting nested calls to
> the underlying toolkit mainloop function is necessary or desirable.

This is a very good point and is something that we have thought
carefully about.  You are very correct, that the functions in
guisupport cannot be used to do a nested mainloop.  Nested calls to
the mainloop should be done in the usual manner by simply calling the
appropriate gui toolkit method for doing so.  We probably need to
clarify this point, but the focus of the functions in guisupport are
*only* the first and main invocation of the event loop.  Basically, we
want to ensure that:

* Projects don't accidentally do nested mainloops because there were
not aware that the outermost eventloop was already running.
* Projects start the outermost eventloop in a manner that other
projects will be able to reliably detect.

I should mention the other approach that we have tried, but that failed:

* Have IPython startup, create an app and start the main loop.
* Then monkeypatch the gui toolkit so that the mainloop calls are no-ops.
* Further monkeypatch the gui toolkit so that it appears that the
mainloop is running (even when it may not be because of PyOS_InputHook
magic).

This allowed us to do everything, BUT obviously, nested mainloops
failed.  Thus, making sure that nested mainloops still work was the
main reason we have created guisupport.  We should better document
these details though.

Cheers,

Brian

> Eric
>
>>
>> I'll let Brian fill in with more details when he has some
>> availability, but I think that's the gist of it.
>>
>> Regards,
>>
>> f
>
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgr...@ca...
ell...@gm...

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Eric F. <ef...@ha...>]

On 09/13/2010 08:54 AM, Brian Granger wrote:
> Eric,
>
> Sorry about the delay, I was on vacation last week...comments inline below...
>
> On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing<ef...@ha...>  wrote:
>> On 09/07/2010 11:07 AM, Fernando Perez wrote:
>>> Hi Eric,
>>>
>>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...>    wrote:
>>>>
>>>> I have been doing a little testing with ipython 0.10 versus
>>>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>>>> There are so many possible modes of operation and combinations of
>>>> versions and backends that all this will take some time to sort out.
>>>>
>>>> Can you give me simple examples of what does *not* work correctly when
>>>> you use mpl *svn* with ipython-newkernel, in either or both of the
>>>> console or gui modes, but *does* work with your guisupport version?
>>>
>>> Thanks for your testing, Eric.
>>>
>>> With matplotlib *alone*, I can't find a way to crash/lock/whatever the
>>> combo of matplotlib(svn)+ipython-newkernel.
>>>
>>> The reason, i believe, is that guisupport.py is available in ipython
>>> itself, and it goes out of its way to avoid creating a second main qt
>>> app, letting matplotlib create it. Since that main app is alive all
>>> the time, there's only one app and one event loop and life is good.
>>> But if I were to open another library that uses Qt and makes a new
>>> main qApp unconditionally, we'd have problems.
>>>
>>> Brian and Evan have recently just added the guisupport.py patch to
>>> Enthought's ETS, so that now it probably will be pretty hard to
>>> actually see the problem: if both ipython and ets go out of their way
>>> to avoid the nested main app issue, mpl can get away with making one
>>> unconditionally and things will probably work OK.
>>>
>>> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
>>> collaborative protocol with a simple api: check for one special
>>> '_in_event_loop' flag in the main toolkit before making one.  That
>>> will make it easier to have interactive code that uses Wx or Qt from
>>> more than one library coexisting in the same process.
>>
>> Fernando,
>
>
>> There are two parts to guisupport: ensure a single main app, and ensure
>> no more than one call to the mainloop.
>
> Yes, that is a good summary.
>
>>   The first makes perfect sense,
>> and cannot cause any problems that I can see.  The second one is the one
>> that I think may be both unnecessary and undesirable.  The reason is
>> that the gui toolkit mainloop functions or methods are designed for
>> nested calls.  This permits blocking within a running mainloop, and
>> allows show() to block when pyplot is not in interactive mode.  This is
>> what is lost with the guisupport mods.  Some changes to mainloop logic
>> may well be needed, but I don't think that prohibiting nested calls to
>> the underlying toolkit mainloop function is necessary or desirable.
>
> This is a very good point and is something that we have thought
> carefully about.  You are very correct, that the functions in
> guisupport cannot be used to do a nested mainloop.  Nested calls to
> the mainloop should be done in the usual manner by simply calling the
> appropriate gui toolkit method for doing so.  We probably need to
> clarify this point, but the focus of the functions in guisupport are
> *only* the first and main invocation of the event loop.  Basically, we
> want to ensure that:
>
> * Projects don't accidentally do nested mainloops because there were
> not aware that the outermost eventloop was already running.
> * Projects start the outermost eventloop in a manner that other
> projects will be able to reliably detect.
>
> I should mention the other approach that we have tried, but that failed:
>
> * Have IPython startup, create an app and start the main loop.
> * Then monkeypatch the gui toolkit so that the mainloop calls are no-ops.
> * Further monkeypatch the gui toolkit so that it appears that the
> mainloop is running (even when it may not be because of PyOS_InputHook
> magic).
>
> This allowed us to do everything, BUT obviously, nested mainloops
> failed.  Thus, making sure that nested mainloops still work was the
> main reason we have created guisupport.  We should better document
> these details though.

Brian,

Thanks for the clarification.  Your basic approach sounds fine. I need 
to think a little more about how to handle this in mpl.  The proposed 
modification is not quite right yet.  I probably can't work on it until 
the weekend.

Eric

>
> Cheers,
>
> Brian
>
>> Eric
>>
>>>
>>> I'll let Brian fill in with more details when he has some
>>> availability, but I think that's the gist of it.
>>>
>>> Regards,
>>>
>>> f
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net Dev2Dev email is sponsored by:
>>
>> Show off your parallel programming skills.
>> Enter the Intel(R) Threading Challenge 2010.
>> http://p.sf.net/sfu/intel-thread-sfd
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>
>
>

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Eric F. <ef...@ha...>]

On 09/13/2010 05:46 PM, Brian Granger wrote:
> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...>  wrote:
>> On 09/03/2010 12:37 PM, Brian Granger wrote:
>>> Hello all,
>>>
>>> I would like to submit the following branch on github for review and
>>> merging into matplotlib trunk:
>>>
>>> http://github.com/ellisonbg/matplotlib/commits/guisupport
>>>
>>> This branch implements the logic needed for the qt4 and wx backends to
>>> fully work with the upcoming IPython 0.11 release.  In our testing we
>>> have run many of the mpl examples (including the new animation
>>> examples) in both qt4/wx in both the terminal based IPython and the
>>> new IPython Qt GUI.  For background on these changes please see this
>>> thread:
>>>
>>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>>>
>>> It is important to note that we have not updated the other matplotlib
>>> backends (gtk, tk, etc.) to have this logic.  This is mainly because
>>> we know almost nothing about these toolkits and could really use some
>>> help from folks who are experts at the respective toolkits.  We have
>>> done some minimal testing and these other backends do work for simple
>>> examples in the terminal IPython, but they won't work in all cases and
>>> will definitely not work in the new IPython Qt based GUI.
>>>
>>> We would love feedback and help testing as these changes are
>>> significant (even though only a few lines of code).  To test this
>>> stuff you will need to grab the following IPython development branch:
>>>
>>> http://github.com/ipython/ipython/tree/newkernel
>>>
>>> You should be able to run the examples in regular ipython:
>>>
>>> ipython --pylab qt4|wx
>>>
>>> Or the new GUI
>>>
>>> ipythonqt --pylab qt4|wx
>>
>> Brian, Fernando,
>>
>> I have been doing a little testing with ipython 0.10 versus
>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>> There are so many possible modes of operation and combinations of
>> versions and backends that all this will take some time to sort out.
>>
>> Can you give me simple examples of what does *not* work correctly when
>> you use mpl *svn* with ipython-newkernel, in either or both of the
>> console or gui modes, but *does* work with your guisupport version?
>
> The problems are when matplotlib and enthought.traits/pyface/mayavi
> stuff are used together.  There are two types of problems:
>
> * Multiple apps are created.  The enthought stuff used to not check
> for existing apps, but that has been fixed.

That one is easy.

> * Event loop is started multiple times.  This one is more subtle and
> on some toolkits the error is not fatal.  This problem shows up when
> IPython is run in terminal mode and event loop integration is handled
> through PyOS_InputHook.  In these situations, if matplotlib or ets
> start the event loop themselves, IPython will hang.

So, this can be checked with nothing but IPython and mpl. I think I may 
have seen this with some combination of configurations, though not with 
what I typically use.

>
> Unfortunately, I am having trouble getting an install of both
> matplotlib svn and ets svn on the same machine, so I can't reproduce
> any of the failures ATM.  I am trying to get things installed so I can
> reproduce the problems.

At least twice in the last couple of years I have tried to get mayavi 
compiled and running, without fouling up my development versions of 
numpy and mpl.  I never succeeded.  Granted, I didn't allocate much time 
and mental energy to it.

In any case, with the help of your recent explanations, I expect we can 
make mpl play by your suggested rules without sacrificing anything. 
Part of the change for mpl 1.0 was to factor all of the show logic out 
into backend_bases.py, leaving only the core mainloop call in the 
specific backends.  I hope we can keep that aspect, retaining the 
ability to work with earlier ipython (0.10) and the ability for show to 
block or not depending on the interactivity state.

Eric

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

[Brian G. <ell...@gm...>]

On Tue, Sep 14, 2010 at 12:59 AM, Eric Firing <ef...@ha...> wrote:
> On 09/13/2010 05:46 PM, Brian Granger wrote:
>>
>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...>  wrote:
>>>
>>> On 09/03/2010 12:37 PM, Brian Granger wrote:
>>>>
>>>> Hello all,
>>>>
>>>> I would like to submit the following branch on github for review and
>>>> merging into matplotlib trunk:
>>>>
>>>> http://github.com/ellisonbg/matplotlib/commits/guisupport
>>>>
>>>> This branch implements the logic needed for the qt4 and wx backends to
>>>> fully work with the upcoming IPython 0.11 release.  In our testing we
>>>> have run many of the mpl examples (including the new animation
>>>> examples) in both qt4/wx in both the terminal based IPython and the
>>>> new IPython Qt GUI.  For background on these changes please see this
>>>> thread:
>>>>
>>>>
>>>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>>>>
>>>> It is important to note that we have not updated the other matplotlib
>>>> backends (gtk, tk, etc.) to have this logic.  This is mainly because
>>>> we know almost nothing about these toolkits and could really use some
>>>> help from folks who are experts at the respective toolkits.  We have
>>>> done some minimal testing and these other backends do work for simple
>>>> examples in the terminal IPython, but they won't work in all cases and
>>>> will definitely not work in the new IPython Qt based GUI.
>>>>
>>>> We would love feedback and help testing as these changes are
>>>> significant (even though only a few lines of code).  To test this
>>>> stuff you will need to grab the following IPython development branch:
>>>>
>>>> http://github.com/ipython/ipython/tree/newkernel
>>>>
>>>> You should be able to run the examples in regular ipython:
>>>>
>>>> ipython --pylab qt4|wx
>>>>
>>>> Or the new GUI
>>>>
>>>> ipythonqt --pylab qt4|wx
>>>
>>> Brian, Fernando,
>>>
>>> I have been doing a little testing with ipython 0.10 versus
>>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>>> There are so many possible modes of operation and combinations of
>>> versions and backends that all this will take some time to sort out.
>>>
>>> Can you give me simple examples of what does *not* work correctly when
>>> you use mpl *svn* with ipython-newkernel, in either or both of the
>>> console or gui modes, but *does* work with your guisupport version?
>>
>> The problems are when matplotlib and enthought.traits/pyface/mayavi
>> stuff are used together.  There are two types of problems:
>>
>> * Multiple apps are created.  The enthought stuff used to not check
>> for existing apps, but that has been fixed.
>
> That one is easy.
>
>> * Event loop is started multiple times.  This one is more subtle and
>> on some toolkits the error is not fatal.  This problem shows up when
>> IPython is run in terminal mode and event loop integration is handled
>> through PyOS_InputHook.  In these situations, if matplotlib or ets
>> start the event loop themselves, IPython will hang.
>
> So, this can be checked with nothing but IPython and mpl. I think I may have
> seen this with some combination of configurations, though not with what I
> typically use.
>
>>
>> Unfortunately, I am having trouble getting an install of both
>> matplotlib svn and ets svn on the same machine, so I can't reproduce
>> any of the failures ATM.  I am trying to get things installed so I can
>> reproduce the problems.
>
> At least twice in the last couple of years I have tried to get mayavi
> compiled and running, without fouling up my development versions of numpy
> and mpl.  I never succeeded.  Granted, I didn't allocate much time and
> mental energy to it.

Yes, it can be a challenge, we will see how far I get...

> In any case, with the help of your recent explanations, I expect we can make
> mpl play by your suggested rules without sacrificing anything. Part of the
> change for mpl 1.0 was to factor all of the show logic out into
> backend_bases.py, leaving only the core mainloop call in the specific
> backends.  I hope we can keep that aspect, retaining the ability to work
> with earlier ipython (0.10) and the ability for show to block or not
> depending on the interactivity state.

The refactored show logic does help a lot.  I am pretty sure that my
branch already does all the things you want though.  If there are
things that it is missing we can definitely add them.  But when you
get a chance to look at this, let us know and we can continue the
discussion.

Cheers,

Brian



> Eric
>
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgr...@ca...
ell...@gm...