Menu
▾
▴
matplotlib-users
|
From: Greg N. <no...@uc...> - 2005-02-23 05:56:38
|
Not everyone uses matplotlib inside of scripts! Judging from the
manual, this is the only approved way to use the library.
There seems to be no way to change the backend in the middle of an
interactive session. Looking through the manual, I find that I can
specify it on the command line, in .matplotlibrc, or via
matplotlib.use('..'), but only "before you import matplotlib.pylab"
I would be extremely happy if there were a way to change backends
midstream from an interactive shell (I use ipython, for example)
Thanks,
Greg
|
|
From: John H. <jdh...@ac...> - 2005-02-23 17:11:37
|
>>>>> "Greg" == Greg Novak <no...@uc...> writes:
Greg> Not everyone uses matplotlib inside of scripts! Judging
Greg> from the manual, this is the only approved way to use the
Greg> library.
Hmm. Not quite true. I hear there is a fellow in Boulder who
sometimes uses matplotlib interactively <wink>. If the manual gives a
different impression, that is unintentional, and may result from
selective reading. Did you read section 1.5 with subheading
"Interactive"? Eg,
The recommended way to use matplotlib interactively from a shell is
with ipython, which has an pylab mode that detects your matplotlib
.matplotlibrc file and makes the right settings to run matplotlib
with your GUI of choice in interactive mode using threading. gtk
users will need to make sure that they have compiled gtk with
threading for this to work. Using ipython in pylab mode is
basically a nobrainer because it knows enough about matplotlib
internals to make all the right settings for you internally.
Fernando and I have spent a lot of time and effort to make matplotlib
work seamlessly in interactive use. The issues are non-trivial
because you have to handle threading to keep the GUI from stealing the
show. Because threading is always involved when using most GUIs from
a python shell, we also provide an example in examples/interactive.py
which three mpl developers collaborated on to show people the
beginnings of how to roll their own custom interactive GTK shell.
I've also added a lot of short aliases for long keywords so you can
do, for example
>>> plot([1,2,3], ls='--', c='red')
instead of
>>> plot([1,2,3], linestyle='--', color='red')
Basically, the claim that we only support a scripting interface is not
true. If you have specific things to suggest to further improve
interactive use, or documentation suggestions, fire away.
Greg> There seems to be no way to change the backend in the middle
Greg> of an interactive session. Looking through the manual, I
Greg> find that I can specify it on the command line, in
Greg> .matplotlibrc, or via matplotlib.use('..'), but only "before
Greg> you import matplotlib.pylab"
matplotlib development usually happens when a developer needs a
feature or one or more users make a case that a feature is important
-- you are officially the second person to request the ability to
switch backends interactively, if memory serves.
The first person was Fernando Perez, who has driven many of the
improvements in interactive use because he makes lots of suggestions
and contributes code. Recently on matplotlib-devel, we discussed the
importance of being able to switch backends interactively
http://sourceforge.net/mailarchive/message.php?msg_id=10724264 and I
provided a pylab_interface function switch_backend which does just
that. As you'll see in that thread, Fernando is looking into ways to
include the backend switching functionality into ipython, eg so you
can run a script with
>>> run -backend PS somefile.py
But even w/o this ease of use, in the current matplotlib release the
pylab switch_backend function exists. It's not documented because we
are still exploring and testing it and it is experimental. But I
added a docstring this morning which I'll paste in here
def switch_backend(newbackend):
"""
Swtich the default backend to newbackend. This feature is
EXPERIMENTAL, and is only expected to work switching to an image
backend. Eg, if you have a bunch of PS scripts that you want to
run from an interactive ipython session, yuo may want to switch to
the PS backend before running them to avoid having a bunch of GUI
windows popup. If you try to interactively switch from one GUI
backend to another, you will explode.
Calling this command will close all open windows.
"""
Greg> I would be extremely happy if there were a way to change
Greg> backends midstream from an interactive shell (I use ipython,
Greg> for example)
Good. You should be extremely happy then. As I suggested above, this
feature is lightly tested, so let us know what you find.
JDH
|
|
From: Greg N. <no...@uc...> - 2005-02-23 06:29:03
|
Ok, I was so rattled when I wrote the previous message that I forgot to ask my actual question: I was trying to change backends midstream because I was only getting b/w plots when I used pcolor--I couldn't change the colormap. At first I thought that this was because I was using the Postscript backend and for some reason it assumed I wanted b/w output for printing. After fooling around with some of the other imaging backends, I used one of the interactive ones (this is a pain b/c I'm running Python remotely and I'm sitting at the end of a slow internet link. Therefore interactive windows take forever to pop up. More on this later), and the plot was still b/w. After significant frustration, I realized that my problem was that I was specifying the color map as 'cm=' instead of 'cmap='; a reasonable guess since the rest of the line is 'cm.hot' The problem was that matplotlib silently ignored my misspecified argument. This is gripe #1: weak typing only works when using undefined variables bites you on the nose. Otherwise we're back to the bad old days of fortran, when misspelling a variable name silently defined a new variable. It would be nice if somewhere in the heirarchy of function calls within matplotlib someone checked to make sure there were no lonely, unused keyword arguments sloshing around. I realize that this is hard to do robustly and with any generality, but there's a lot at stake. Python would be unusable if use-before-definition didn't generate an exception. Gripe #2 is that it would be nice if the same word were abbreviated the same way everywhere. Ie, either cmap or cm, not both. Gripe #3 is related to interactive windows when Python and the X11 server are connected by a slow link. All of the interactive backends I've used for matplotlib look great, but render very slowly in this situation b/c they're sending all the pixel data. Since most plots consist of a few lines, a bit of text, and a few dots, it'd be nice if the windows rendered quickly over slow connections. For example, Gnuplot runs very well in this configuration. I realize that this doesn't invovle matplotlib per se, I just wanted to throw it out there as a concern. I apologize for the negative tone of this mail. Don't get me wrong--I use matplotlib and I like it b/c it's the best thing I've found. However, I just had one of those "GAAARGHGGHHHH!" moments and it's taking a little while to de-stress. Cheers, Greg |
|
From: John H. <jdh...@ac...> - 2005-02-23 17:33:13
|
>>>>> "Greg" == Greg Novak <no...@uc...> writes:
Greg> This is gripe #1: weak typing only works when using
Greg> undefined variables bites you on the nose. Otherwise we're
This is a good point -- Norbert submitted a patch addressing this a
couple of months ago but it broke python2.2 so we unrolled it. I've
added it to my list of things to do. It's tricky to get right, but
it's manageable and important.
Greg> Gripe #2 is that it would be nice if the same word were
Greg> abbreviated the same way everywhere. Ie, either cmap or cm,
Greg> not both.
I don't think I agree with you here. matplotlib.cm is a *module* and
cm is a convenient reference to that module for pylab interactive
users who want to save on keystrokes. I think it would lead to
confusion and hard to find bugs if a module name and a kwarg were the
same. Now cm may not be the best module name, perhaps cmaps would be
better. I don't see this as a huge problem since a simple
>>> pcolor?
in ipython would have shown you the correct kwarg. And if mpl had
raised an exception on seeing cm as a kwarg as you suggest in gripe#1,
it wouldn't have been a problem for you.
BTW, do you know you can interactively change the colormap for an
existing plot with the jet, gray, hot, et al commands
>>> pcolor(rand(12,12)) # use your default colormap
>>> hot() # colormap is now hot
>>> summer() # colormap is now summer
Greg> Gripe #3 is related to interactive windows when Python and
Greg> the X11 server are connected by a slow link. All of the
Greg> interactive backends I've used for matplotlib look great,
Greg> but render very slowly in this situation b/c they're sending
Greg> all the pixel data. Since most plots consist of a few
Greg> lines, a bit of text, and a few dots, it'd be nice if the
Greg> windows rendered quickly over slow connections. For
Greg> example, Gnuplot runs very well in this configuration. I
Greg> realize that this doesn't invovle matplotlib per se, I just
Greg> wanted to throw it out there as a concern.
Perry already addressed this -- basically because we have so many
python GUIs it becomes a maintenance nightmare to add new features if
we use native drawing. Using agg to render to a bitmap which is then
transferred to a GUI canvas means we can add a feature in one place
and Tk, Wx, GTk, Fltk and QT users automagically get it with *no*
changes to the GUI backend. It comes with a cost, as you observed.
You do have the option of using a GUI native drawing backend, eg GTK
instead of GTKAgg or WX instead of WXAgg. Because we use double
buffering in GTK, it may not help (Steve -- would it be possible to
make double buffering optional?) The native GTK drawing is inferior
to Agg, in my view, but you may be willing to trade quality for
speed.
There is hope for you, however, that you can have the best of both
worlds. Steve says that GTK is moving to a Cairo renderer for GTK
2.8, which does provide nice graphics. And it is a happy confluence
of circumstances that 1) matplotlib has a Cairo backend, 2) the GTK
maintainer (Steve Chaplin) is also the author of the Cairo and
GTKCairo backends and 3) Steve is very good about keeping the GTK*
stuff up to the latest developments in the gtk world. So in the near
future, we may have a GTK backend that uses native drawing and looks
good too. In which case you may get speed over an X11 connection as
well as nice looking plots.
Greg> I apologize for the negative tone of this mail. Don't get
Greg> me wrong--I use matplotlib and I like it b/c it's the best
Greg> thing I've found. However, I just had one of those
Greg> "GAAARGHGGHHHH!" moments and it's taking a little while to
Greg> de-stress.
No problem. With all the fan mail we're getting around here, we're
starting to get a little soft :-). Some well placed criticism keeps
us on our toes.
Thanks,
JDH
|
|
From: Perry G. <pe...@st...> - 2005-02-23 14:21:32
|
On Feb 23, 2005, at 1:28 AM, Greg Novak wrote: > Gripe #3 is related to interactive windows when Python and the X11 > server are connected by a slow link. All of the interactive backends > I've used for matplotlib look great, but render very slowly in this > situation b/c they're sending all the pixel data. Since most plots > consist of a few lines, a bit of text, and a few dots, it'd be nice if > the windows rendered quickly over slow connections. For example, > Gnuplot runs very well in this configuration. I realize that this > doesn't invovle matplotlib per se, I just wanted to throw it out there > as a concern. > This is a consequence of how most of the interactive backends are implemented. Since matplotlib is using agg to render graphics for them, all updates to graphs mean a new image must be transmitted to the window. If it is a remote window, that update is going to cost. There may be ways of improving the backends so that not the whole window needs to be updated, but I have a feeling that isn't going to yield a big improvement in many cases (a new plot of points means changes over most of the image even if it can be characterized by a relatively small number of points and labels). I don't see any great solution to this (maybe John has some good ideas). I pointed this out as a consequence of going this way before it was done (and I recommended going this way :-). Implementing the backends using their built in plotting commands may be a way around this, but it means having much higher maintenance costs for backends and lots of annoying capability mismatches (some backends don't support rotated text, alpha blending, etc.) Perry |
|
From: Greg N. <no...@uc...> - 2005-02-24 06:18:07
|
Upfront disclaimer: like I said yesterday, I wrote those e-mails when
my frustration with a number of things completely boiled over. Then I
just core dumped everything that was bothering me into the mails.
Probably the only thing that's useful beyond the scope of the specific
problems I was experiencing are my comments about unused keyword
arguments.
* Perry Greenfield <pe...@st...> wrote:
>>Gripe #3 is related to interactive windows when Python and the X11
>>server are connected by a slow link.
> This is a consequence of how most of the interactive backends are
> implemented.
> ...
> Implementing the backends using their built in plotting commands
> may be a way around this, but it means having much higher maintenance
> costs for backends and lots of annoying capability mismatches (some
> backends don't support rotated text, alpha blending, etc.)
I figured it was something like this, and I can understand that the
benefits of a common rendering engine outweigh the admittedly small
benefit of fast remote X11 redraws. Furthermore I've more or less
solved the problem for myself by using non-interactive backends with a
combination of scripts and ssh tunnels that watch for changes in a
file on the remote machine and, when it is modified, sends the file
over the link, where (similarly) a script is watching for changes in
the _local_ file and lauches a viewer. Not sure why this seems to
make a difference (all the pixel data is going over the channel in
either case) but it seems to make a big difference.
* John Hunter <jdh...@ni...> wrote:
> Greg> Gripe #2 is that it would be nice if the same word were
> Greg> abbreviated the same way everywhere. Ie, either cmap or cm,
> Greg> not both.
> I don't think I agree with you here. matplotlib.cm is a *module* and
> cm is a convenient reference to that module for pylab interactive
> users who want to save on keystrokes. I think it would lead to
> confusion and hard to find bugs if a module name and a kwarg were the
> same.
Not sure why this is the case. I realize that they're different
entities in the language, but they both help you with the same task.
Context always (?) allows the Python interpreter to know the
difference, and the exceptions that are thrown by problems with each
are different. As a user, the thought that goes through my head when
I'm using either one is "Something related to colormaps" and there's
an extra cognitive load associated with remembering that "Modules
related to colormaps" requires cm while "Arguments related to
colormaps" requires cmap.
> I don't see this as a huge problem since a simple pcolor?
> in ipython would have shown you the correct kwarg.
True, but we're aiming high here. If I didn't care about things like
this, I'd use Fortran or C because everyone else does and I certainly
wouldn't have taken an interest in Python. If you can just guess
without referring to documentation and have software do what you want,
I think that's a sign of an elegant interface: it has anticipated my
desires.
> And if mpl had raised an exception on seeing cm as a kwarg as you
> suggest in gripe#1, it wouldn't have been a problem for you.
Quite true! The extra cognitive load would still be there, but
addressing grip #1 effectively would make this into a very small
annoyance rather than something that can cause real frustration.
> So in the near future, we may have a GTK backend that uses native
> drawing and looks good too. In which case you may get speed over an
> X11 connection as well as nice looking plots.
This sounds great. :-)
> Greg> Not everyone uses matplotlib inside of scripts! Judging
> Greg> from the manual, this is the only approved way to use the
> Greg> library.
> If the manual gives a different impression, that is unintentional,
> and may result from selective reading.
> ...
> Basically, the claim that we only support a scripting interface is
> not true.
I didn't mean to imply that this was a policy decision, only that it
seems to be an assumption that's made often in the manual.
Specifically the thing that set me off was what I referred to here:
> Greg> Looking through the manual, I
> Greg> find that I can specify it on the command line, in
> Greg> .matplotlibrc, or via matplotlib.use('..'), but only "before
> Greg> you import matplotlib.pylab"
I couldn't figure out why matplotlib.use(...) didn't seem to be doing
anything, and then I found the bit about it only working before you
import matplotlib.pylab. This is where my frustration boiled over,
and I dashed off the e-mail. That probably wasn't a great idea
(writing while angry) but what can I say? All of this together set me
off.
> matplotlib development usually happens when a developer needs a
> feature or one or more users make a case that a feature is important
> -- you are officially the second person to request the ability to
> switch backends interactively, if memory serves.
I'm certainly open to the idea of contributing code, but at this point
I'm obviously not in a position to do so since I'm still learning my
way around. Also, I can see that interactively switching
backends will be a seldom used feature. My desire for it (and
excessive frustration when I failed to find it) were due to an unhappy
confluence of events.
> that. As you'll see in that thread, Fernando is looking into ways to
> include the backend switching functionality into ipython, eg so you
> can run a script with
>>> run -backend PS somefile.py
It's true that this is done from an interactive Python shell, but I
wouldn't call this switching backends interactively. In general my
.py files contain only function definitions. So I would envision
being able to do things like this:
>>> %run defs.py
>>> z = read_data()
>>> make_plot(z)
>>> switch_backend('PS')
>>> make_plot(z)
Now, please keep in mind my comments above: that once I use mpl a
little more and establish a regular pattern of use, I don't expect
to need this feature much, if at all. I'm just commenting on the
example you gave.
> But even w/o this ease of use, in the current matplotlib release the
> pylab switch_backend function exists.
Cool, I'll certainly give it a shot.
Thanks,
Greg
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X