From: Andrew S. <as...@in...> - 2003-10-01 01:12:34
|
Hi John et al., I just had a chance to play with matplotlib for a few minutes, and I'm very encouraged! A feature request: I frequently use Adobe Illustrator to touch up postscript files that contain my figures. In this case, it is REALLY handy when the postscript files group nicely. Knowing little-to-nothing about PostScript and Illustrator, I have no idea how hard the behavior is to implement, but it would be fantastic if it did. I just tried a few things with the axes_demo and the errorbar_demo in the examples directory. I liked that the points grouped together. I didn't like that in the errorbar_demo that the points and the errorbars grouped together almost inseparably. With the demos tested, the primary curve or points grouped with a rectangle around the plotting region that had no fill or stroke but seemed to clip the contents to within that box. I wonder if it would be nicer to produce postscript output where the clipping is done before rendering to a file, thus eliminating the need for this rather strangely behaved box? Also, the generated plots have some two boxes, one with a white stroke and one with a white fill, surrounding the figure. These, too, seem unnecessary. Cheers! Andrew |
From: John H. <jdh...@ac...> - 2003-10-01 03:48:32
|
>>>>> "Andrew" == Andrew Straw <as...@in...> writes: Andrew> Hi John et al., I just had a chance to play with Andrew> matplotlib for a few minutes, and I'm very encouraged! Glad to hear it .... keep me informed of bugs and feature requests. Andrew> A feature request: I frequently use Adobe Illustrator to Andrew> touch up postscript files that contain my figures. In this Andrew> case, it is REALLY handy when the postscript files group Andrew> nicely. Knowing little-to-nothing about PostScript and Andrew> Illustrator, I have no idea how hard the behavior is to Andrew> implement, but it would be fantastic if it did. I know nothing about Illustrator, and have been learning postscript as I go, so bear with me. Andrew> I just tried a few things with the axes_demo and the Andrew> errorbar_demo in the examples directory. I liked that the Andrew> points grouped together. I didn't like that in the Andrew> errorbar_demo that the points and the errorbars grouped Andrew> together almost inseparably. What do you mean by "grouped together". I assume this has something to do with editing in Illustrator, but can you explain in more detail? Learning (a little bit of) postscript has been a mind opening experience. I know a lot of programming languages, and postscript introduced me to several new ideas. It is difficult to take a (somewhat) state independent OO representation of a graphical object and translate it into the postscript state machine efficiently, especially, when the postscript backend has to act like the other backends at the interface level. Simple example: suppose you want to draw all the axis tick labels, each of which has the same font information. The abstract interface makes a separate call for each label, which causes the postscript backend to generate the same font information over and over again. A smart postscript backend keep track of this information so it wouldn't needlessly regenerate the information leading to file bloat. I would like to make these improvements, but my first goal was to get something that works. Most of the improvements I've envisioned for the PS backend have been in the realm of file size efficiency (I've seen some damn large PS files in my day). So I'm interested to get your feedback about these other areas that I don't yet understand. Andrew> With the demos tested, the primary curve or points grouped Andrew> with a rectangle around the plotting region that had no Andrew> fill or stroke but seemed to clip the contents to within Andrew> that box. I wonder if it would be nicer to produce Andrew> postscript output where the clipping is done before Andrew> rendering to a file, thus eliminating the need for this Andrew> rather strangely behaved box? Could you also give me some detail here? Is the "box" the rectangular border of the axes? With regards to a specific demo, what is "the primary curve" and "rectangle"? I do use postscript clipping of lines and other objects etc so that they do not extend beyond the axes borders. Generally, I think this is *a good thing*. The general organization of matplotlib is figures contain axes which contain lines, patches and text. Normally, I don't want lines, patches and text spilling out of their axes containers. Can you explain a little more what you are trying to achieve in Illustrator so I can get a better idea of what is missing? What exactly is the 'strangely behaved box'? Andrew> Also, the generated plots have some two boxes, one with a Andrew> white stroke and one with a white fill, surrounding the Andrew> figure. These, too, seem unnecessary. Yes, this is a holdover from the GUI. In a GUI presentation, the plots look nicer with a boundary -- see eg, http://matplotlib.sourceforge.net/screenshots/subplot_demo_large.png where the gray border is the default figure background -- matlab does this. So the figure (which contains the axes) renders a rectangular border with a fill color. For the postscript backend, I simply made these white and when I print on white paper, I never see them. They can easily be done away with by commenting out the line self._figurePatch.draw(drawable) in backends/backend_ps.py. I don't really have a problem removing it entirely as I don't see much need for it in the PS backend, unless someone wants to frame their plots with background rectangle. I mainly left it in their for vestigial compatibility with the other backends. But, so I can get a better understanding of the twisted mind of Illustrator, could you explain to me what kind of problem this is causing you? JDH |
From: Andrew S. <as...@in...> - 2003-10-01 05:59:12
|
John Hunter wrote: > Andrew> I just tried a few things with the axes_demo and the > Andrew> errorbar_demo in the examples directory. I liked that the > Andrew> points grouped together. I didn't like that in the > Andrew> errorbar_demo that the points and the errorbars grouped > Andrew> together almost inseparably. > > What do you mean by "grouped together". I assume this has something > to do with editing in Illustrator, but can you explain in more detail? Basically, there are two select modes in Illustrator. The first, "Selection tool", selects a whole group of paths. The second, the "Direct selection tool" selects the path segment or otherwise smallest path portion possible. Let's take the example of 5 circles which have been drawn and then grouped in Illustrator. Clicking on one with the selection tool will select all of them, because they are all grouped. Clicking one with the direct select tool will only get one (or actually probably only a single path component between anchor points or the anchor point itself if it was clicked). There is some way Illustrator extracts this information from all PostScript files, but it probably just makes intelligent guesses when it's dealing with "foreign" PS. (I think it must embed Illustrator-specific comments or other directives when it saves an "Illustrator .eps".) > Learning (a little bit of) postscript has been a mind opening > experience. I know a lot of programming languages, and postscript > introduced me to several new ideas. It is difficult to take a > (somewhat) state independent OO representation of a graphical object > and translate it into the postscript state machine efficiently, > especially, when the postscript backend has to act like the other > backends at the interface level. (Sounds like OpenGL!) I know less about PS than you, but it seems one way to go about doing what you describe is to build a virtual PS engine and render to it, and have it spit out only the state-changing instructions it received. > A smart postscript backend keep track of this information so it wouldn't > needlessly regenerate the information leading to file bloat. I would > like to make these improvements, but my first goal was to get > something that works Always a good first step! :) Seriously, I realized that things were probably not very baroque yet, so I thought I'd pipe up to let you know about what is, IMHO, an important feature of good PS rendering. > Andrew> With the demos tested, the primary curve or points grouped > Andrew> with a rectangle around the plotting region that had no > Andrew> fill or stroke but seemed to clip the contents to within > Andrew> that box. I wonder if it would be nicer to produce > Andrew> postscript output where the clipping is done before > Andrew> rendering to a file, thus eliminating the need for this > Andrew> rather strangely behaved box? > > Could you also give me some detail here? Is the "box" the rectangular > border of the axes? Yes. > With regards to a specific demo, what is "the > primary curve" and "rectangle"? The "primary curve" consists of the main data points, either plotted as points/circles (in the case of the errorbar demo) or as line segments (in the axes_demo). "box" == "rectangle". > I do use postscript clipping of lines and other objects etc so that > they do not extend beyond the axes borders. Yes, I see what you mean -- with this clipping box, the leftmost circle in the errorbars demo does not extend beyond the axes, and is therefore half cut-off. I'm not sure if this is desirable or not, but at least with the current behavior I could just go in and remove the clipping box. FYI, the circles, the errorbars (vertical lines), the "caps" on the bars, and the clipping rectangle all group together in Illustrator. There is something that seems inconsistent to me with the current behavior -- the lower error "caps" that are completely beyond the clipping rectangle aren't present in the PS file at all. However, the errorbar does extend below the clipping rectangle to the position where the cap would be. Would things be more consistent if, when a clipping rectangle is used to do the clipping, all primitives get rendered and only the clipping rectangle handles clipping? > Generally, I think this > is *a good thing*. The general organization of matplotlib is figures > contain axes which contain lines, patches and text. Normally, I don't > want lines, patches and text spilling out of their axes containers. Yes, I just wonder about the explicit-ness of a decision about whether it's matplotlib or PS that does clipping. I don't know enough to feel strongly, but if file-size is a factor, it should presumably be done by matplotlib. On the other hand, I think optimizations (even for file size) should happen later and for now maybe rendering everything to PS and letting it handle clipping is best. On the third(!) hand, huge files are clearly undesirable and perhaps the best plan is what seems to already be done -- any primitives totally outside the clipping area aren't drawn, but otherwise, they are drawn with PS itself doing the clipping. This point is just food for thought. > Can you explain a little more what you are trying to achieve in > Illustrator so I can get a better idea of what is missing? What > exactly is the 'strangely behaved box'? "strangely behaved" == if you remove one corner from a clipping rectangle, it then becomes a clipping triangle that leaves half of your plot normal and the other half disappears. This happens up to some distance away from the corner you just deleted. That's why I call it strangely behaved, but I think I do understand it. > Andrew> Also, the generated plots have some two boxes, one with a > Andrew> white stroke and one with a white fill, surrounding the > Andrew> figure. These, too, seem unnecessary. > > Yes, this is a holdover from the GUI. In a GUI presentation, the > plots look nicer with a boundary -- see eg, > http://matplotlib.sourceforge.net/screenshots/subplot_demo_large.png > where the gray border is the default figure background -- matlab does > this. So the figure (which contains the axes) renders a rectangular > border with a fill color. For the postscript backend, I simply made > these white and when I print on white paper, I never see them. They > can easily be done away with by commenting out the line > > self._figurePatch.draw(drawable) > > in backends/backend_ps.py. I don't really have a problem removing it > entirely as I don't see much need for it in the PS backend, unless > someone wants to frame their plots with background rectangle. I > mainly left it in their for vestigial compatibility with the other > backends. But, so I can get a better understanding of the twisted > mind of Illustrator, could you explain to me what kind of problem this > is causing you? No real problem, I'm just (mildly) against idea of invisible primitives in PS files. (This probably stems from me dealing with PS output from matlab5 many years ago when I remember sorting through layer after layer after layer of "strangely behaved rectangles" just to manipulate my data. It's quite funny to me that matplotlib produces the most matlab-like PS files I've seen in a while! Still nowhere near the number of layers, though!) Anyhow, I'd love to dive into the code and help you with the PS/Illustrator improvements, but I have no time at the moment... Cheers! Andrew |
From: John H. <jdh...@ac...> - 2003-10-01 12:48:09
|
>>>>> "Andrew" == Andrew Straw <as...@in...> writes: Andrew> Basically, there are two select modes in Illustrator. The Andrew> first, "Selection tool", selects a whole group of paths. Andrew> The second, the "Direct selection tool" selects the path Andrew> segment or otherwise smallest path portion possible. Hi Andrew -- thanks for the detailed information vis-a-vis Illustrator. Things are clearer to me now, but it looks like I need to get my hands on Illustrator and play with a few plots to get a better feel for it. I am curious what kinds of editing you are doing with your plots. The reason I ask is that I wanted to make sure you know that you can control a lot from within matplotlib using the return values of the functions. These return the text, line, and patch objects, and you can use any method in their API to control them, including position, size, linewidth, colors, symbols, etc. Ditto for the axes -- their position, line styles, and colors can all be controlled with the matplotlib API symbols, lines = errorbar(t, s, e, fmt='o') set(lines, 'linewidth', 3) set(symbols, 'markerfacecolor', 'g') set(symbols, 'markeredgecolor', 'r') set(symbols, 'markersize', 10) If you want to have all the error bars in the axes, you can change the ylimits of the axes with set(gca(), 'ylim', [-1,10]) Likewise, you can control the location and properties of the tickmarks, ticklabels, etc... Not that this obviates the need to make a sane PS renderer, but I just wanted to make sure you were aware of the possibility. I generally try to do everything in code and minimize the amount of manual editing because invariably I have to regenerate the figure with new data or new stats and then I have to redo everything manually again. So if you tell me the kinds of things you're trying to do, perhaps I can point you to existing features that provide this, or include some of them in the matplotlib API. This would also give me a better idea of how to think about redesigning the PS engine to make it more palatable to PS editors. Andrew> Yes, I just wonder about the explicit-ness of a decision Andrew> about whether it's matplotlib or PS that does clipping. I Andrew> don't know enough to feel strongly, but if file-size is a Andrew> factor, it should presumably be done by matplotlib. On Andrew> the other hand, I think optimizations (even for file size) Andrew> should happen later and for now maybe rendering everything Andrew> to PS and letting it handle clipping is best. On the Andrew> third(!) hand, huge files are clearly undesirable and Andrew> perhaps the best plan is what seems to already be done -- Andrew> any primitives totally outside the clipping area aren't Andrew> drawn, but otherwise, they are drawn with PS itself doing Andrew> the clipping. This point is just food for thought. This is generally the approach I take. lines handle clipping themselves. This lets me plot really long signals (5 minutes of EEG) with only 10 second windows on the screen at the time -- a matplotlib clip. In addition, I rely on the graphics context clip (GTK / PS) to handle the little things like circle markers extending outside the axes lines, the top of a curve that extends past the top of the axes, etc.... You can read the gory details in the _set_clip function in the lines module, when you need a distraction from your thesis. Without too much work, I can probably make a "no clip" an option of the figure or axes if this would be useful. Andrew> No real problem, I'm just (mildly) against idea of Andrew> invisible primitives in PS files. (This probably stems Andrew> from me dealing with PS output from matlab5 many years ago Andrew> when I remember sorting through layer after layer after Andrew> layer of "strangely behaved rectangles" just to manipulate Andrew> my data. It's quite funny to me that matplotlib produces Andrew> the most matlab-like PS files I've seen in a while! Oh, now that was a low blow -- matlab5! And just what kind of manipulation of your data are you doing -- you're not trying to cheat on those error bars are you <wink>? John Hunter |
From: Charles R. T. <ct...@ma...> - 2003-10-02 04:28:26
|
Andrew, John's right, you almost certainly want to remove the manual editing step. Think code. It saves lots of time and is reproducible. Let's see if your tasks are easily code-able. John, the PS backend is great -- just used it in a paper -- but after Nathan's post I'm wondering about effort duplication. I looked over the examples on the PyX website (pyx.sourceforge.net): they _are_ impressive. I don't want to offend, but I'm wondering if we could use PyX for the postscript backend, or otherwise merge? Two great tastes and all... -Charles }I am curious what kinds of editing you are doing with your plots. }The reason I ask is that I wanted to make sure you know that you can -- Charles R. Twardy, Res.Fellow, Monash University, School of CSSE ctwardy at alumni indiana edu +61(3) 9905 5823 (w) 5146 (fax) "in much of the rest of the world, rich people live in gated communities and drink bottled water." --Jared Diamond |
From: John H. <jdh...@ac...> - 2003-10-02 15:05:17
|
>>>>> "Charles" == Charles R Twardy <ct...@ma...> writes: Charles> John, the PS backend is great -- just used it in a paper Charles> -- but after Nathan's post I'm wondering about effort Charles> duplication. I looked over the examples on the PyX Charles> website (pyx.sourceforge.net): they _are_ impressive. I didn't see any post by Nathan on this subject. But my email has been a little flaky over the last month -- could you forward it to me? Charles> I don't want to offend, but I'm wondering if we could use Charles> PyX for the postscript backend, or otherwise merge? Two Charles> great tastes and all... Wow, pyx looks great. I didn't even know it existed; I'm not sure how I missed it. pyx has an advantage over matplotlib because it is designed around postscript, which as you know is an extremely powerful drawing program. It is, however, difficult to implement as an abstract renderer, eg, to set up an interface that takes advantage of postscript's power while still being able to render to a GTK drawing area. That said, I think your suggestion is a good one, to have a pyx backend where matplotlib draws to a pyx canvas and then the user can have access to that canvas to add things like latex markup, fancy stuff that matplotlib can't do, etc.... I haven't had time to dive into pyx yet, but it is certainly seems suitable for something like that. Had I seen pyx before starting on the PS backend, I certainly would have used it. That said, I don't regret doing it myself since 1) it gave me the chance to learn some postscript and 2) now matplotlib can generate PS with the only dependence being Numeric. pyx has some C code in it, which makes it more difficult for win32 users since they have to have the extension compiled for their particular python version. I wrote an EEG and CT analysis application which must run on linux and win32 that depends on around 10-15 other packages, and it is a pain to get all packages for a given version of python. I've spent long hours trying to compile them myself if a win installer is not available. But I suspect most people wanting postscript/tex are on a linux/unix platform. Thanks for the suggestion, John Hunter |
From: Charles R. T. <ct...@ma...> - 2003-10-02 23:10:56
|
John, Looking at pyx was Nathan's main point. He also included a very short program that showed how to preview PyX output in Python: import pygtk; pygtk.require("2.0") import sys import gtk import bonobo import bonobo.ui win = gtk.Window() win.connect("delete-event", gtk.mainquit) win.show() container = bonobo.ui.Container() control = bonobo.ui.Widget("file://home/ctwardy/Library/mcallister.pdf", container.corba_objref()) # A control widget is just like any other GtkWidget. control.show() win.add(control) gtk.main() }Wow, pyx looks great. I didn't even know it existed; I'm not sure how I didn't know either. } }That said, I don't regret doing it myself since... Absolutely. -- Charles R. Twardy, Res.Fellow, Monash University, School of CSSE ctwardy at alumni indiana edu +61(3) 9905 5823 (w) 5146 (fax) "Incongruous places often inspire anomalous stories." -- S.J. Gould |
From: Andrew S. <as...@in...> - 2003-10-09 02:08:36
|
> Wow, pyx looks great. I didn't even know it existed; I'm not sure how While we're pointing to potential backends, may I suggest (for the distant future) that VTK be used as a backend once matplotlib goes 3D? I've been using (the Python bindings for) it for a while now and think it blows everything else out of the water. A matplotlib bridge to it would be awesome as a simple interface, and data could be automatically saved in a .vtk format, making arbitrarily sophisticated things possible. (Mayavi being a good intermediate.) Cheers! Andrew |
From: John H. <jdh...@ac...> - 2003-10-09 12:22:07
|
>>>>> "Andrew" == Andrew Straw <as...@in...> writes: Andrew> While we're pointing to potential backends, may I suggest Andrew> (for the distant future) that VTK be used as a backend Andrew> once matplotlib goes 3D? I've been using (the Python Andrew> bindings for) it for a while now and think it blows Andrew> everything else out of the water. A matplotlib bridge to Andrew> it would be awesome as a simple interface, and data could Andrew> be automatically saved in a .vtk format, making Andrew> arbitrarily sophisticated things possible. (Mayavi being a Andrew> good intermediate.) I have worked a lot with VTK and as part of a subdural 3D electrode localization application (python/pygtk/vtk) I wrote but haven't released yet (screenshots below). I was planning on going to VTK for the 3D support in the mythical matplotlib 1.0. VTK is truly an amazing library -- I was just reading the textbook again last night (which I recommend if you haven't read it -- the textbook is better than the user's guide). I think I could spend a year just trying to get my head fully wrapped around VTK. The one area of VTK that I haven't been impressed by is the fonts, which is one reason I didn't go with it when I made my initial choice of backends. The other reason is that is a big library so I was hesitant to make it a prerequisite. But now that enthought distributes python with vtk built-in, and it could be one backend among the others, it is a great idea. The VTK people would probably like it too, since it would ease chart/graph production for them. So much to do! Electrode localization program http://nitace.bsd.uchicago.edu:8080/summer/jdh/loc3djr_shot3.png High coherences between cortical electrodes http://nitace.bsd.uchicago.edu:8080/summer/jdh/coherence.png Thought you might be interested to see them. JDH |