From: John H. <jdh...@ac...> - 2004-07-09 02:35:14
|
What's new in matplotlib-0.60.1 * figure images (pixel-by-pixel, not resampled) with the figimage command. Multiple figure images (ie mosaics) with alpha blending are supported. See http://matplotlib.sf.net/examples/figimage_demo.py * multiple axes images with imshow using alpha blending. See http://matplotlib.sf.net/screenshots.html#layer_images * unified color limit and color mapping arguments to pcolor, scatter, imshow and figimage. Interactive control of colormap and color scaling with new matplotlib.matlab commands jet, gray and clim. New matplotlib rc parameters for default image params. image origin can be upper or lower - see http://matplotlib.sf.net/examples/image_origin.py * colorbar - http://matplotlib.sf.net/matplotlib.matlab.html#-colorbar - now works with imshow, pcolor, and scatter * new 'draw' command to redraw the figure - use this in place of multiple calls to show. This is equivalent to doing get_current_fig_manager().canvas.draw(), but takes less typing :-) * support for py2exe - see http://matplotlib.sf.net/py2exe_examples.zip * New finance demo shows off many of the features of matplotlib - see screenshot at http://matplotlib.sf.net/screenshots.html#finance_work2 * new matplotlib.matlab command 'rc' for dynamic control of rc parameters. See http://matplotlib.sf.net/matplotlib.matlab.html#-rc and example http://matplotlib.sf.net/examples/customize_rc.py * Andrew Straw submitted a dynamic_image example. The wx version is still in progress and has some flicker problems, but the gtk version is pretty cool - http://matplotlib.sf.net/examples/dynamic_image_gtkagg.py * Bug fixes: dynamic_demo_wx, figure legends, memory leaks, axis scaling bug related to singleton plots, mathtext bug for '6', some numarray bug workarounds See http://matplotlib.sf.net/CHANGELOG for details Downloads at http://sourceforge.net/projects/matplotlib Enjoy! JDH |
From: Gregory L. <gre...@ff...> - 2004-07-15 11:10:39
|
Hi, I try to benchmark some modif I did to speed up Agg rendering (basically, avoid re-creation of an Agg renderer if draw is called without changing previous fig size and DPI)... To do so, I changed the dynamic_image demo to use TkAgg (except my fltk backend, it's the only one working on my workstation for the moment, I do not have Wx not GTK). First tests shows no improvement :-(, but in fact I can not reproduce the speed mentioned when you discussed this demo (4 FPS or 10 FPS), only got 0.9 FPS. I thus tryed to check why I got such slow animation, and the results are as follow (I use the current CVS matplotlib, with numarray): Example with no drawing (manager.canvas.draw commented out in the updatefig method) : stabilize around 50 FPS, this is the best we can hope using numarray which is, I think, the only limitting factor in this case)... Example with call to tkagg.blit but no Agg canvas drawing (done replacing the draw method in FigureCanvasTkAgg, see below * ): stabilize around 19 FPS * old draw: def draw(self): FigureCanvasAgg.draw(self) tkagg.blit(self._tkphoto,self.renderer._renderer, 2) self._master.update_idletasks() new version: def draw(self): try: tkagg.blit(self._tkphoto,self.renderer._renderer, 2) except: FigureCanvasAgg.draw(self) tkagg.blit(self._tkphoto,self.renderer._renderer, 2) self._master.update_idletasks() Example with Agg canvas drawing + blitting: (normal TkAgg backend): stabilize around 1 FPS It seems thus that Agg drawing is the main limiting factor here, all the tricks to avoid using strings (or reallocating Agg renderer, for that matter) are not too usefull... What I do not understand is why I got such low values, compared to the 4 or 10 FPS: I guess, given the impact of Agg drawing, all the *Agg backends should have about the same speed...Is there something I miss here? My workstation is not current top of class, but it's a PIV 2.3 GHz, so certainly not slow either...I do not think the graphic subsystem is at fault, cause except for a mistake of my part, blit only test shows that Agg is really the origin of the poor FPS... Any idea about this? Thanks, Best regards, Greg. ---- Dynamic_image_tkagg.py ---- #!/usr/bin/env python """ An animated image """ import sys, time, os, gc from matplotlib import rcParams import matplotlib matplotlib.use("TkAgg") from matplotlib.matlab import * import Tkinter as Tk fig = figure(1) a = subplot(111) x = arange(120.0)*2*pi/120.0 x = resize(x, (100,120)) y = arange(100.0)*2*pi/100.0 y = resize(y, (120,100)) y = transpose(y) z = sin(x) + cos(y) im = a.imshow( z, cmap=cm.jet)#, interpolation='nearest') manager = get_current_fig_manager() cnt = 0 tstart = time.time() class loop: def __init__(self, master): self.master = master self.updatefig() # start updating def updatefig(self): global x, y, cnt, start x += pi/15 y += pi/20 z = sin(x) + cos(y) im.set_array(z) manager.canvas.draw() cnt += 1 if not cnt%20: print 'FPS', cnt/(time.time() - tstart) self.master.after(1, self.updatefig) cnt = 0 loop(manager.canvas._tkcanvas) show() |
From: John H. <jdh...@ac...> - 2004-07-15 14:49:35
|
>>>>> "Gregory" == Gregory Lielens <gre...@ff...> writes: Gregory> It seems thus that Agg drawing is the main limiting Gregory> factor here, all the tricks to avoid using strings (or Gregory> reallocating Agg renderer, for that matter) are not too Gregory> usefull... What I do not understand is why I got such Gregory> low values, compared to the 4 or 10 FPS: I guess, given Gregory> the impact of Agg drawing, all the *Agg backends should Gregory> have about the same speed...Is there something I miss Gregory> here? My workstation is not current top of class, but Gregory> it's a PIV 2.3 GHz, so certainly not slow either...I do Gregory> not think the graphic subsystem is at fault, cause except Gregory> for a mistake of my part, blit only test shows that Agg Gregory> is really the origin of the poor FPS... My best guess: your numerix settings don't agree. This will cause very poor performance, since the extension code has to fall back on the python sequence API (is this actually the correct explanation of why it's slow, Todd?) http://matplotlib.sourceforge.net/faq.html#SLOW To make sure, rm -rf the matplotlib build dir and the site-packages/matplotlib install dir and rebuild with NUMERIX = 'numarray' in setup.py, and make sure numerix is set to numarray in your rc file. I get 10FPS on the example you posted (3.4GHz P4). It's a faster machine than yours, but it's not 10 times faster. If I use numarray in my rc file and build with Numeric, I get 1.6FPS. JDH |
From: Todd M. <jm...@st...> - 2004-07-15 15:12:25
|
On Thu, 2004-07-15 at 10:25, John Hunter wrote: > >>>>> "Gregory" == Gregory Lielens <gre...@ff...> writes: > > Gregory> It seems thus that Agg drawing is the main limiting > Gregory> factor here, all the tricks to avoid using strings (or > Gregory> reallocating Agg renderer, for that matter) are not too > Gregory> usefull... What I do not understand is why I got such > Gregory> low values, compared to the 4 or 10 FPS: I guess, given > Gregory> the impact of Agg drawing, all the *Agg backends should > Gregory> have about the same speed...Is there something I miss > Gregory> here? My workstation is not current top of class, but > Gregory> it's a PIV 2.3 GHz, so certainly not slow either...I do > Gregory> not think the graphic subsystem is at fault, cause except > Gregory> for a mistake of my part, blit only test shows that Agg > Gregory> is really the origin of the poor FPS... > > My best guess: your numerix settings don't agree. This will cause very > poor performance, since the extension code has to fall back on the > python sequence API (is this actually the correct explanation of why > it's slow, Todd?) I believe that's correct. If matplotlib is compiled against Numeric, the array API calls don't see arrays, they see sequences (numarrays) which must be converted into (Numeric) arrays. This adds both constructor overhead and sequence protocol overhead (almost certainly the dominant factor for all array sizes). Regards, Todd |
From: Gregory L. <gre...@ff...> - 2004-07-15 17:17:16
|
John Hunter wrote: > > My best guess: your numerix settings don't agree. This will > cause very > > poor performance, since the extension code has to fall back on the > > python sequence API (is this actually the correct > explanation of why > > it's slow, Todd?) Todd Miller wrote: > I believe that's correct. If matplotlib is compiled against > Numeric, the array API calls don't see arrays, they see > sequences (numarrays) which must be converted into (Numeric) > arrays. This adds both constructor overhead and sequence > protocol overhead (almost certainly the dominant factor for > all array sizes). Thanks a lot, I think it was that indeed! My fault for not reading the FAQ! Now I got around a 5 time increase in FPS, not the 10 FPS you have on your computer but not too bad... :-) I have redone my timings: Classic TkAgg: 4.99 FPS "Improved" TkAgg with no Agg realloc when h,w, DPI is constant: 5.18 FPS FltkAgg (same as improved TkAgg, but use a new Agg tobuffer_rgba method to reuse the Agg buffer instead of copying it: this is possible using the fltk toolkit): 6.3 FPS I still mainly measure the Agg drawing performance, so I did a new test to check with a lighter drawing (included below, an annular mode (order 5) animation...) Here are the timings for thoses: Classic TkAgg: 9.98 FPS Improved TkAgg: 10.6 FPS FltkAgg: 16.7 FPS These timings are with figures of the same size (it has an influence on the FPS of course) So it seems my optimisation has an impact, although moderate... On the other hand, the copy mechanism induce some lag in the TkAgg backend, while reusing the buffer in FltkAgg seems a nice improvement...To check that, I disabled the copy in the TkAgg (tkagg.blit call), and got 16.4 FPS). I think thus my FltkAgg backend has the same speed as bare Agg, while some optim are maybe possible on TkAgg (if Tk can reuse an extern buffer, I am a complete beginner in Tk so maybe my conclusion are invalid, if there is a flaw in my examples... Depending on what you think of that, I can submit patches for the Agg optimisation, exposing the Agg buffer as a python buffer object (allowing buffer sharing instead of buffer copying, if toolkit support this). For the fltk backend, I am ready to support it but it should wait acceptance of some modif I made in the python bindings of fltk, for now it does not work with stock pyfltk bindings... Best Regards, Greg. |
From: John H. <jdh...@ac...> - 2004-07-15 18:31:28
|
>>>>> "Gregory" == Gregory Lielens <gre...@ff...> writes: Gregory> So it seems my optimisation has an impact, although Gregory> moderate... On the other hand, the copy mechanism induce Gregory> some lag in the TkAgg backend, while reusing the buffer Gregory> in FltkAgg seems a nice improvement...To check that, I Gregory> disabled the copy in the TkAgg (tkagg.blit call), and got Gregory> 16.4 FPS). I think thus my FltkAgg backend has the same Gregory> speed as bare Agg, while some optim are maybe possible on Gregory> TkAgg (if Tk can reuse an extern buffer, I am a complete Gregory> beginner in Tk so maybe my conclusion are invalid, if Gregory> there is a flaw in my examples... I didn't see where the optimization was helping - could you clarify? It looks like about a 5% for tkagg and there are no comparisons for the fltkagg with "classic vs optimized" Classic TkAgg: 9.98 FPS Improved TkAgg: 10.6 FPS FltkAgg: 16.7 FPS As you may know, blitting in tk is pretty slow and there doesn't appear to be anything we can do about it short of some platform dependent extension code (which might be worth doing at some point for linux and win32). With your example I get approx 14 FPS with tkagg on my system and 45 FPS with gtkagg. My point is that unless there are some fundamental limitations in fltk as there are in tk, you might want to look to gtkagg as a performance benchmark rather than tkagg. gtkagg is typically 3x faster than tkagg for dynamic plots. Gregory> Depending on what you think of that, I can submit patches Gregory> for the Agg optimisation, exposing the Agg buffer as a Gregory> python buffer object (allowing buffer sharing instead of Gregory> buffer copying, if toolkit support this). For the fltk Gregory> backend, I am ready to support it but it should wait Gregory> acceptance of some modif I made in the python bindings of Gregory> fltk, for now it does not work with stock pyfltk Gregory> bindings... Please send them in, either to the list or directly to Todd and myself. Todd will be interested in anything pertaining to tkagg. I understand the rational behind not creating a new agg buffer object with each draw, but at the same time your numbers seem to suggest that it isn't too important, performance wise. Also, I am not sure about the necessity of creating a python buffer object - perhaps you can explain this a bit more. tkagg and gtkagg both use the agg rendering buffer directly with no string copy, eg RendererAgg* aggRenderer = static_cast<RendererAgg*>(args[1].ptr()); //gtk gdk_draw_rgb_32_image(drawable, gc, 0, 0, width, height, GDK_RGB_DITHER_NORMAL, aggRenderer->pixBuffer, width*4); // tk block.pixelPtr = aggRenderer->pixBuffer; where pixBuffer is the agg pixel buffer. What does exposing the buffer in the python layer buy you? It is best (for me) for you to submit your changes as complete files so I can merge / compare with ediff; my dev tree is often out of sync with the non-devel cvs tree so applying diffs is hard. Thanks! JDH |
From: Gregory L. <gre...@ff...> - 2004-07-15 19:40:33
|
> I didn't see where the optimization was helping - could you > clarify? It looks like about a 5% for tkagg and there are no > comparisons for the fltkagg with "classic vs optimized" > > Classic TkAgg: 9.98 FPS > Improved TkAgg: 10.6 FPS > FltkAgg: 16.7 FPS I will check the non optimized (ie with no reuse of Agg buffers) FltkAgg backend tomorrow, but you are right, the improvement are reproductible but small. The change for reusing buffer is trivial though, and it may help if there is memory leak... > As you may know, blitting in tk is pretty slow and there > doesn't appear to be anything we can do about it short of > some platform dependent extension code (which might be worth > doing at some point for linux and win32). With your example > I get approx 14 FPS with tkagg on my system and 45 FPS with > gtkagg. My point is that unless there are some fundamental > limitations in fltk as there are in tk, you might want to > look to gtkagg as a performance benchmark rather than tkagg. > gtkagg is typically 3x faster than tkagg for dynamic plots. You are right, I should definitely install pygtk on my system to check that, I wonder how a 3X increase in FPS is possible: after all, de-activating blit in the TkAgg produce only a 1.5X increase (and no image of course ;-) ): something else than Agg must hold Tk (and Fltk) back then: I have to test the FPS one have without Agg drawing and blitting, only canvas update, with GTK, Tk and Fltk, to be sure... > Please send them in, either to the list or directly to Todd > and myself. Todd will be interested in anything pertaining > to tkagg. Ok, I send the complete relevant files as attachement tomorrow, to you and Todd. > I understand the rational behind not creating a > new agg buffer object with each draw, but at the same time > your numbers seem to suggest that it isn't too important, > performance wise. It could be more important for GTKAgg, if it is closer to raw Agg speed than TkAgg and FltkAgg...Something to test would be to use the Agg backend without exporting the pixmap buffer but doing the drawing, and check how many draw/second one can have on the examples, to really check the penalty associated to the different toolkits, and the maximum gain we can have by reusing Agg buffer instead of creating one for each draw...Is this doable? > Also, I am not sure about the necessity of > creating a python buffer object - perhaps you can explain > this a bit more. tkagg and gtkagg both use the agg rendering > buffer directly with no string copy, eg > > RendererAgg* aggRenderer = static_cast<RendererAgg*>(args[1].ptr()); > > > > //gtk > > gdk_draw_rgb_32_image(drawable, gc, 0, 0, > width, > height, > GDK_RGB_DITHER_NORMAL, > aggRenderer->pixBuffer, > width*4); > // tk > block.pixelPtr = aggRenderer->pixBuffer; > > where pixBuffer is the agg pixel buffer. > > What does exposing the buffer in the python layer buy you? In these cases nothing performance wise, in fact the way you use should me marginally faster (avoiding the creation/destruction of python buffer objects ). Python buffer objects is a way for me to implement the transfer without copy of the Agg buffer to fltk in more "abstract" way: no need to implement a c extension that know both the internals of agg and fltk, I split it using the python buffer as a standard protocol (buffer objects were intended for just this use, I think...). This is not very important, but it could simplify things if there is multiple renderer and multiple toolkits to bind...like if alternative to Agg is implemented (or multiple version of Agg must be supported)...and I guess that the overhead of this creation/destruction of python buffer objects is really negligeable. Best regards, Greg. |
From: Perry G. <pe...@st...> - 2004-07-15 19:48:08
|
> > You are right, I should definitely install pygtk on my system to check > that, I wonder how a 3X increase in FPS is possible: after all, > de-activating blit in the TkAgg produce only a 1.5X increase (and no > image of course ;-) ): something else than Agg must hold Tk (and Fltk) > back then: I have to test the FPS one have without Agg drawing and > blitting, only canvas update, with GTK, Tk and Fltk, to be sure... > I guess I'm surprised at that. How are you disabling the blit? If no blit is being done, where else is the time going if not for agg? Puzzled, Perry |
From: Gregory L. <gre...@ff...> - 2004-07-15 20:13:23
|
> > You are right, I should definitely install pygtk on my > system to check > > that, I wonder how a 3X increase in FPS is possible: after all, > > de-activating blit in the TkAgg produce only a 1.5X > increase (and no > > image of course ;-) ): something else than Agg must hold Tk > (and Fltk) > > back then: I have to test the FPS one have without Agg drawing and > > blitting, only canvas update, with GTK, Tk and Fltk, to be sure... > > > I guess I'm surprised at that. How are you disabling the > blit? If no blit is being done, where else is the time going > if not for agg? In the draw method of the FigureCanvasTkAgg class (in matplotlib/backends/backend_tkagg.py file), I just commented out the second line, going from: def draw(self): FigureCanvasAgg.draw(self) tkagg.blit(self._tkphoto, self.renderer._renderer, 2) self._master.update_idletasks() To: def draw(self): FigureCanvasAgg.draw(self) #tkagg.blit(self._tkphoto, self.renderer._renderer, 2) self._master.update_idletasks() This makes the image dissapear from screen of course, but still draw the image in the in-memory Agg buffer...and ask the Tk canvas to redraw itself (allways with the same empty image), I guess...so I indeed suspect that the Tk canvas updating is causing the slow down somehow...I will check that tomorrow for sure! Greg, puzzled too ;-) |
From: Perry G. <pe...@st...> - 2004-07-15 19:50:28
|
Though I suppose it may simply be in the mechanism that tk uses to update it's own buffer (it does its own blitting behind the scenes as I understand it)... |
From: John H. <jdh...@ac...> - 2004-07-15 20:17:41
|
>>>>> "Gregory" == Gregory Lielens <gre...@ff...> writes: Gregory> You are right, I should definitely install pygtk on my Gregory> system to check that, I wonder how a 3X increase in FPS Gregory> is possible: after all, de-activating blit in the TkAgg Gregory> produce only a 1.5X increase (and no image of course ;-) The gtk advantage over tk depends on the example. For images (eg for Andrew's dynamic image) the advantage is approx 1.5x. For the simple line drawing dyanmic example you posted, it's about 3x. I suppose the difference is that in the image example, there is a much larger, shared computational burden in agg, whereas with the simple line drawing example the blitting difference is more pronounced. Gregory> It could be more important for GTKAgg, if it is closer to Gregory> raw Agg speed than TkAgg and FltkAgg...Something to test Gregory> would be to use the Agg backend without exporting the Gregory> pixmap buffer but doing the drawing, and check how many Gregory> draw/second one can have on the examples, to really check Gregory> the penalty associated to the different toolkits, and the Gregory> maximum gain we can have by reusing Agg buffer instead of Gregory> creating one for each draw...Is this doable? I'm certainly happy to try it. One last thing you may want to do before sending in your updates tomorrow is to run backend_driver.py in the examples subdir to make sure your changes don't create any problems with the known examples. Gregory> In these cases nothing performance wise, in fact the way Gregory> you use should me marginally faster (avoiding the Gregory> creation/destruction of python buffer objects ). Python Gregory> buffer objects is a way for me to implement the transfer Gregory> without copy of the Agg buffer to fltk in more "abstract" Gregory> way: no need to implement a c extension that know both Gregory> the internals of agg and fltk, I split it using the Gregory> python buffer as a standard protocol (buffer objects were Gregory> intended for just this use, I think...). This is not Gregory> very important, but it could simplify things if there is Gregory> multiple renderer and multiple toolkits to bind...like if Gregory> alternative to Agg is implemented (or multiple version of Gregory> Agg must be supported)...and I guess that the overhead of Gregory> this creation/destruction of python buffer objects is Gregory> really negligeable. Another area it may help is in backend_gtk and backend_wx, which *do* use string methods to access the agg image buffer. JDH |
From: Gregory L. <gre...@ff...> - 2004-07-16 15:38:58
|
I just checked (after spending all day solving network problem :-() what effect the re-use of Agg buffer optimisation has on FltkAgg backend: Classic TkAgg: 9.98 FPS Improved TkAgg: 9.98 FPS classic FltkAgg: 16.1 FPS Improved FltkAgg: 17.2 FPS I also installed pyGtk to be able to test the framerate I gor on may computer: Classic GTKAgg: 16 FPS Modified GTKAgg (avoid switch backend, which make my re-use of agg buffer impossible, use multiple inheritance instead): 16 FPS Improved modified GTKAgg: 16.8 FPS And finaly, to have a complete picture, I tried Tk without blitting (as explained in a previous message) classic TkAgg without blit: 14.3 FPS improved TkAgg without blit: 16 FPS There is various strange things in these benchmarks: first I was not able to reproduce the (very small but reproductible) advantage I observed yesterday for Agg buffer re-use optimization in the TkAgg backend...It is observable in the FltkAgg and TkAgg backend, though...and even more so in the TkAgg without blitting one...Maybe this is maked by the blitting time in the normal TkAgg, though, so this does not disturb me too much... Next, I do not observe the high performance of the GTKAgg backend here, on the contrary the FPS it gives me are in line with the Fltk ones, and even with Tk when we get rid of the slow blitting...This is quite surprising... On the other hand, I have less trouble understanding these results than the higher GTKAgg performance you report, so to be really complete I tried an bare Agg rendering: Just use the same example as my light dynamic plot (mode animation), but with a matplotlib.use("Agg") and a while True: updatefig() loop instead of idle callback... Here are the timings: Classic Agg: 19 FPS (or, better said, RenderingPerSecond) Optimized Agg: 20.55 RPS So this confirm what I have observed on the various *Agg backends, which in summary would be: -Optimization of Agg, to reuse buffer if possible: Gain from 8% to 0%, depending on the backend (max on Agg, small to non existent on TkAgg). It depends also on the complexity of the rendering, the gain will be higher when simple drawings are done, and minimal when very complex figures are drawn...This could help for memory leak maybe, though...and as you will see is a very minimal hack... Performance of the various *Agg backends, using Agg as reference and a very simple dynamic plot (for the new reuse Agg buffer scheme, current "new Agg every draw" should be very similar): TkAgg: 49% (ouch!) TkAgg without blit: 78% GTKAgg: 82% FltkAgg: 84% So Fltk and GTK are fast (the 20 % overhead is due to transfer to screen buffer, double buffering, and callbacks/idle mechanisms, I guess, no way to get better than that) TkInter is a slower toolkit, mainly cause of blit, and also for other reasons it seems. Only remaining mystery (but it is a big one!) is why you observe very different things for GTKAgg? Is it a GTK version problem? A compilation option? This is really surprising, given the bare Agg test give me RPS in line with my FPS...Only thing I can think of is a option during Agg compile that decrease the performance of my Agg somehow... You will find included a tar.gz of all the files I modified (including small examples and my FltkAgg backend - even if it is not too usefull before pyfltk has been updated)...If you need any more information or want to discuss this, I would be glad to help :-) PS: I forward this to matplotlib-devel, without the attachment: I do not know if the mailinglist would accept such a thing... |
From: Gregory L. <gre...@ff...> - 2004-07-15 17:21:06
|
Oups- forgot to include the "light" dynamic example: #!/usr/bin/env python """ An animated image """ import sys, time, os, gc from matplotlib import rcParams import matplotlib matplotlib.use("TkAgg") from matplotlib.matlab import * import Tkinter as Tk fig = figure(1) a = subplot(111) a = arange(121.0)*2*pi/120.0 dR = 0.1*sin(5*a) x_0=sin(a) y_0=cos(a) line = plot(x_0,y_0) axis([ -1.5,1.5, -1.5, 1.5 ]) manager = get_current_fig_manager() cnt = 0 tstart = time.time() t=0 class loop: def __init__(self, master): self.master = master self.updatefig() # start updating def updatefig(self): global t,x_0,y_0, dR, cnt, start,tstart t += pi/20 R=1+sin(t)*dR line[0].set_data(R*x_0,R*y_0) manager.canvas.draw() cnt += 1 if not cnt%100: print 'FPS', 100.0/(time.time() - tstart) tstart=time.time() self.master.after(1, self.updatefig) cnt = 0 loop(manager.canvas._tkcanvas) show() |