Menu
▾
▴
matplotlib-devel
|
From: Ryan M. <rm...@gm...> - 2008-03-13 17:00:33
|
Hi, I've been poking around with pcolor, trying to see what makes it tick, since its performance is biggest thing that drives me nuts about matplotlib. I do pcolor plots of weather radar data with ~100000 polygons in them. Unfortunately, these polygons are at best trapezoids, so I can't treat it as image data. With this data, pcolor takes ages to zoom and pan, even on my new workstaion. My end goal is to be able to use OpenGL to do some 2D rendering, since in my experience, this simply flies at rendering such data for interactive analysis. I noticed that when I run the pcolor_demo.py (using current SVN trunk), 29396 separate calls are registered to RendererGDK->draw_path to simply draw the image the first time. Is there any reason why these can't be batched up and passed in one block to the renderer? This would make life easier in setting up OpenGL to render in one pass. Thanks Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma |
|
From: Eric F. <ef...@ha...> - 2008-03-13 17:38:02
|
Ryan, The pcolor implementation is fundamentally unsuited to large arrays. Therefore I made the pcolorfast axes method, which tries to use the fastest available Agg extension code, depending on the characteristics of the spatial grid. If the grid is rectangular and regular in both directions it uses a slight modification of the image code; if it is rectangular but with irregular spacing, it uses the nonuniform image code; and if it is not rectangular it uses the quadmesh code. It sounds like what you need is the quadmesh version, which you can access either via pcolormesh or pcolorfast. Neither is exposed via the pylab or pyplot API at present; both are axes methods. The pcolorfast API also may change slightly in the future; it probably needs a little more work. The quadmesh code has problems with masked arrays in the released version of mpl, but not in the svn version. It is *much* faster than pcolor, but may not be fast enough for your needs. If you are looking into what sounds like an OpenGL backend, or component to a backend, then the place to start is still probably pcolormesh or pcolorfast, not pcolor. Eric Ryan May wrote: > Hi, > > I've been poking around with pcolor, trying to see what makes it tick, > since its performance is biggest thing that drives me nuts about > matplotlib. I do pcolor plots of weather radar data with ~100000 > polygons in them. Unfortunately, these polygons are at best trapezoids, > so I can't treat it as image data. With this data, pcolor takes ages to > zoom and pan, even on my new workstaion. My end goal is to be able to > use OpenGL to do some 2D rendering, since in my experience, this simply > flies at rendering such data for interactive analysis. > > I noticed that when I run the pcolor_demo.py (using current SVN trunk), > 29396 separate calls are registered to RendererGDK->draw_path to simply > draw the image the first time. Is there any reason why these can't be > batched up and passed in one block to the renderer? This would make > life easier in setting up OpenGL to render in one pass. > > Thanks > > Ryan > |
|
From: Eric F. <ef...@ha...> - 2008-03-13 18:27:28
|
Shannon Jaeger wrote: > Hi Ryan and Eric, > > I work on software that plots bits and pieces of radio astronomy > data. We've run into similar issues and will be running into > even more related to performance. > > We have datasets that have millions of points, potentially into > the billions in the not so distant future. Sometimes the way > the users have choosen what bits and pieces to plot we need to > break up the data into chunks and plot each chunk separately. > To further complicate we allow them to overplot other large plots > on top. > > Anyway I'm very interested in learning more about how to use the > quadmesh code and using it as well. Although I haven't played > much with the source code at all, any advice on how to expose this > at the API level would be appreciated. If modifying the backend > is all that's needed we've already done a bit of this to tailor > things to our needs. Shannon, I'm not sure what you mean by "expose this at the API level"; quadmesh is already exposed via the pcolormesh and pcolorfast axes methods. Do you mean via a pyplot/pylab wrapper? This can be done easily, but my intention was to work out the kinks in the pcolorfast method API before wrapping it in pyplot, and possibly deprecate pcolormesh on the grounds that its functionality are included in pcolorfast. Instead of a proliferation of variations on this type of plot, I was hoping to consolidate everything into no more than two alternatives: the original pcolor, which can draw outlines around the boxes and draws only unmasked boxes; and pcolorfast, which omits those pesky outlines, and uses the fastest available algorithm for the type of data that are provided. Eric > > Thanks for any suggestions in advance. > > Shannon > ------------ > Shannon Jaeger > Atacama Large Millimetre Array > Dept. of Physics & Astronomy > University of Calgary, > Calgary, AB, Canada > email: sdj...@uc... > ph: +1 403 210 8495 > > > > On Thu, March 13, 2008 11:37 am, Eric Firing said: >> Ryan, >> >> The pcolor implementation is fundamentally unsuited to large arrays. >> Therefore I made the pcolorfast axes method, which tries to use the >> fastest available Agg extension code, depending on the characteristics >> of the spatial grid. If the grid is rectangular and regular in both >> directions it uses a slight modification of the image code; if it is >> rectangular but with irregular spacing, it uses the nonuniform image >> code; and if it is not rectangular it uses the quadmesh code. It sounds >> like what you need is the quadmesh version, which you can access either >> via pcolormesh or pcolorfast. Neither is exposed via the pylab or >> pyplot API at present; both are axes methods. The pcolorfast API also >> may change slightly in the future; it probably needs a little more work. >> >> The quadmesh code has problems with masked arrays in the released >> version of mpl, but not in the svn version. It is *much* faster than >> pcolor, but may not be fast enough for your needs. >> >> If you are looking into what sounds like an OpenGL backend, or component >> to a backend, then the place to start is still probably pcolormesh or >> pcolorfast, not pcolor. >> >> Eric >> >> Ryan May wrote: >>> Hi, >>> >>> I've been poking around with pcolor, trying to see what makes it tick, >>> since its performance is biggest thing that drives me nuts about >>> matplotlib. I do pcolor plots of weather radar data with ~100000 >>> polygons in them. Unfortunately, these polygons are at best trapezoids, >>> so I can't treat it as image data. With this data, pcolor takes ages to >>> zoom and pan, even on my new workstaion. My end goal is to be able to >>> use OpenGL to do some 2D rendering, since in my experience, this simply >>> flies at rendering such data for interactive analysis. >>> >>> I noticed that when I run the pcolor_demo.py (using current SVN trunk), >>> 29396 separate calls are registered to RendererGDK->draw_path to simply >>> draw the image the first time. Is there any reason why these can't be >>> batched up and passed in one block to the renderer? This would make >>> life easier in setting up OpenGL to render in one pass. >>> >>> Thanks >>> >>> Ryan >>> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Matplotlib-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> >> >> > > |
|
From: Ryan M. <rm...@gm...> - 2008-03-14 20:25:41
|
Eric Firing wrote: > Ryan, > > The pcolor implementation is fundamentally unsuited to large arrays. > Therefore I made the pcolorfast axes method, which tries to use the > fastest available Agg extension code, depending on the characteristics > of the spatial grid. If the grid is rectangular and regular in both > directions it uses a slight modification of the image code; if it is > rectangular but with irregular spacing, it uses the nonuniform image > code; and if it is not rectangular it uses the quadmesh code. It sounds > like what you need is the quadmesh version, which you can access either > via pcolormesh or pcolorfast. Neither is exposed via the pylab or > pyplot API at present; both are axes methods. The pcolorfast API also > may change slightly in the future; it probably needs a little more work. Yeah, I think for purposes of plotting radar data, quadmesh is what i need. > The quadmesh code has problems with masked arrays in the released > version of mpl, but not in the svn version. It is *much* faster than > pcolor, but may not be fast enough for your needs. My latest testing with the trunk with both pcolorfast and pcolormesh would seem to indicate that it's still a bit too slow for me. > If you are looking into what sounds like an OpenGL backend, or component > to a backend, then the place to start is still probably pcolormesh or > pcolorfast, not pcolor. This is the approach I'm considering. I've been poking around, and managed to create something that rendered, but not too well. However this was basically modifying the Gtk backend, and I'm sure there are some Gtk things that could be replaced. I noticed there are some other draw_* methods in other backends that aren't in the gtk, like draw_quad_mesh and draw_rectangle. Is there a definitive list somewhere? I tried RendererBase in backend_bases, but it did not have draw_rectangle. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma |
|
From: Michael D. <md...@st...> - 2008-03-18 18:42:24
|
Ryan May wrote: > Eric Firing wrote: > >> Ryan, >> >> The pcolor implementation is fundamentally unsuited to large arrays. >> Therefore I made the pcolorfast axes method, which tries to use the >> fastest available Agg extension code, depending on the characteristics >> of the spatial grid. If the grid is rectangular and regular in both >> directions it uses a slight modification of the image code; if it is >> rectangular but with irregular spacing, it uses the nonuniform image >> code; and if it is not rectangular it uses the quadmesh code. It sounds >> like what you need is the quadmesh version, which you can access either >> via pcolormesh or pcolorfast. Neither is exposed via the pylab or >> pyplot API at present; both are axes methods. The pcolorfast API also >> may change slightly in the future; it probably needs a little more work. >> > > Yeah, I think for purposes of plotting radar data, quadmesh is what i need. > > >> The quadmesh code has problems with masked arrays in the released >> version of mpl, but not in the svn version. It is *much* faster than >> pcolor, but may not be fast enough for your needs. >> > > My latest testing with the trunk with both pcolorfast and pcolormesh > would seem to indicate that it's still a bit too slow for me. > > >> If you are looking into what sounds like an OpenGL backend, or component >> to a backend, then the place to start is still probably pcolormesh or >> pcolorfast, not pcolor. >> > This is the approach I'm considering. I've been poking around, and > managed to create something that rendered, but not too well. However > this was basically modifying the Gtk backend, and I'm sure there are > some Gtk things that could be replaced. I noticed there are some other > draw_* methods in other backends that aren't in the gtk, like > draw_quad_mesh and draw_rectangle. Is there a definitive list > somewhere? I tried RendererBase in backend_bases, but it did not have > draw_rectangle. The pure Gtk backend has no optimizations for draw_quad_mesh. You may find that the Agg (e.g. GtkAgg) backend is significantly faster with quad meshes because it uses optimized C++ code to draw the mesh. The Gtk version must make (as you discovered) a Python function call for each quadrilateral. The definitive list of methods on rendering backends is in backend_bases.py. There are "required" methods, and methods that may be provided for optimization reasons (the latter are the ones that mention having a "fallback" implementation in RendererBase). Hope that helps, Mike |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X