RE: [Matplotlib-users] large data sets and performance

[Perry G. <pe...@st...>]

How are you plotting the data? As a scatter plot (e.g., symbols
or points) or as a connected line plot. The former can be quite
a bit slower and we have some thoughts about speeding that up
(which we haven't broached with JDH yet). How long is it taking
and how much faster do you need it?

Perry Greenfield

> -----Original Message-----
> From: mat...@li...
> [mailto:mat...@li...]On Behalf Of Peter
> Groszkowski
> Sent: Wednesday, February 11, 2004 2:14 PM
> To: mat...@li...
> Subject: [Matplotlib-users] large data sets and performance
> 
> 
> Hello:
> 
> We will be dealing with large (> 100,000 but in some instances as big as 
> 500,000 points) data sets. They are to be plotted, and I would like to 
> use matplotlib. I did a few preliminary tests, and it seems like 
> plotting that many pairs is a little too much for the system to handle. 
> Currently we are using (as a backend to some other software) gnuplot for 
> doing this plotting. It seems to be "lighting-fast" but I suspect (may 
> be wrong!) that it reduces this data before the plotting takes place, 
> and only selects every nth point. I have to go through the code that 
> calls it to be certain. I would imagine that it is not necessary to get 
> evrey point in 100,000 to produce a page-size plot, but I'm not sure if 
> simply grabbing every nth point and reducing the data like that is the 
> best way about this. So my question is to anyone else out there who is 
> also dealing with these large (and very large) data sets? What do you 
> do? Any library routines that you use before plotting to massage that 
> data? Are there any ways (ie flags to set) to optimize this in 
> matplotlib? Any other software you use? I should note that I use the GD 
> backend and pipe the output to stdout for a cgi scrpit to pick up.
> 
> Thanks.
> 
> -- 
> Peter Groszkowski         Gemini Observatory
> Tel: +1 808 974-2509      670 N. A'ohoku Place
> Fax: +1 808 935-9235      Hilo, Hawai'i 96720, USA
> 
> 
> 
> 
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>