Menu

#44 Could PyX run a little quicker with large datafiles?

closed
nobody
None
5
2012-10-16
2006-07-06
No

Just to pass on a comment that I've received in the
alpha testing of PyXPlot: it has been remarked that
PyXPlot runs surprisingly slowly for plots with large
numbers of points on them -- say, more than 10^4.

Although I can conceive of few cases where it would be
useful to have a plot with so many datapoints, it would
be nice not to expect the user to downsample large
datasets before plotting, and it does seems surprising
that the performance is as slow as it is.

I did some profiling to see whether my code or PyX was
taking all the CPU cycles, and the answer seems to be
that both share some blame for this. In the attached
plot, I plotted datasets consisting of random numbers,
gradually increasing the number of datapoints (moving
along the x-axis). The lines represent the time taken,
in seconds, for a short python script to call PyX and
produce an EPS plot, then for the latest PyXPlot
release to do likewise, then for a development
optimised version of PyXPlot, and finally for gnuplot.
The benchmarking machine was a 2 GHz Athlon with 512 Mb
RAM.

Another related issue seems to be PyX's memory usage. I
found that to plot 10^6 datapoints, PyX rapidly needs
more than 100 Mb RAM, which seems very surprising.

I realise this is quite a vague problem, but have you
tried profiling PyX to see whether its performance can
trivially be improved?

Discussion

  • Dominic Ford

    Dominic Ford - 2006-07-06

    Plot of PyX's performance versus number of datapoints.

     
  • Jörg Lehmann

    Jörg Lehmann - 2006-08-17

    Logged In: YES
    user_id=390410

    Hello Dominic,

    we profile PyX from time to time to find out about the
    worst performance bottlenecks. On the other hand, the·
    development of PyX currently focuses on adding functionality
    rather than improving the perfomance. The latter might
    happen at a later stage. So while I appreciate the effort
    you put into profiling, at the moment there is not much we
    can do. I thus close the bug on Sourceforge but not in my
    mind. ;-)

    Jörg·
    
     

Log in to post a comment.