Re: [Matplotlib-users] large data sets and performance

[John H. <jdh...@ac...>]

>>>>> "Peter" == Peter Groszkowski <pgr...@ge...> writes:
    >> 
    Peter> With antialiasing off, the performance is superb!.. I plot
    Peter> 500,000 points in ~4-5 seconds.. The visual quality of the
    Peter> graphs is (naturally) inferior to the antialiased
    Peter> counterparts, but the software is now feasible for my
    Peter> purposes.

Glad to hear it...  The next big performance boost will come from some
frontend refactoring along the lines Perry discussed, but I'm glad to
hear it's usable for you now.

    Peter> 1) Seems like setting 'lod' to true does not improve
    Peter> performance?  I would imagine it should, because it limits
    Peter> the amount of points used. What am I missing?

I'll look into this further.  In the special case of EEG (128 channels
plotted simultaneously over the same time axis), I do see significant
benefits but I cache the sampling indexes from one line to the next.
It may be that for single lines the time it takes to do the
subsampling balances the time it takes to plot them in a fast backend.

    Peter> 2) Is there any way to make the graphs look "prettier"?
    Peter> They really look quite OK but in some cases having a little
    Peter> more detail would be nice. Is it possible specify just how
    Peter> much antialiasing is needed? Are there any other "visual
    Peter> enchantment options" that can be set, and will not impact
    Peter> performace too much?

Well, fortunately for you, I just finished the agg backend this
morning.  This backend draws antialiased lines as fast as GD draws
unaliased lines.  I still don't have support for turning off
antialiasing in agg, but it sounds like you want to have it.  Also, it
doesn't suffer from a known color allocation and fill bug that GD has.
See install instructions at the end of this email.

    Peter> 3) When I do:

        plot1 = plot(arange(10000), arange(20000,30000))
        lod, aa = False, False
        set(l, 'lod', lod, 'antialiased', aa)

This code isn't correct.  plot returns a list of lines.  The set
command should operate on that list of lines.  It applies only to the
lines returned.  So *you can* apply antialising with respect to one
set of lines and not another, in the same axes

        lines1 = plot(arange(10000), arange(20000,30000))
        set(lines1, 'antialiased', False)

        lines2 = plot([1,2,3])  # a small plot
        set(lines2, 'antialiased', True)

Now lines1 is aliased and lines2 is antialiased.

    Peter> I have been playing around with the dpi setting a
    Peter> little. Is it supposed to change the size of the image
    Peter> and/or the resolution??

The figure size in pixels is determined by the figsize parameter and
dpi.  

  width, height = figsize
  width *= dpi
  height *= dpi

Everything scales with DPI, line width, text size, dash spacing,
etc..  So the answer to your question is: both figure size and
resolution increase with dpi.  If you want to change figure size w/o
changing resolution, change the figsize argument to figure.

The agg backend

  Warning: you will be the first agg crash test dummy.  I just ran a
  suite of examples across all backends and agg was the fastest - it's
  even faster than template, which does no rendering or filesaving!
  And in my opinion it also produced the highest quality output.

  Features that are implemented

    * capstyles and join styles
    * dashes
    * linewidth 
    * lines, rectangles, ellipses, polygone
    * clipping to a rectangle
    * output to RGBA and PNG
    * alpha blending
    * DPI scaling -  (dashes, linewidths, fontsizes, etc)
    * freetype1

  TODO:
     * use ttf manager to get font - right now I just use Vera

  INSTALLING 

   Grab the latest matplotlib from
   http://nitace.bsd.uchicago.edu:8080/files/share/matplotlib-0.50l.tar.gz

  REQUIREMENTs

    python2.2+
    Numeric 22+
    agg2 (see below)
    freetype 1
    libpng
    libz ?
    
  Install AGG2 (cut and paste below into xterm should work)

    wget http://www.antigrain.com/agg2.tar.gz
    tar xvfz agg2.tar.gz
    cd agg2
    make

    (Optional) if you want to make the examples:
    cd examples/X11
    make

  Installing backend_agg

   Edit setup.py: change aggsrc to point to the agg2 src tree and
   replace if 0: with if 1: in the backend_agg section

   Then just do the usual thing: python setup.py build

   Please let me know if you encounter build problems, and tell me
   platform, gcc version, etc...  Currently the paths in setupext.py
   assume as linux like filesystem (eg X11 include dir, location of
   libttf, etcc) so you may need to tweak these.  But if I recall
   correctly, we're both on RHL9 so you shouldn't have a problem.

  Using agg backend

    python somefile.py -dAgg   

  or

    import matplotlib
    matplotlib.use('Agg')
      
Let me know how it works out!  Note also that backend agg is the first
backend to support alpha blending; see scatter_demo2.py.

JDH