matplotlib-users

[Matplotlib-users] Scatter plots and GTK backend

[Dominique O. <do...@da...>]

Hello,

I am having trouble viewing scatter plots of large data sets
with the GTK backend. The plots are sparsity patterns of
sparse matrices, much in the way of Matlab's spy function.
First of all, there may be a conflict somewhere with my
version of GTK/PyGTK; issuing a plot() or scatter() command
often results in the message 'None Active' being displayed.
After a couple of seconds, the python prompt comes back, and
the 'scatter' command results in a huge number of messages
of the form 

    <matplotlib.patches.Circle instance at 0x4531730c>

begin displayed; litterally hundreds of them---it takes some
10 seconds. Finally, a show() opens up a Figure window, but
no plot. The matrix has 80519  nonzero elements and is
symmetric, so the scatter plot contains roughly twice as
many points. I have a Gnuplot interface and a spy-like
function which works just fine and displays the pattern in a
matter of a fraction of a second.

Sometimes, a plot using the GTK backend gets stuck in
gtk.mainloop(), and i have to interrupt it with Ctrl-C. I am
using SuSE Linux 8.0. I have installed the most recent
versions of GTK and PyGTK.

Thanks in advance for any suggestions.
Dominique

Re: [Matplotlib-users] Scatter plots and GTK backend

[John H. <jdh...@ac...>]

>>>>> "Dominique" == Dominique Orban <do...@da...> writes:

    Dominique> Hello, I am having trouble viewing scatter plots of
    Dominique> large data sets with the GTK backend. The plots are
    Dominique> sparsity patterns of sparse matrices, much in the way
    Dominique> of Matlab's spy function.  First of all, there may be a
    Dominique> conflict somewhere with my version of GTK/PyGTK;
    Dominique> issuing a plot() or scatter() command often results in
    Dominique> the message 'None Active' being displayed.  After a
    Dominique> couple of seconds, the python prompt comes back, and
    Dominique> the 'scatter' command results in a huge number of
    Dominique> messages of the form

    Dominique>     <matplotlib.patches.Circle instance at 0x4531730c>

Hi Dominique, 

It looks like there is a "print" statement somewhere in your code.
It's possible that this was from a vestigial debug command I left in.
I don't get get it on my system.  Are you using matplotlib-0.50?  I
also don't get the "None Active" line.

I'm using this as a test script  

    from matplotlib.matlab import *

    x = 100*rand(100000)
    y = 100*rand(100000)
    s = rand(100000)

    scatter(x,y,s)
    #plot(x,y,'o')
    show()

Takes about 30s on my system.  Note that plot with circles can be must
faster that scatter if you don't need to vary the size or color of the
symbols.

    Dominique> begin displayed; litterally hundreds of them---it takes
    Dominique> some 10 seconds. Finally, a show() opens up a Figure
    Dominique> window, but no plot. The matrix has 80519 nonzero
    Dominique> elements and is symmetric, so the scatter plot contains
    Dominique> roughly twice as many points. I have a Gnuplot
    Dominique> interface and a spy-like function which works just fine
    Dominique> and displays the pattern in a matter of a fraction of a
    Dominique> second.

There are several areas where matplotlib performance is subpar -
mostly for large numbers of patches (circles for scatter, rectangles
for pcolor).  Fixing this is a fairly high priority and I have a good
idea how to go about it - see
http://sourceforge.net/mailarchive/message.php?msg_id=7142332 for a
recent discussion.  I think in the next 3-4 weeks I can get this
fixed.  Basically, the plan is to set up an additional backend method
or two that the various backends may optionally implement in extension
code for performance.

    Dominique> Sometimes, a plot using the GTK backend gets stuck in
    Dominique> gtk.mainloop(), and i have to interrupt it with
    Dominique> Ctrl-C. I am using SuSE Linux 8.0. I have installed the
    Dominique> most recent versions of GTK and PyGTK.

Are you using the default GTK that comes with SuSE or did you upgrade?
I have gotten myself into a world of pain before trying to upgraded
GTK libs on a linux box.  It does look like you are getting some
unusual behavior.  Make sure you are using the latest matplotlib and
try running the test script I posted above.  If you still get the same
errors, something is whacked with your install or paths.  Otherwise,
stay tuned for performance enhancements coming soon to theaters
everywhere.

JDH

Re: [Matplotlib-users] Scatter plots and GTK backend

[Dominique O. <do...@da...>]

* On Mon, 01 Mar 2004, John Hunter wrote:

> It looks like there is a "print" statement somewhere in your code.
> It's possible that this was from a vestigial debug command I left in.
> I don't get get it on my system.  Are you using matplotlib-0.50?  I
> also don't get the "None Active" line.

I am in Windows 2000 and XP. I am using 0.50e in Linux, because i am
upgrading my gcc. The newer 0.50 won't compile with the gcc i have now.
I only get the 'None Active' in Linux though. I suspect it comes from
my GTK. I am using KDE and installing Gnome libraries when i need them.
There might be a glitch somewhere.
 
> I'm using this as a test script  
> 
>     from matplotlib.matlab import *
> 
>     x = 100*rand(100000)
>     y = 100*rand(100000)
>     s = rand(100000)
> 
>     scatter(x,y,s)
>     #plot(x,y,'o')
>     show()
> 
> Takes about 30s on my system.  Note that plot with circles can be must
> faster that scatter if you don't need to vary the size or color of the
> symbols.

I see that plot is usually faster than scatter. The script works fine
with smaller data sets, and takes a pretty long time with larger sets.
 
> There are several areas where matplotlib performance is subpar -
> mostly for large numbers of patches (circles for scatter, rectangles
> for pcolor).  Fixing this is a fairly high priority and I have a good
> idea how to go about it - see
> http://sourceforge.net/mailarchive/message.php?msg_id=7142332 for a
> recent discussion.  I think in the next 3-4 weeks I can get this
> fixed.  Basically, the plan is to set up an additional backend method
> or two that the various backends may optionally implement in extension
> code for performance.

That would be wonderful. I have never looked into the Gnuplot code but
they have a fairly efficient algorithm. Plotting data sets like that i
described takes less than a second on my laptop (from the Python
interface). It would be great if matplotlib could do it too.
 
> Are you using the default GTK that comes with SuSE or did you upgrade?
> I have gotten myself into a world of pain before trying to upgraded
> GTK libs on a linux box.  It does look like you are getting some
> unusual behavior.  Make sure you are using the latest matplotlib and
> try running the test script I posted above.  If you still get the same
> errors, something is whacked with your install or paths.  Otherwise,
> stay tuned for performance enhancements coming soon to theaters
> everywhere.

You can be sure i'm staying tuned.

Dominique

Re: [Matplotlib-users] Scatter plots and GTK backend

[John H. <jdh...@ac...>]

>>>>> "Dominique" == Dominique Orban <do...@da...> writes:

    Dominique> I am in Windows 2000 and XP. I am using 0.50e in Linux,
    Dominique> because i am upgrading my gcc. The newer 0.50 won't
    Dominique> compile with the gcc i have now.  I only get the 'None
    Dominique> Active' in Linux though. I suspect it comes from my
    Dominique> GTK. I am using KDE and installing Gnome libraries when
    Dominique> i need them.  There might be a glitch somewhere.

Note, you can just set all the BUILD_* flags at the top of setup.py
and matplotlib won't try and compile anything.  It will install all of
the pre 0.50 modules (GTK, WX, PS, etc) but you won't have access to
Agg.

For the record, what gcc were you using and what error message did you
get?  

Thanks,
JDH

Re: [Matplotlib-users] Scatter plots and GTK backend

[Dominique O. <do...@da...>]

* On Tue, 02 Mar 2004, John Hunter wrote:
>
> >>>>> "Dominique" == Dominique Orban <do...@da...> writes:
> 
>     Dominique> I am in Windows 2000 and XP. I am using 0.50e in Linux,
>     Dominique> because i am upgrading my gcc. The newer 0.50 won't
>     Dominique> compile with the gcc i have now.  I only get the 'None
>     Dominique> Active' in Linux though. I suspect it comes from my
>     Dominique> GTK. I am using KDE and installing Gnome libraries when
>     Dominique> i need them.  There might be a glitch somewhere.
> 
> Note, you can just set all the BUILD_* flags at the top of setup.py
> and matplotlib won't try and compile anything.  It will install all of
> the pre 0.50 modules (GTK, WX, PS, etc) but you won't have access to
> Agg.

Yes i will try that to start with. In the end, i'll want to use Agg
though, given the apparent quality of the plots.
 
> For the record, what gcc were you using and what error message did you
> get?  

gcc version 2.95.3 20010315 (SuSE)

I wanted to try the Intel c++ compiler for Linux, but am not sure how
to tell setup.py which compiler i want to use and with which options.

'python setup.py build' eventually gives

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -Isrc -Iagg2/include -I/usr/include/freetype1 -I/usr/local/include/python2.3 -c src/_backend_agg.cpp -o build/temp.linux-i686-2.3/src/_backend_agg.o
In file included from /usr/local/include/python2.3/Python.h:8,
   from src/_backend_agg.h:40,
   from src/_backend_agg.cpp:8:
  /usr/local/include/python2.3/pyconfig.h:847: warning: `_POSIX_C_SOURCE' redefined
  /usr/include/features.h:171: warning: this is the location of the previous definition
  src/_backend_agg.cpp:286: multiple storage classes in declaration of `RendererAgg_Type'
error: command 'gcc' failed with exit status 1

Many thanks for the help.
Dominique