matplotlib

On Fri, Aug 29, 2008 at 12:01 PM, Ray Salem <raysalem@...> wrote:

>    This is my first post, I really enjoy using python/numpy/scipy/matplotlib
> - great replacement for matlab. I am having a problem, with plot 1 point at
> a time. i am attempt to emulate an issue in the lab, where we get samples
> every second and plot them. The figure becomes non-responsive and eventually
> stops plotting, does anyone have an idea.

Yep, this is a bad idea.  The line object in matplotlib is a fairly
heavy object which has a lot of stuff under the hood.  Each plot
command creates a new line object.  What you want to do is create just
one (or a few) line objects, each of which handles many points.  Below
is the first pass naive implementation to show you how to do this --
you will probably want to improve on this by using a better data
structure to store the growing points, eg a buffer that drops old
points off the end, or a dynamically resizing numpy array which grows
intelligently when it gets full.  The main point is that we create a
single line object and add the new data to it....

import time
import matplotlib
matplotlib.use('TkAgg')
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.collections as collections


plt.ion() # interactive plotting

fig = plt.figure()
ax = fig.add_subplot(111, autoscale_on=False)

xdata, ydata = [0.], [0.]
line, = ax.plot(xdata, ydata, 'o')

dt = 0.1
for i in range(100):
    x = xdata[-1] + dt
    y = np.sin(2*np.pi*x)
    xdata.append(x)
    ydata.append(y)
    line.set_data(xdata, ydata)
    ax.set_xlim(x-1, x)
    ax.set_ylim(-1.1, 1.1)
    fig.canvas.draw()
    time.sleep(0.1)


plt.show()

On Sat, Aug 30, 2008 at 9:49 AM, Ray Salem <raysalem@...> wrote:
> Hey John. Your suggestion has definitely made a large improvement, but while
> it is still running, if you select any of the windows, python or the figure,
> the figure will still enter a state of non-response. I have tried something
> very similar in matlab and have observed no problems. Do you have any other
> advice or suggestions

Please respond on list so others can benefit and contribute to the
conversation.  You need to "reply to all" to reply to the list.

In order to do this well, you need to use your user interface idle
handler.  The example I used was just to illustrate the line handling,
but for real work you should set you update function in either an idle
handler or a timeout handler.  Look in the examples/animation
directory http://matplotlib.sourceforge.net/examples/animation for
different examples for each user interface (see also the animations
cookbook tutorial at
http://www.scipy.org/Cookbook/Matplotlib/Animations).

The way to do this differs for each toolkit, and we have examples for
many in the examples directory I pointed you to above.  Does anyone
have an example of how to do proper idle handling with tk?

JDH

On Thursday 28 August 2008 09:17:26 John Hunter wrote:
> On Thu, Aug 28, 2008 at 6:19 AM, Sandro Tosi <matrixhasu@...> wrote:
> > Enthought suite is used a lot in scientific area, so mpl and enth
> > almost share their users, so have it enabled would be a plus, but we
> > are mainly interested in generate "no harm", so we'd like to ask you
> > to confirm that enabling that support won't brake anything (but indeed
> > provide a valuable asset for users).
>
> Well, one thing to make sure of is that you do not install the version
> of traits that ships with matplotlib, since it will break other
> enthought packages.  My understanding is that one either uses the
> default rc config or the traits enabled one.  Since the latter is
> experimental and we are not sure if we will eventually adopt it, I
> would not recommend enabling it in the debian distro.  Is this your
> view Darren?

Sorry, I haven't been able to keep up with the list recently.

Yes, this is my view too, I see no reason to include it for distribution.

I'm going to again defer the real question of the functionality and 
interface etc. to Eric and/or John etc., though it seems good in general.

I just have a couple of follow on comments related to implementation 
details below.

Grégory Lielens wrote:
> std::vector would have been nice, but I tried to respect the original
> coding style, maybe this could be changed but I guess the original
> author should have something to say about it....
>   
I think we want to use std::vector where possible for new code, and 
convert as we go through old code.  Manual malloc/free is just too error 
prone.  (We're forced to do some by interfacing with Python, but it 
should be kept to a minimum).  Maybe this should be added to the coding 
guidelines?
> Only thing I did not change is the allocation of acols/arows even when
> they are not used. It is not difficult to do, but the code will be a
> little more complex and I think that the extra memory used is not
> significant: The memory for those mapping structure is doubled (1 float
> and 1 int vector of size N, instead of a single int vector of size N), 
> but those mapping structures are an order of magnitude smaller than the
> image buffer of size N*N of 4 char anyway...
>   
> If one agree that this is to be saved at the (slight) expense of code
> conciseness/simplicity, I can add this optimisation...
>   
I don't understand the complexity.  Isn't it as simple as moving:

 acols = reinterpret_cast<float*>(PyMem_Malloc(sizeof(float)*cols));

(and the following NULL check) inside of the "if (interpolation == 
Image::BILINEAR)" block?

Heap allocations can be expensive, so it would be great to minimize them.
> Thanks for pointing the error at the left/bottom pixel lines, it is
> corrected now :-)
>   
Great!
> And I set interpolation to NEAREST by default, so that the behavior of
> NonUniformImage remains the same as before if the user do not specify
> otherwise (I forgot to do it in the first patch)...
>   
Sounds reasonable -- though perhaps we want to be consistent instead 
with the standard (uniform) images, which default to the 
"image.interpolation" setting in matplotlibrc.  Of course, 
"image.interpolation" has a bunch of options that nonuniform doesn't 
currently support...  What do others think?

Cheers,
Mike
>
> I include the new patch,
>
> Best regards,
>
> Greg.
>
> On Fri, 2008-08-15 at 15:45 -0400, Michael Droettboom wrote:
>   
>> Thanks for all the work you put into this patch.
>>
>> As you say, it would be nice to have a generic framework for this so 
>> that new types of interpolation could be easily added and to be able to 
>> support arbitrary (non-axis-aligned) quadmeshes as well.  But that's 
>> even more work -- if we keep waiting for everything we want, we'll never 
>> get it... ;)  I agree that Agg probably won't be much help with that. 
>>
>> There are a couple of comments with the patch as it stands -->
>>
>> There seems to be a gap extrapolating over the left and bottom edge (see 
>> attached screenshot from pcolor_nonuniform.py).
>>
>> Memory management looks problematic, some of which I think you inherited 
>> from earlier code.  For example, arows and acols are never freed.  
>> Personally, I think these temporary buffers should be std::vector's so 
>> they'll be free'd automatically when scope is left.  It might also be 
>> nice to move all of the Py_XDECREF's that happen when exceptions are 
>> thrown to either a master try/catch block or an "exit" goto label at the 
>> bottom.  The amount of duplication and care required to ensure 
>> everything will be freed by all of the different exit paths is a little 
>> cumbersome.
>>
>> Also, acols and arows are only used in BILINEAR interpolation, but they 
>> are allocated always.
>>
>> Once these issues are addressed, it would be great to have someone who 
>> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at 
>> this patch for any regressions etc..  Assuming none, I'll be happy to 
>> commit it (but I won't be around for a week or so).
>>
>> Cheers,
>> Mike
>>
>> Grégory Lielens wrote:
>>     
>>> Hi all,
>>>
>>> here is a patch which implement bilinear interpolation on irregular grid
>>> ( i.e. it allows  NonUniformImage
>>> to accept  both 'nearest' and 'bilinear' interpoaltion, instead of only
>>> 'nearest'.)
>>>
>>> It is not perfect, given the current architecture of the image module  I
>>> think there is not simple way
>>> to specify other interpolations (except for 'nearest', all
>>> interpolations are implemented at the AGG level, not in
>>> matplotlib itself).
>>> For the same reason, it is not possible to interpolate before colormap
>>> lookup instead of after (and this 
>>> is usually where the biggest effect is, when dealing with coarse grid).
>>> However, I think it is already quite useful and the best one ca do
>>> without a -very- extensive rewrite
>>> of the matrix map modules....
>>>
>>> BTW, I think it also corrects a small bug in the 'nearest'
>>> interpolation: the last intervals was ignored
>>> in the CVS version, now it is taken into account.
>>>
>>> BOTH nearest and bilinear interpolation do the same for values oustside
>>> the data grid: they just use boundary values (constant extrapolation).
>>> I avoided linear extrapolation for 'bilinear', as extrapolation is
>>> usually dangerous or meaningless (we can go outside the colormap very
>>> fast)...
>>>
>>> I have included a small example showing how both interpolation works....
>>>
>>> Any remarks, could this be added before the next release? ;-)
>>>
>>>
>>> Greg.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 2008-08-11 at 16:50 +0200, Grégory Lielens wrote:
>>>   
>>>       
>>>> On Fri, 2008-08-08 at 16:05 +0200, Grégory Lielens wrote:
>>>>     
>>>>         
>>>>> Hello everybody,
>>>>>
>>>>> I have sent this message to the user group, but thinking of it, it may be more
>>>>> relevant to the development mailing list...so here it is again.
>>>>>
>>>>>
>>>>>
>>>>> We are looking for the best way to plot a waterfall diagram in
>>>>> Matplotlib. The 2 functions which could be used 
>>>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>>>> small script that use both to compare the output:
>>>>>
>>>>> -----------------
>>>>>
>>>>> from pylab import *
>>>>>
>>>>>
>>>>> delta = 0.2
>>>>> x = arange(-3.0, 3.0, delta)
>>>>> y = arange(-2.0, 2.0, delta)
>>>>> X, Y = meshgrid(x, y)
>>>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>>>> # difference of Gaussians
>>>>> Z = 10.0 * (Z2 - Z1)
>>>>> figure(1)
>>>>> im = imshow(Z,extent=(-3,3,-2,2))
>>>>> CS = contour(X, -Y, Z, 6,
>>>>>              colors='k', # negative contours will be dashed by default
>>>>>              )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using imshow')
>>>>> figure(2)
>>>>> im = pcolormesh(X,-Y,Z)
>>>>> CS = contour(X, -Y, Z, 6,
>>>>>              colors='k', # negative contours will be dashed by default
>>>>>              )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using pcolormesh')
>>>>> show()
>>>>>
>>>>> ---------------------
>>>>>
>>>>>
>>>>> The problem is that we need some of the flexibility of pcolormesh (which
>>>>> is able to map the matrix of value on any deformed mesh), while
>>>>> we would like to use the interpolations available in imshow (which
>>>>> explain why the imshow version is much "smoother" than the pcolormesh
>>>>> one).
>>>>>
>>>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>>>> (which can map the grid to any kind of shape), we "only" have to deal
>>>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>>>
>>>>> Is there a drawing function in Matplotlib which would be able to work
>>>>> with such a rectangular non-uniform grid?
>>>>> And if not (and a quick look at the example and the code make me think 
>>>>> that indeed the capability is currently not present),
>>>>> what about an extension of imshow which would work as this:
>>>>>  
>>>>> im = imshow(Z,x_gridpos=x, y_gridpos=y)  #specify the
>>>>> position of the grid's nodes, instead of giving the extend and assuming
>>>>> uniform spacing.
>>>>>
>>>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>>>> current behavior, averaging the color of the grids node to get a uniform
>>>>> cell color, 
>>>>> is quite rough except for a large number of cells...And even then, it
>>>>> soon shows when you zoom in...
>>>>>
>>>>> The best would be to allow the same interpolations as in imshow (or a
>>>>> subset of it), and also allows to use interpolation before colormap
>>>>> lookup (or after), 
>>>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>>>> specifying Gouraud (interpolation after color
>>>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>>>> Phong is usually much better but also more CPU intensive. Phong is
>>>>> especially when using discrete colormap, producing  banded colors
>>>>> equivalent to countour lines, while Gouraud does not work in those
>>>>> cases.
>>>>>
>>>>> Of course, the performance will be impacted by some of those
>>>>> interpolation options, which would degrade performance in animations for
>>>>> example.... but I think that having the different options available
>>>>> would be very useful, it allows to have the highest map quality, or have
>>>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>>>> map, animation or not, ...).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Greg.
>>>>>       
>>>>>           
>>>> I have found a method which implement the proposed extension to imshow:
>>>> NonUniformImage...
>>>>
>>>> However, this image instance support only nearest neighbor
>>>> interpolation. Trying to set the interpolation (using the
>>>> set_interpolation method)
>>>> to something akin imshow throw a "NotImplementedError: Only nearest
>>>> neighbor supported" exception....
>>>>
>>>> So basically I am still stuck, it seems that currently there is no way
>>>> in matplotlib to plot interpolated
>>>> colormap on irregular rectangular grid, and even less on arbitrarily
>>>> mapped grid...
>>>>
>>>> Is there any plans to add support for more interpolation in
>>>> NonUniformImage in the future? Or maybe there is another 
>>>> drawing function that I did not find yet, with this ability?
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>>         
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Matplotlib-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michael Droettboom wrote:
> I think we want to use std::vector where possible for new code, and 
> convert as we go through old code.  Manual malloc/free is just too error 
> prone.  (We're forced to do some by interfacing with Python, but it 
> should be kept to a minimum).

My understanding is that this is exactly the issue -- with std::vector 
and friends you can't both get the data pointer and create a vector with 
a data pointer, which would be what you'd want to do to interface 
efficiently with numpy arrays.

I've been looking for a way to address this problem, but my C++ is 
pretty week. maybe smart pointers and/or other boost classes could help.


I do have an idea, though. The goal is a set of classes for working with 
data that are convenient to use with pure C++, but also play with numpy 
arrays.

I imagine a simple vector/array set of classes build on top of a simple 
object: a reference counted data block. This data block object would 
simply hold a pointer to a block of data, maybe know how large it is, 
maybe know what type the data is, and hold a reference count. when that 
count drops to zero it deletes itself.

The vector/array classes built on top of it would use the data block to 
store their data, and increment/decrement the reference count as need 
be. This would allow them to share data, have one array that is a subset 
of another, using the same data block, etc.

It should be easy to build numpy arrays from a system like this.

Ideally (but I'm not sure how), the data blocks reference counting 
system could be integrated with Python's, so that when a numpy array was 
constructed from one, its reference count would be managed by python as 
well as your C++ code.

But maybe that's all just too much work...


-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@...

It's funny you should mention this.

One of the things I realized after SciPy this year is that there is a 
lot of interfacing of Numpy with C++ (as opposed to C) that could really 
benefit from a standard C++ wrapper around Numpy.  A bunch of different 
projects all had home-grown, semi-complete solutions. 

A standard wrapper would be good for all of the reasons you mention 
(basically resource management), but also to have a cleaner syntax for 
accessing the elements that would abstract away striding, ordering etc.  
It may actually be possible to build it around boost::multiarray (the 
closest thing the upcoming C++-0x standard has to a n-dimensional array).

Can't imagine I'll find time for it anytime soon, but it would be fun to 
work on...

Cheers,
Mike

Christopher Barker wrote:
> Michael Droettboom wrote:
>   
>> I think we want to use std::vector where possible for new code, and 
>> convert as we go through old code.  Manual malloc/free is just too error 
>> prone.  (We're forced to do some by interfacing with Python, but it 
>> should be kept to a minimum).
>>     
>
> My understanding is that this is exactly the issue -- with std::vector 
> and friends you can't both get the data pointer and create a vector with 
> a data pointer, which would be what you'd want to do to interface 
> efficiently with numpy arrays.
>
> I've been looking for a way to address this problem, but my C++ is 
> pretty week. maybe smart pointers and/or other boost classes could help.
>
>
> I do have an idea, though. The goal is a set of classes for working with 
> data that are convenient to use with pure C++, but also play with numpy 
> arrays.
>
> I imagine a simple vector/array set of classes build on top of a simple 
> object: a reference counted data block. This data block object would 
> simply hold a pointer to a block of data, maybe know how large it is, 
> maybe know what type the data is, and hold a reference count. when that 
> count drops to zero it deletes itself.
>
> The vector/array classes built on top of it would use the data block to 
> store their data, and increment/decrement the reference count as need 
> be. This would allow them to share data, have one array that is a subset 
> of another, using the same data block, etc.
>
> It should be easy to build numpy arrays from a system like this.
>
> Ideally (but I'm not sure how), the data blocks reference counting 
> system could be integrated with Python's, so that when a numpy array was 
> constructed from one, its reference count would be managed by python as 
> well as your C++ code.
>
> But maybe that's all just too much work...
>
>
> -Chris
>
>
>
>

Michael Droettboom wrote:
> It's funny you should mention this.
> 
> One of the things I realized after SciPy this year is that there is a 
> lot of interfacing of Numpy with C++ (as opposed to C) that could really 
> benefit from a standard C++ wrapper around Numpy.

Absolutely. Though now that you mention it, isn't there one in 
boost::python? or at least one that worked with Numeric.

> projects all had home-grown, semi-complete solutions.
> A standard wrapper would be good for all of the reasons you mention 
> (basically resource management), but also to have a cleaner syntax for 
> accessing the elements that would abstract away striding, ordering etc.  

I think I'm thinking of something slightly different. What I'd like is 
set of array classes that can interface well with numpy, and also be 
quite usable all by themselves in C++ code that has nothing to do with 
python.

I don't really envision a C++ ndarray -- I generally think of C++ as 
static enough that I'm happy to define that I need, for example, a NX2 
array of doubles. I don't see using a generic array like ndarray that 
could hold any type, shape, etc. But maybe others do -- it would be 
pretty cool.

I guess to some extent the question is whether you are writing 
extensions for python, or a C++ that you want to use from C++, and also 
wrap for python.

But maybe both need can be met by the same C++ classes (templates?)

> It may actually be possible to build it around boost::multiarray

Do you know off hand if multiarray can be constructed from an existing 
pointer to a data block?

> Can't imagine I'll find time for it anytime soon,

too bad, but if you do -- I'd love to hear about it!

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@...

Grégory Lielens wrote:
> Thanks a lot for reviewing my patch!
> I have corrected most of the problems (I think ;-) )
> I indeed introduced memory leak, I think it is fixed now, I have also
> reorganized the code to avoid duplication of the cleanup code. I used an
> helper function instead of the goto, because this cleanup is needed for
> normal exit too, so the helper function can come in handy for that.
> std::vector would have been nice, but I tried to respect the original
> coding style, maybe this could be changed but I guess the original
> author should have something to say about it....
> Only thing I did not change is the allocation of acols/arows even when
> they are not used. It is not difficult to do, but the code will be a
> little more complex and I think that the extra memory used is not
> significant: The memory for those mapping structure is doubled (1 float
> and 1 int vector of size N, instead of a single int vector of size N), 
> but those mapping structures are an order of magnitude smaller than the
> image buffer of size N*N of 4 char anyway...
> 
> If one agree that this is to be saved at the (slight) expense of code
> conciseness/simplicity, I can add this optimisation...
> 
> Thanks for pointing the error at the left/bottom pixel lines, it is
> corrected now :-)
> 
> And I set interpolation to NEAREST by default, so that the behavior of
> NonUniformImage remains the same as before if the user do not specify
> otherwise (I forgot to do it in the first patch)...
> 
> 
> I include the new patch,

Gregory,

I am sorry to be late in joining in, but I have a basic question about
what you are trying to do.  It arises from the fact that there are two
styles of image-like plots:

1) In the original Image and NonuniformImage, the colors and locations
of "pixel" (or patch or subregion or whatever) centers are specified.

2) In pcolor, quadmesh, and pcolorimage, the patch *boundaries* are
specified.  pcolorimage is my modification of the original 
NonuniformImage code to work with boundaries.  I left the original in 
place, since presumably it meets some people's needs, even though it 
does not meet mine.

When the grid is uniform, it doesn't really make any difference; but
when the grid is nonuniform, then I don't think style #1 makes much
sense for pcolor-style plotting, in which there is no interpolation; 
there is no unique way to specify the boundaries, so it is up to
the plotting routine, and that is bad.

For an image, it is clear how the interpolation should work: the color 
values are given at the centers of the "pixels" (meaning on the original 
image grid, not on the screen or other output device grid), which are 
uniformly spaced, and interpolated between.

Exactly what is your patch supposed to do for the nonuniform case? Would 
you describe and demonstrate it, please, for a simple case with a 2x2 
array of colors and 3x3 arrays for boundary x and y?

Thanks.

Eric


> 
> Best regards,
> 
> Greg.
> 
> On Fri, 2008-08-15 at 15:45 -0400, Michael Droettboom wrote:
>> Thanks for all the work you put into this patch.
>>
>> As you say, it would be nice to have a generic framework for this so 
>> that new types of interpolation could be easily added and to be able to 
>> support arbitrary (non-axis-aligned) quadmeshes as well.  But that's 
>> even more work -- if we keep waiting for everything we want, we'll never 
>> get it... ;)  I agree that Agg probably won't be much help with that. 
>>
>> There are a couple of comments with the patch as it stands -->
>>
>> There seems to be a gap extrapolating over the left and bottom edge (see 
>> attached screenshot from pcolor_nonuniform.py).
>>
>> Memory management looks problematic, some of which I think you inherited 
>> from earlier code.  For example, arows and acols are never freed.  
>> Personally, I think these temporary buffers should be std::vector's so 
>> they'll be free'd automatically when scope is left.  It might also be 
>> nice to move all of the Py_XDECREF's that happen when exceptions are 
>> thrown to either a master try/catch block or an "exit" goto label at the 
>> bottom.  The amount of duplication and care required to ensure 
>> everything will be freed by all of the different exit paths is a little 
>> cumbersome.
>>
>> Also, acols and arows are only used in BILINEAR interpolation, but they 
>> are allocated always.
>>
>> Once these issues are addressed, it would be great to have someone who 
>> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at 
>> this patch for any regressions etc..  Assuming none, I'll be happy to 
>> commit it (but I won't be around for a week or so).
>>
>> Cheers,
>> Mike
>>
>> Grégory Lielens wrote:
>>> Hi all,
>>>
>>> here is a patch which implement bilinear interpolation on irregular grid
>>> ( i.e. it allows  NonUniformImage
>>> to accept  both 'nearest' and 'bilinear' interpoaltion, instead of only
>>> 'nearest'.)
>>>
>>> It is not perfect, given the current architecture of the image module  I
>>> think there is not simple way
>>> to specify other interpolations (except for 'nearest', all
>>> interpolations are implemented at the AGG level, not in
>>> matplotlib itself).
>>> For the same reason, it is not possible to interpolate before colormap
>>> lookup instead of after (and this 
>>> is usually where the biggest effect is, when dealing with coarse grid).
>>> However, I think it is already quite useful and the best one ca do
>>> without a -very- extensive rewrite
>>> of the matrix map modules....
>>>
>>> BTW, I think it also corrects a small bug in the 'nearest'
>>> interpolation: the last intervals was ignored
>>> in the CVS version, now it is taken into account.
>>>
>>> BOTH nearest and bilinear interpolation do the same for values oustside
>>> the data grid: they just use boundary values (constant extrapolation).
>>> I avoided linear extrapolation for 'bilinear', as extrapolation is
>>> usually dangerous or meaningless (we can go outside the colormap very
>>> fast)...
>>>
>>> I have included a small example showing how both interpolation works....
>>>
>>> Any remarks, could this be added before the next release? ;-)
>>>
>>>
>>> Greg.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 2008-08-11 at 16:50 +0200, Grégory Lielens wrote:
>>>   
>>>> On Fri, 2008-08-08 at 16:05 +0200, Grégory Lielens wrote:
>>>>     
>>>>> Hello everybody,
>>>>>
>>>>> I have sent this message to the user group, but thinking of it, it may be more
>>>>> relevant to the development mailing list...so here it is again.
>>>>>
>>>>>
>>>>>
>>>>> We are looking for the best way to plot a waterfall diagram in
>>>>> Matplotlib. The 2 functions which could be used 
>>>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>>>> small script that use both to compare the output:
>>>>>
>>>>> -----------------
>>>>>
>>>>> from pylab import *
>>>>>
>>>>>
>>>>> delta = 0.2
>>>>> x = arange(-3.0, 3.0, delta)
>>>>> y = arange(-2.0, 2.0, delta)
>>>>> X, Y = meshgrid(x, y)
>>>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>>>> # difference of Gaussians
>>>>> Z = 10.0 * (Z2 - Z1)
>>>>> figure(1)
>>>>> im = imshow(Z,extent=(-3,3,-2,2))
>>>>> CS = contour(X, -Y, Z, 6,
>>>>>              colors='k', # negative contours will be dashed by default
>>>>>              )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using imshow')
>>>>> figure(2)
>>>>> im = pcolormesh(X,-Y,Z)
>>>>> CS = contour(X, -Y, Z, 6,
>>>>>              colors='k', # negative contours will be dashed by default
>>>>>              )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using pcolormesh')
>>>>> show()
>>>>>
>>>>> ---------------------
>>>>>
>>>>>
>>>>> The problem is that we need some of the flexibility of pcolormesh (which
>>>>> is able to map the matrix of value on any deformed mesh), while
>>>>> we would like to use the interpolations available in imshow (which
>>>>> explain why the imshow version is much "smoother" than the pcolormesh
>>>>> one).
>>>>>
>>>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>>>> (which can map the grid to any kind of shape), we "only" have to deal
>>>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>>>
>>>>> Is there a drawing function in Matplotlib which would be able to work
>>>>> with such a rectangular non-uniform grid?
>>>>> And if not (and a quick look at the example and the code make me think 
>>>>> that indeed the capability is currently not present),
>>>>> what about an extension of imshow which would work as this:
>>>>>  
>>>>> im = imshow(Z,x_gridpos=x, y_gridpos=y)  #specify the
>>>>> position of the grid's nodes, instead of giving the extend and assuming
>>>>> uniform spacing.
>>>>>
>>>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>>>> current behavior, averaging the color of the grids node to get a uniform
>>>>> cell color, 
>>>>> is quite rough except for a large number of cells...And even then, it
>>>>> soon shows when you zoom in...
>>>>>
>>>>> The best would be to allow the same interpolations as in imshow (or a
>>>>> subset of it), and also allows to use interpolation before colormap
>>>>> lookup (or after), 
>>>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>>>> specifying Gouraud (interpolation after color
>>>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>>>> Phong is usually much better but also more CPU intensive. Phong is
>>>>> especially when using discrete colormap, producing  banded colors
>>>>> equivalent to countour lines, while Gouraud does not work in those
>>>>> cases.
>>>>>
>>>>> Of course, the performance will be impacted by some of those
>>>>> interpolation options, which would degrade performance in animations for
>>>>> example.... but I think that having the different options available
>>>>> would be very useful, it allows to have the highest map quality, or have
>>>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>>>> map, animation or not, ...).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Greg.
>>>>>       
>>>> I have found a method which implement the proposed extension to imshow:
>>>> NonUniformImage...
>>>>
>>>> However, this image instance support only nearest neighbor
>>>> interpolation. Trying to set the interpolation (using the
>>>> set_interpolation method)
>>>> to something akin imshow throw a "NotImplementedError: Only nearest
>>>> neighbor supported" exception....
>>>>
>>>> So basically I am still stuck, it seems that currently there is no way
>>>> in matplotlib to plot interpolated
>>>> colormap on irregular rectangular grid, and even less on arbitrarily
>>>> mapped grid...
>>>>
>>>> Is there any plans to add support for more interpolation in
>>>> NonUniformImage in the future? Or maybe there is another 
>>>> drawing function that I did not find yet, with this ability?
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Matplotlib-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>     
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Matplotlib-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________ Matplotlib-devel mailing list Matplotlib-devel@... https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel@...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Christopher Barker wrote:
> Michael Droettboom wrote:
>   
>> It's funny you should mention this.
>>
>> One of the things I realized after SciPy this year is that there is a 
>> lot of interfacing of Numpy with C++ (as opposed to C) that could really 
>> benefit from a standard C++ wrapper around Numpy.
>>     
>
> Absolutely. Though now that you mention it, isn't there one in 
> boost::python? or at least one that worked with Numeric.
>
>   
Yes, but AFAICT it forces the use of boost::python, at least to convert 
the object.  It would be nicer (in some cases) to have something that 
could be wrapper-system-agnostic.

I spent a little time working on this today, and I was actually 
pleasantly surprised by how far I got.

I have something that exposes Numpy arrays as boost::multi_arrays on the 
C++ side, and also allows for creating new Numpy-backed arrays from 
C++.  The memory is still managed with Python reference counting, and 
the references are automatically freed from the constructor.  It 
requires boost and Numpy, of course, but not boost::python.  A 
standalone C++ class for this would perhaps have been nicer, but that's 
not an afternoon's 200-line hack like this is.

I wouldn't really consider this a "wrapper" around Numpy, since it 
doesn't actually use Numpy for any of the indexing or computation.  It 
simply converts the Numpy description (pointer + strides, etc.) to a 
boost::multi_array, without copying the underlying data.
>   
>> projects all had home-grown, semi-complete solutions.
>> A standard wrapper would be good for all of the reasons you mention 
>> (basically resource management), but also to have a cleaner syntax for 
>> accessing the elements that would abstract away striding, ordering etc.  
>>     
>
> I think I'm thinking of something slightly different. What I'd like is 
> set of array classes that can interface well with numpy, and also be 
> quite usable all by themselves in C++ code that has nothing to do with 
> python.
>   
This should fit that bill.  Just templatize all your algorithms to 
accept anything with the "boost::multi_array" interface.  When building 
extensions for Numpy, pass in a wrapped numpy array -- when not, use a 
pure boost::multi_array.  Of course, the pure boost::multi_array doesn't 
have reference-counted memory management.  Use of boost::shared_ptr may 
help, but it's not "built-in" -- views of boost::multi_arrays don't 
automatically keep their parent data alive.  It's the usual C++ "you're 
on your own" approach.
> I don't really envision a C++ ndarray -- I generally think of C++ as 
> static enough that I'm happy to define that I need, for example, a NX2 
> array of doubles. I don't see using a generic array like ndarray that 
> could hold any type, shape, etc. But maybe others do -- it would be 
> pretty cool.
>   
My approach (being based on boost::multi_array) has a fixed type and 
number of dimensions at compile time.  If you try to pass in a Numpy 
array that can't be converted to that, an exception is thrown.  
Personally, I feel that's good enough for most domain-specific 
algorithms.  Theoretically, one could write algorithms that are 
templatized on the data type and then dispatch at run time to different 
instantiations of that template -- but my code doesn't do anything like 
that yet.
> I guess to some extent the question is whether you are writing 
> extensions for python, or a C++ that you want to use from C++, and also 
> wrap for python.
>   
I'm just doing this out of curiosity rather than to meet a need.  We 
don't have much C++ code at the Institute.  I could see this being very 
handy in matplotlib, however, but the dependency on Boost is probably a 
showstopper... :(
> But maybe both need can be met by the same C++ classes (templates?)
>   
I think so.  See above.
>   
>> It may actually be possible to build it around boost::multiarray
>>     
>
> Do you know off hand if multiarray can be constructed from an existing 
> pointer to a data block?
>   
Yep.  multi_array_ref seems custom-built for that purpose.
>   
>> Can't imagine I'll find time for it anytime soon,
>>     
I went ahead and created a Google Code project for this here:

   http://code.google.com/p/numpy-boost/

Just because it's up there doesn't mean it's anything more than an 
experimental hack.  It needs documentation and more complete unit 
tests... etc...

Mike

On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee.j.joon@...> wrote:

> First of all, I borrowed this idea from the PyX which is in GPL.
> Although there is little of copying, other than the basic idea, I'm
> not 100% sure if this could be BSD-compatible.

I think it is fine to borrow the idea; what we need to do is a clean
room implementation with no copying.  You can best answer that, so if
you tell us your patch is cleanly implemented, we can accept it.

JDH

On Thu, Aug 28, 2008 at 4:18 PM, John Hunter <jdh2358@...> wrote:
> On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee.j.joon@...> wrote:
>
>> First of all, I borrowed this idea from the PyX which is in GPL.
>> Although there is little of copying, other than the basic idea, I'm
>> not 100% sure if this could be BSD-compatible.
>
> I think it is fine to borrow the idea; what we need to do is a clean
> room implementation with no copying.  You can best answer that, so if
> you tell us your patch is cleanly implemented, we can accept it.
>
> JDH
>

Thanks for the response.

Well, the only part I borrowed from PyX is TeX related commands they
use (there is not much of implementation as far as TeX-related code is
concerned). From their code, I learned the meaning and usage of the
following TeX commands

\newbox
\setbox
\immediate\write16

And I used the same TeX commands in my code.
But I personally think this is not a (code) copy.

Other than this, the code is clean.
Regards,

-JJ

Sphinx contains one way to do this in its new "pngmath" extension.  It 
uses the LaTeX package "preview" which does all of this magic 
internally.  And I believe it's a little more general.  If I recall, the 
approach you're taking won't work with some LaTeX constructs such as:

  \begin{align}
     x & = 2
     y & = 2
  \end{align}

Plus, Sphinx is BSD-licensed, so it should be fine to copy-and-paste 
whatever code is necessary.

Of course, latex-preview is required to be installed, but I think it's a 
pretty common package.

See here:

  http://svn.python.org/projects/doctools/trunk/sphinx/ext/pngmath.py

Cheers,
Mike

Jae-Joon Lee wrote:
> On Thu, Aug 28, 2008 at 4:18 PM, John Hunter <jdh2358@...> wrote:
>   
>> On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee.j.joon@...> wrote:
>>
>>     
>>> First of all, I borrowed this idea from the PyX which is in GPL.
>>> Although there is little of copying, other than the basic idea, I'm
>>> not 100% sure if this could be BSD-compatible.
>>>       
>> I think it is fine to borrow the idea; what we need to do is a clean
>> room implementation with no copying.  You can best answer that, so if
>> you tell us your patch is cleanly implemented, we can accept it.
>>
>> JDH
>>
>>     
>
> Thanks for the response.
>
> Well, the only part I borrowed from PyX is TeX related commands they
> use (there is not much of implementation as far as TeX-related code is
> concerned). From their code, I learned the meaning and usage of the
> following TeX commands
>
> \newbox
> \setbox
> \immediate\write16
>
> And I used the same TeX commands in my code.
> But I personally think this is not a (code) copy.
>
> Other than this, the code is clean.
> Regards,
>
> -JJ
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Thanks,

I quickly went through the code of the pngmath.py, and it seems that
the depth(descent) of the dvi file is reported by "dvipng" (but the
preview package must be used in the tex file for this to work
correctly). Therefore, with this method, we need to run dvipng even if
we use ps of pdf backend. Although this seems fine to me, I'll see if
I can extract the depth of the text without running the dvipng.

Regards,

-JJ




On Fri, Aug 29, 2008 at 7:59 AM, Michael Droettboom <mdroe@...> wrote:
> Sphinx contains one way to do this in its new "pngmath" extension.  It uses
> the LaTeX package "preview" which does all of this magic internally.  And I
> believe it's a little more general.  If I recall, the approach you're taking
> won't work with some LaTeX constructs such as:
>
>  \begin{align}
>    x & = 2
>    y & = 2
>  \end{align}
>
> Plus, Sphinx is BSD-licensed, so it should be fine to copy-and-paste
> whatever code is necessary.
>
> Of course, latex-preview is required to be installed, but I think it's a
> pretty common package.
>
> See here:
>
>  http://svn.python.org/projects/doctools/trunk/sphinx/ext/pngmath.py
>
> Cheers,
> Mike
>
> Jae-Joon Lee wrote:
>>
>> On Thu, Aug 28, 2008 at 4:18 PM, John Hunter <jdh2358@...> wrote:
>>
>>>
>>> On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee.j.joon@...>
>>> wrote:
>>>
>>>
>>>>
>>>> First of all, I borrowed this idea from the PyX which is in GPL.
>>>> Although there is little of copying, other than the basic idea, I'm
>>>> not 100% sure if this could be BSD-compatible.
>>>>
>>>
>>> I think it is fine to borrow the idea; what we need to do is a clean
>>> room implementation with no copying.  You can best answer that, so if
>>> you tell us your patch is cleanly implemented, we can accept it.
>>>
>>> JDH
>>>
>>>
>>
>> Thanks for the response.
>>
>> Well, the only part I borrowed from PyX is TeX related commands they
>> use (there is not much of implementation as far as TeX-related code is
>> concerned). From their code, I learned the meaning and usage of the
>> following TeX commands
>>
>> \newbox
>> \setbox
>> \immediate\write16
>>
>> And I used the same TeX commands in my code.
>> But I personally think this is not a (code) copy.
>>
>> Other than this, the code is clean.
>> Regards,
>>
>> -JJ
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win great
>> prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the
>> world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel@...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>
>

Manuel Metz wrote:
> Hi all,
>   I ran into a problem where I wanted to plot a step-plot with dashed 
> lines instead of solid lines which can be important for print media. 
> This isn't possible with the current matplotlib version, so I added 
> support for this. The patch is attached, but I didn't commit it yet 
> since I wanted to ask for feedback first. It basically adds support to do
> 
> pylab.plot(x, y, 'steps--')
> 
> to create a step plot with dashed lines.

Hm, okay - I just noticed that the patch doesn't work with legend. This 
needs to be fixed ...

mm

Michael Droettboom wrote:
> On RHEL4, I get the following message:
> 
>   This OpenGL does not support framebuffer objects
> 
> I understand that I'm probably just suffering from a relatively old 
> Mesa/OpenGL stack here.  And obviously, just because some older systems 
> won't support this is not a reason to not include it as an optional 
> backend.
> 
> Can you possibly send a screen shot to the list for those of us who 
> can't see it?  Is the quadmesh interpolated?  That would be a huge 
> improvement over anything we can do currently.
> 
> As for developing a full OpenGL backend, note that the process of 
> writing a backend has been greatly simplified as of 0.98.x (there's only 
> about four rendering methods to implement now, with three more optional 
> ones to re-implement for performance reasons).   You can start by 
> looking at backend_bases.py.  I'm happy to help with any questions you 
> have as you go along, but as I don't have a working pyglet/OpenGL, I 
> can't be of much help for testing.
> 

I'll let you know if I need any help.  0.98 definitely seems simpler and 
I think Pyglet will handle a lot of the context stuff to make things 
simpler as well.

Your error on RHEL4 indicates more that it doesn't support our 
particular approach, which uses offscreen rendering to render to an 
image which is then blitted in.  It's an expedient approach to get a 
drop in quadmesh replacement.  A full pyglet backend would not suffer 
from this limitation.

I don't have this hooked up fully within matplotlib yet, so right now it 
just generates a test image with a 2000x2000 mesh of quadrilaterals 
which is displayed using imshow(). (simple example attached.)  I have 
not looked at interpolation or even turned on antialiasing (should just 
be one or two function calls).  I can tell you that most of the time is 
spent generating the list of vertices in a form suitable for OpenGL 
QUAD_LISTS.  The actual rendering and handing off to the graphics card 
is *really* fast.

I have a couple other things on my plate ATM, but hopefully next week 
some time I will get back to this, as I really, really, really want/need 
to pursue the opengl stuff further.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Sorry about that, and thanks for your investigation.  It's easy enough 
to change the default. 

I think (and it was a while ago, so I may be misremembering) I probably 
made that change myself in error.  (There used to be two completely 
separate code paths for doing step plots, that I merged into one). 

Any objections?  Is it possible that change was made deliberately (to 
conform with matlab or gnuplot etc.)?

Cheers,
Mike

Neil Crighton wrote:
> Now I see there are more options in 0.98 - 'steps-pre', 'steps-post',
> 'steps-mid'. The default should be steps-post for backwards
> compatibility.  In 0.98.3 the default is steps-pre.  And sorry for the
> testy tone of the previous email :)
>
> Neil
>
> 2008/8/28 Neil Crighton <neilcrighton@...>:
>   
>> linestyle='steps' has changed behaviour between 0.91.2 and 0.98.3. The
>> 'step' between two points used to move horizontally and then
>> vertically from the left point neighbouring right point, now it moves
>> vertically then horizontally.
>>
>> Was this change intentional? I hope not, because I've just spent the
>> past hour working out it was the reason for my plotting routines not
>> working properly :-/
>>
>> Neil
>>
>>     
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>   

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA