matplotlib-devel — matplotlib developers

Re: [matplotlib-devel] colormaps in pcolor

[John H. <jdh...@ac...>]

>>>>> "Jon" == Jon Peirce <jw...@ps...> writes:

    Jon> John, loving matplotlib - thx.

    Jon> Was using pcolor today but needed a gray colormap rather than
    Jon> jet. Made my own version (see attached) using a class
    Jon> Colormap with attribute color (which can be set to
    Jon> 'jet'). Seemed a bit more adaptable and more like matlab. I
    Jon> linked ColormapJet back to this class so that other people's
    Jon> code wont break (hopefully ;) ). Probably worth allowing
    Jon> users to supply there own as an array too, but I didn't have
    Jon> time to do that today.

I've been wanting to include multiple colormaps, so this is a start in
the right direction.  A word of warning: some versions of Numeric are
broken with 'from __future__ import division'

  from __future__ import division
  ...snip...
  self.red = arange(self.N+1)/self.N

It's safer to do

  self.red = 1/self.N*arange(self.N+1)   

or

  self.red = divide(arange(self.N+1), self.N)

    Jon> On a different topic slightly, I wonder if it would be worth
    Jon> having a plot type based on image widgets. For large arrays
    Jon> pcolor is still very slow under gtk. Maybe either using image
    Jon> widgets for pcolor itself or having a different plot type
    Jon> (like matlabs 'image' or 'imagesc').

I don't think a specialized plot type or image widget is the way to
go, since it wouldn't port across backends very well.  The plot
commands are fairly high level and are used to construct the lower
level graphics primitives.  

I think it better perhaps to introduce some new graphics primitives
(on the same level as line, patch, text) that handle 2D arrays and
colormaps efficiently.  The cases I would like to be able to handle
are

 1) 2D image data: eg, RGB or plain old image files such as PNG

 2) 2D scalar data: points with colormap

 3) 2D scalar data: rectangle patch + colormap + optional gradient
    interpolation

In the existing design of matplotlib, the backends don't handle
transformations, scaling, etc... Consistent with this, we could
provide frontend code to take existing image data (construed broadly
to cover all the cases above), scale it to the axes display
coordinates, do the interpolation as necessary, and construct an MxN
array (axes window is MxN pixels) of RGBA data (RGBA is in normalized
0,1 scale).  In other words, we agree on a single image data structure
all implemented in extension code, and then make the backends
implement a method to handle that structures in the same way they have
to now handle a rectangular patch.

Eg, we would need only one additional backend method

  renderer.draw_rgba(imageData)

and it only has to do a little extra work to render it.  In GTK all
that would be required is to scale the RGB by 65536 and use the GDK
draw rgb method.

We would definitely want to use some decent image library to do the
frontend scaling, interpolation, file loading, etc.  libart and agg
have been discussed on and off list lately as candidates.  VTK is also
a possibility.  Although it is primarily 3D library, it also has
support for all the 2D stuff you can imagine and everything else too.
VTK is a big and hairy package, but it runs everywhere, does
everything well, and opens the door for going 3D.

I've had some discouraging experiences with libart over the last few
days.  In trying to implement clipping for the paint backend, I've
come across some known libart numerical instabilities, and in my
questions to the libart list I've been steered to agg by other
frustrated libart users.  

JDH

[matplotlib-devel] colormaps in pcolor

[Jon P. <jw...@ps...>]

John,

loving matplotlib - thx.

Was using pcolor today but needed a gray colormap rather than jet. Made 
my own version (see attached) using a class Colormap with attribute 
color (which can be set to 'jet'). Seemed a bit more adaptable and more 
like matlab. I linked ColormapJet back to this class so that other 
people's code wont break (hopefully ;) ). Probably worth allowing users 
to supply there own as an array too, but I didn't have time to do that 
today.

On a different topic slightly, I wonder if it would be worth having a 
plot type based on image widgets. For large arrays pcolor is still very 
slow under gtk. Maybe either using image widgets for pcolor itself or 
having a different plot type (like matlabs 'image' or 'imagesc').

all the best
Jon

-- 
Jon Peirce
Nottingham University
+44 (0)115 8467176 (tel)
+44 (0)115 9515324 (fax)

http://www.psychology.nottingham.ac.uk/staff/jwp/

Re: [matplotlib-devel] Tables for matplotlib

[John G. <jn...@eu...>]

>     John> Hi John, I've moved on a fair bit with this, still quite a
>     John> way to go.
> 
> Nice work.  table_demo.py looks fairly wild -- prone to induce
> seizures -- but table_demo2.py is nice.  


I do a lot of this work on the train in and out of work -- have to be
careful running table_demo.py in case anyone is watching the screen.


>     John> axes.patch is a patch to axes.py that adds table support.
> 
> Could you provide context diffs, or just the whole file.  I can diff
> it myself or just copy and paste the relevant code.


I've attached my version of axes.py.

Thanks for the axes tip -- that is much better.

One other glitch that is showing up is when I resize the window (I'm
using the GTK backend) for table_demo2.py the table with the row labels
goes out of alignment with the other table -- any ideas what is causing
that?

I like the coloured patches with text idea -- i'll do some experiments
and see how it looks.  One thought I had is to turn the colours into
light pastel shades so that dark text still stands out.

John

[matplotlib-devel] paint backend added to CVS

[John H. <jdh...@ac...>]

David Moore has stealthily written a paint backend for matplotlib.
paint is a libart wrapper

  http://www.levien.com/libart

The official libart documentation is sparse, but I found these
documents very helpful

  http://www.gnome.org/~mathieu/libart
  http://developer.gnome.org/doc/books/WGA/graphics-libart.html

In a nutshell, libart is a svg oriented, high performance, 2D graphics
engine that supports lots of nifty features.  The paint backend
exposes some of them, and David has been extending it to expose more.
Look for pypaint.sourceforge.net in the near future.  The official
release of paint at http://www.object-craft.com.au/projects/paint
won't work with backend_paint.  David can you announce here when you
release it?  If anyone wants to test it out before then, perhaps you
should contact David at da...@sj....

The latest version of backend_paint.py can be obtained from CVS.  Here
is a status list of known issues

 * Text and linewidths don't scale with DPI - FIXED
 
 * No dots or dashed lines - FIXED

 * Patch edge colors not displayed - FIXED

 * circle center locations off - FIXED

 * font manager support not included - OPEN.  I moved the gd truetype
   font manager stuff to matplotlib.backends.ttf_font_manager so it
   could be adapted for use with all true type backends, but haven't
   incorporated this into the paint backend.  The default font is
   Vera.

 * no clipping - OPEN.  The libart function art_svp_intersect is used
   for intersecting arbitrary sorted vector paths (clipping), but this
   needs to be exposed in the paint wrapper.

 * draw_lines moved into the paint extension module - OPEN.
   draw_lines is probably the most common operation performed in
   matplotlib, sometimes with very long arrays, and it would be a big
   gain and an easy port to do this at the C level.  the function
   should probably just construct and return a path that can be dashed
   or stroked as needed at the python level.

In the longer term, I would like to be able to support some path
operations at the renderer level.  Now that matplotlib has two vector
oriented backends (paint and postscript) and I would like to add PDF
and SVG, it would be nice to support some path operations in the front
end, to take advantage of their sophisticated drawing capabilities.
The question is: how to bring the other backends along?

JDH

Re: [matplotlib-devel] Tables for matplotlib

[John H. <jdh...@ac...>]

>>>>> "John" == John Gill <jn...@eu...> writes:

    John> Hi John, I've moved on a fair bit with this, still quite a
    John> way to go.

Nice work.  table_demo.py looks fairly wild -- prone to induce
seizures -- but table_demo2.py is nice.  

    John> axes.patch is a patch to axes.py that adds table support.

Could you provide context diffs, or just the whole file.  I can diff
it myself or just copy and paste the relevant code.

    John> Currently you have to specify the column widths for the
    John> table.  Really two cases should be supported: allow the user
    John> to specify the widths (this is needed so you can line up
    John> tables with bar plots) and allow the Table to auto-adjust
    John> the widths to make everything just big enough for whatever
    John> text is present (this is handy for legend type stuff).  I've
    John> done nothing for auto-guessing of widths as yet.

It can be a pain.  I think you're taking the right approach - do the
easy case first.  As you probably noticed, the legend does some
autolayout using bboxes and you can emulate this code if you want to
go this route.

    John> I've made the grid within the tables an option + I think
    John> with a little more work legend.py could just use table.py to
    John> do what it has to do.

    John> There is an ugly hack, rowOffset that I use in
    John> table_demo2.py to get the row labels to align with the
    John> appropriate data.  My idea was that it might be simpler to
    John> have a basic table object and then build up more complicated
    John> stuff such as including row-labels by creating multiple
    John> tables.

    John> Currently I can't decide whether i wouldn't be better trying
    John> to support the row/column labels all in the one object -- I
    John> suspect this is the way to go since it is easier to get
    John> things to line up this way rather than trying to align lots
    John> of separate tables (eg if i start supporting different fonts
    John> for different tables then we'll really be in trouble).

    John> Now as I said there is still lots to do here.

    John> There are all sorts of minor and not so minor alignment
    John> issues to address -- a lot of these I think I can solve by
    John> paying a bit more attention to the legend.py code.

    John> The most significant problem at the moment is that the
    John> tables are getting clipped -- I'm not sure how I control the
    John> size of the border around the axes.

You need to manually set the axes.  The figure size is fixed and the
axes take up a fixed proportion of the figure.  In table_demo.py, put
the following at the top of your code

  axes([0.3, 0.3, 0.6, 0.6])

The default subplot(111) is much bigger than this and you can't fit
the whole axes and the table on the figure.

    John> I'd also like to make it so you can specify that text should
    John> be left/right/centre aligned.

    John> There is also quite a bit of work to do on the actual
    John> interface to the table objects, basically making them smart
    John> about the parameters they are given so that simple cases
    John> just work by magic.  They are in danger of sprouting a
    John> plethora of options + no-one will be able to figure out how
    John> to use the things.

    John> Anyway, so far it has been fun working with this stuff.

Last night I had a completely different idea that might be very nice.
Create a Cell object that subclasses patch.Rectangle but is
initialized with text.  draw the text in the cell. You could easily
layout the text within the cell to be right, left, top, bottom,
justified using the text alignment properties.  You would also have
full control of the face and edge color of the cells, so you could
have alternating colors for the rows, etc.  Or you could make the
inner cells have a white face color and black edge color (standard
table look) or a white edge color.  Ie, you would have total control.

You would build the table by placing a bunch of cells in rows and
columns.

One potential downside of this approach is that it might be hard to
see text over a dark color, and your current side-by-side approach
avoids this.  But you could workaround this by having a colored cell
with no text next to a while cell with text, and so on.  The other
downside is that it would mean more or less starting over.  My guess
is that it would be a clean design and easy to implement.

If you go this route, I might advise you not to follow the legend
example and your current approach which is to pass line/patch
instances and use introspection to automagically determine the colors.
It would simplify your life if you just provided helper methods like

  set_row_facecolor(colorarg, i)
  set_row_edgecolor(colorarg, i)
  set_col_facecolor(colorarg, j)
  set_col_edgecolor(colorarg, j)
  set_cell_edgecolor(colorarg, i, j)  and so on

Let the user build the table.  We can supply a helper function to
automatically build tables from stacked bar charts using
introspection, but I think from a design standpoint you will make your
life a lot easier by separating out this functionality.

JDH

[matplotlib-devel] Tables for matplotlib

[John G. <jn...@eu...>]

Hi John,

I've moved on a fair bit with this, still quite a way to go.

I'm attaching some new files and patches which will make it easier for
me to explain where i am at with all this.

table.py is my hacked version of legend.py that does tables.

axes.patch is a patch to axes.py that adds table support.

table_demo.py and table_demo2.py are a couple of demos of what I've got
so far.

table_demo.py shows tables being places all over the place.

table_demo2.py shows the sort of thing I'm really after.

The basic idea with table is that it allows you to create a table which
can be placed either inside the axes (as per legend), or outside the
axes.  This is controlled by the loc arguement.

Each cell in a table can have an optional handle as per legend and some
text.

Currently you have to specify the column widths for the table.   Really
two cases should be supported: allow the user to specify the widths
(this is needed so you can line up tables with bar plots) and allow the
Table to auto-adjust the widths to make everything just big enough for
whatever text is present (this is handy for legend type stuff).    I've
done nothing for auto-guessing of widths as yet.

I've made the grid within the tables an option + I think with a little
more work legend.py could just use table.py to do what it has to do.

There is an ugly hack, rowOffset that I use in table_demo2.py to get the
row labels to align with the appropriate data.   My idea was that it
might be simpler to have a basic table object and then build up more
complicated stuff such as including row-labels by creating multiple
tables.   

Currently I can't decide whether i wouldn't be better trying to support
the row/column labels all in the one object -- I suspect this is the way
to go since it is easier to get things to line up this way rather than
trying to align lots of separate tables (eg if i start supporting
different fonts for different tables then we'll really be in trouble).

Now as I said there is still lots to do here.   

There are all sorts of minor and not so minor alignment issues to
address -- a lot of these I think I can solve by paying a bit more
attention to the legend.py code.

The most significant problem at the moment is that the tables are
getting clipped -- I'm not sure how I control the size of the border
around the axes.

I'd also like to make it so you can specify that text should be
left/right/centre aligned.

There is also quite a bit of work to do on the actual interface to the
table objects, basically making them smart about the parameters they are
given so that simple cases just work by magic.  They are in danger of
sprouting a plethora of options + no-one will be able to figure out how
to use the things.   

Anyway, so far it has been fun working with this stuff.

John

[matplotlib-devel] Re: Adding tables

[John H. <jdh...@ac...>]

>>>>> "John" == John Gill <jn...@eu...> writes:

    John> I was hoping I could do things like specify negative
    John> y-positions to draw below the axes, but I now think I'm
    John> deluded in thinking this 'cos matplotlib is smart and every
    John> time something gets drawn the axes are automagically
    John> adjusted to make sure the latest lines/rectangles are
    John> included.

If this is the case it appears to me that you are using the
axes.add_line command, no?  That is where matplotlib does the
autoscale view limits thingie.  There is nothing in the architecture
that prevents you from drawing outside the axes view limits, except
for clipping, which you can set.

I think you should have a Table class which is contained by the axes.
The Table should derive from an Artist and implement the required
_draw method, which is called with a renderer instance.  This method
should forward the draw call to all the text, lines and patches
contained by the table, just as legend does.

class Table
    def _draw(self, renderer):
        for line in self._lines: line.draw(renderer)
        for t in self._texts: t.draw(renderer)

This is how Legend does it.  If you set it up this way, the axes
instances won't know anything about the line instances and you can
definitely draw outside the axes bbox.  If not, matplotlib.axes has
achieved consciousness and we are no longer in control <wink>.  Note
that it is critical that you make the call

   self.line1.set_clip_on(False)

to allow drawing outside the axes bbox.

Here's an example table class that you can use to draw inside or
outside the axes bbox.  Note there is no reason we can't do this at
the figure level, but the advantage of doing it at the axes level is
that you may want some of the lines to be in data coords, eg, the
vertical lines lining up with the xticks in the example you showed me.
It might be worth taking some time to figure out how to have figure
tables or axes tables, eg with a base class and 2 derived classes, but
for now focus on the axes table and we can generalize once you have
the nuances worked out.

class Table(Artist):
    def __init__(self, axes):
        Artist.__init__(self, axes.dpi, axes.bbox)
        self.axes = axes
        left = 0.9
        right = 1.2
        bottom = 0.9
        top = 1.5
        
        self.line1 = Line2D(
            axes.dpi, axes.bbox,
            (left, left), (bottom, top),
            transx=self.axes.xaxis.transAxis,
            transy=self.axes.yaxis.transAxis
            )
        self.line1.set_clip_on(False)

    def _draw(self, renderer):
        self.line1.draw(renderer)

You can add an add_table method to Axes, and define a list of 

    class Axes(Artis):
        def __init__(self, blah, blah):
            # ..snip the other Axes init stuff
            self._tables = []  

        def add_table(self, table):
            self._tables.append(table)

        def _draw(self, renderer):
            # ..snip the other draw calls

            for table in self._tables:
                table.draw(renderer)

I tested my code above with this scheme and it works - it drew a
vertical blue line from inside the axes to outside.

If you have any more troubles send me some code and I'll take a look.

JDH

[matplotlib-devel] more on image backends

[John H. <jdh...@ac...>]

I've spent some more time working on the idea of using a common image
renderer for the GUIs.  As I mentioned before, this will remain
optional so no need to be concerned about losing support for the
current GTK or WX backend, but this will enable us to add capabilities
to the GUI backends that they may not natively support.

I've been working on the GTKGD backend as a testbed since we already
have the GUI architecture in GTK and the drawing architecture in GD.
This already backend provides additional capabilities to GTK, namely
antialiased lines and arbitrary text rotation.  I wrote some C code to
transfer the image from GD->GTK so it is now fast enough to be usable,
though not as fast as the native GTK solution.  There are some
performance bottlenecks in GD that I've identified in the profiler so
the current speed can be improved.

David Moore has implemented a paint backend (a libart wrapper).
libart is a sophisticated render engine that is currently used as the
renderer for Gnome Canvas and is ported to all the major platforms.
Although he is waiting on some paint patches he applied to be
incorporated, this provides another candidate backend for a common
image renderer.

I've updated CVS.  In setup.py there is a line 'if 0' that needs to be
replaced with 'if 1' to compile the extension module (does anybody
know how to set flags for distutils?)  The GTKGD backend now passes
all the regression tests (though there is a color allocation but that
seems to be a gdmodule problem) and serves as a template for GUI
implementers who want to get something up and running fast.  With this
approach, the backend writer does not need to implement either a
Renderer or a GraphicsContext.  Once you have the GUI architecture
setup, adding a different image renderer is as simple as doing a
importing a different FigureCanvasBackend and writing an image->gui
canvas transfer function.

If there are any brave souls who want to test this out and working,
I'd be much obliged.  You'll need the requirements for the GTK and GD
backends installed as described on
http://matplotlib.sourceforge.net/backends.html, including the
gdmodule patch.  Let me know if you encounter any compile problems.

JDH

[matplotlib-devel] Re: Adding tables

[John G. <jn...@eu...>]

John,

Thanks for the hints.

First I've tried subscribing to the devel list -- sourceforge is sulking
again, so no joy so far, i've cc'ed the list on this, so if it accepts
posts from non-subcribees it should get there.

I've looked a bit at the transforms stuff + things are making a bit more
sense, but I'm still unable to achieve what I'd like.

I think the basic problem is that Line2D and Rectangle are intended to
draw on the axes, whereas what I'd ideally like to do is draw outside
the axes (see the screenshot i sent originally) -- ie instead of doing
what is done with the legend and have it appear somewhere within the
axes of the plot I'd like the table of data to be outside this.

I was hoping I could do things like specify negative y-positions to draw
below the axes, but I now think I'm deluded in thinking this 'cos
matplotlib is smart and every time something gets drawn the axes are
automagically adjusted to make sure the latest lines/rectangles are
included.

I suspect I need some new sort of object to draw outside the axes - can
you confirm that is the case?

Plan B. would be to just live with putting the tables within the plot,
as per the legend, but this doesn't work too well in general 'cos the
table tends to obscure some important part of the plot.

Let me know if this is still hard to understand and I'll try and get
what I have into a state which demonstrates the problem I am running
into.

John

[matplotlib-devel] Re: Adding tables

[John H. <jdh...@ac...>]

>>>>> "John" == John Gill <jn...@eu...> writes:

Hi John, could you also subscribe to matplotlib-devel and CC the
messages to me (matplotlib-devel sometimes has a long lag).  It would
be nice to have these discussions there for archival purposes and so
that others can offer suggestions.  Hopefully, you'll have an easier
time getting sighed up there than you did on matplotlib-users.

    John> John, First the good news.  I've had a good read of
    John> legend.py and now pretty much understand how it is working.

Excellent, between the 2 of us, that makes at least one person who
understands that code <wink>.

    John> Now the bad news.  The way I'm thinking about these tables
    John> is that I will want to draw them below the main plot area,
    John> sort of where the xticklabels go at the moment + the area
    John> below.

But you should try and code it generally so people can place them
wherever they want right?  Perhaps you should construct the table with
a list of horizontal lines and a list of vertical lines and their
respective transforms (see below).  Or are you already doing this?

    John> Now I tried hacking about a table.py copy of legend.py and
    John> drawing a grid of lines in this area -- the code runs fine
    John> but I don't get a nice grid of lines :( (see snippet of code
    John> below).

    John> If I cheat and arrange for the drawing to take place within
    John> the main plot (by carefully fixing the ypos stuff below)
    John> then things work fine.

    John> I'm guessing I need to do something to let matplotlib know
    John> the extent of what I'm drawing, but I am at a loss as to
    John> what that something is.

    John> Can you point me in the right direction?

Without a complete code example, I can only guess.  My guess is that
you are using the wrong transforms.  Each axis instance has a
transAxis and a transData attribute you can use.  When you want to
specify a coordinate in axis units (0,1), use the transAxis instance.
If you want to specify a coordinate in data units, use transData.

Eg, if you want the vertical lines (x coords) of your table to line up
with the xticks and the horizontal lines (y coords) of your table to
be in axis units (eg 10% of the axis apart), you would initialize your
line like

   ypos = 0.1   # 10% of axes height
   line = Line2D( self.dpi, self.bbox,
        xdata=(xpos, xpos), ydata=(0, ypos),
        color='k', linestyle=':',
        transx = self.axes.xaxis.transData,
        transy = self.axes.yaxis.transAxis,
       )

Make sure you turn clipping off (it appears you did for your
examples), particularly while developing.

    John> (aside: it did occur to me that one way i could cheat an
    John> nearly get what i want is to use the bar() method to draw my
    John> grid + then use the text() method to enter all the text I
    John> want in the grid...)

Cheating is usually a bad thing.  I think you'll need to construct
your own text instances with the same x and y transforms you use for
your lines, eg, x coordinate in data units and y coordinate in axis
units. This will also encapsulate the table as a single instance which
will make it easier to manipulate; eg to set text properties for the
entire table.

Hope this helps, if not send me a complete example with demo script
and I can take a closer look.

JDH

Re: [matplotlib-devel] substantial refactoring of backends

[Jeremy O'D. <je...@o-...>]

On Wednesday 28 January 2004 10:06 pm, John Hunter wrote:
> I just finished reorganizing the backend code.  The most significant
> change is that backends no longer derive their own figures.  Figures
> now derive from Artist and are totally backend independent.  All the
> GUI functionality formerly in FigureBackend, is now in a new class
> FigureCanvasBackend, which does Rendererer installation, GUI event
> handling, etc...  For GUI classes, FigureCanvasBackend should derive
> from a widget you can insert into a widget container.
>
> FigureManagerBackend is initialized with a canvas.  The attribute name
> canvas is standardized across backends.
>
> So the containment is manager contains canvas contains figure.
>
> The importance of these changes is
>
>  1) Backend switching is now perfected since figure instances contain
>     *no* backend specific information - of course mainloops will prevent
>     switching between gtk and wx
>
>  2) This enables a backend to render to any other non interactive
>     backend (eg, PS saves from GTK or WX).  More importantly, it
>     enables us to have a sophisticated image backend (eg agg which
>     supports alpha and antialiased drawing http://www.antigrain.com/)
>     or gd (which is getting better all the time) and render to the GUI
>     from the image - see attachment below.
>
>     In other words, instead of each GUI implementing their own drawing
>     and dealing with fonts and rotation and window extents etc, all
>     this can be relegated to a single high quality image backend and
>     the GUI canvas updated from the image.  Since we're already doing
>     double buffered drawing, there would be no little or no additional
>     performance hit.
>
>     All at once this buys us font standardization across GUI backends,
>     arbitrary text rotation across GUI backends, and pixel for pixel
>     compatibility across GUI backends.  I think it's an idea worth
>     serious consideration, so please weigh in.

This is something to shoot for, but worth bearing in mind that it will 
probably become quite tricky to maintain across the supported platforms. As 
we stood a couple of weeks ago, Matplotlib worked on Linux (all backends), 
Windows (at least wx, and non-GUI backends except GD), Mac (I believe) and 
probably most other platforms.

I know that GD is not very easy to make work on Windows, and I worry that if 
Matplotlib starts to have large numbers of external dependencies, it will 
reduce the overall attractiveness of the library.

>     It would probably entail some specialized C code to move images
>     from the image backend to the GUI canvas for speed.  I've
>     implemented a proof of concept GTK backend called GTK2.  It uses
>     GD for drawing the image.  It's slow, because I use python to
>     transfer the image, but it works.  And note it is only 80 lines of
>     code (matplotlib.backends.backend_gtk2), which shows how easy it
>     is to plug an arbitrary image renderer into an arbitrary GUI
>     backend under the new framework.  As before, you can also export
>     PS and EPS from this backend.

C code, in particular, can be tricky to write in an optimal way across 
platforms (e.g. Mac/Sparc are big-endian, X86 is little endian, making fast 
bit blitting routines potentially tricky when used in conjunction with a 
multiple set of backends.

If we can find a truly cross-platform way to render to a bitmap, which is 
actively developed and supported on multiple platforms, then this would be 
great - my worry is that we end up discovering that GTK, wx, Tk and so on are 
actually the closest thing we have to this.

>     If you want to try this out, you'll need gdmodule-0.51 (and the GD
>     dependencies described at
>     http://matplotlib.sourceforge.net/backends.html#GD).  gd lib has
>     recently added clipping and antialiased line drawing.  I've
>     patched the 0.51 gdmodule wrapper to add support for this and will
>     email the maintainer the patch when I get some time.  In the mean
>     time, just replace _gdmodule.c in the 0.51 distro with the file
>     I'm attaching below.

I worry that all of the above sounds rather negative, and it isn't meant to 
be. However, (unlike most other Matplotlib users) my main target platform in 
Windows - force of working necessity :-( - and I don't want to get left out 
of the party...

However, if we can find the right way to do this, it would be an excellent 
solution.

> This CVS update breaks WX (sorry Jeremy!).  Since Jeremy is otherwise
> occupied :-), I'll try and port WX tomorrow.
>
> For those of you using matplotlib in GUI apps, the new setup requires
> some minor (one liner) API changes -- see embedding_in_gtk.py for an
> example and the CVS file API_CHANGES for more info

Re: [matplotlib-devel] substantial refactoring of backends

[John H. <jdh...@ac...>]

>>>>> "Jeremy" == Jeremy O'Donoghue <je...@o-...> writes:

    Jeremy> This is something to shoot for, but worth bearing in mind
    Jeremy> that it will probably become quite tricky to maintain
    Jeremy> across the supported platforms. As we stood a couple of
    Jeremy> weeks ago, Matplotlib worked on Linux (all backends),
    Jeremy> Windows (at least wx, and non-GUI backends except GD), Mac
    Jeremy> (I believe) and probably most other platforms.

A bit better than this actually.  If you hadn't been so busy having a
baby <wink>, you would have seen on matplotlib-users that Stefan
Kuzminski managed to get gd compiled on win32 and posted compile notes
and a statically linked dll.  As soon as I get the time, I'm going to
test it out and update the gd info on the web page.

Also, GTK works great on win32 and OSX.  So all four backends are
confirmed to work on linux, win32 and OSX, and I suspect the major
unixes but I don't have many reports.

    Jeremy> I know that GD is not very easy to make work on Windows,
    Jeremy> and I worry that if Matplotlib starts to have large
    Jeremy> numbers of external dependencies, it will reduce the
    Jeremy> overall attractiveness of the library.

    Jeremy> C code, in particular, can be tricky to write in an
    Jeremy> optimal way across platforms (e.g. Mac/Sparc are
    Jeremy> big-endian, X86 is little endian, making fast bit blitting
    Jeremy> routines potentially tricky when used in conjunction with
    Jeremy> a multiple set of backends.

    Jeremy> If we can find a truly cross-platform way to render to a
    Jeremy> bitmap, which is actively developed and supported on
    Jeremy> multiple platforms, then this would be great - my worry is
    Jeremy> that we end up discovering that GTK, wx, Tk and so on are
    Jeremy> actually the closest thing we have to this.

Interesting that you say this.  I haven't done any rigorous
performance tests, but gd certainly seems to be slower than the GTK
backend, in my experience.  Performance is a major issue for me, so I
wouldn't consider anything that is more than a little bit slower than
what we can do now with the native GUI solutions.  I didn't mean to
imply that GD would be the image backend of choice, only one to
consider.  I used it as a proof-of-concept in the gtk2 backend simply
because it already existed.

    Jeremy> I worry that all of the above sounds rather negative, and
    Jeremy> it isn't meant to be. However, (unlike most other
    Jeremy> Matplotlib users) my main target platform in Windows -
    Jeremy> force of working necessity :-( - and I don't want to get
    Jeremy> left out of the party...

There's no worry here for you.  Half of our downloads are for the exe
installer, so you're certainly not alone.  And in my own work, I
distribute apps internally to the hospital where the users are
exclusively win32.  So win32 compatibility and performance is an
absolute requirement.

In any event, using an image renderer to supply the GUI backends would
be optional.  I don't see any reason to remove the existing
functionality we have for GTK and WX.  Eg, in the backend_gtk2
example, I replaced the native GTK calls with the gd renderer in very
few lines of code.  It would be relatively straight forward to mixin a
GUI framework with an image renderer, at least from a design
standpoint.  Performance, as you point out, is another issue.

    Jeremy> However, if we can find the right way to do this, it would
    Jeremy> be an excellent solution.

The 3 main benefits I see are 

 * cross GUI image compatibility - not critically important but
   worthwhile

 * more sophisticated image rendering than may be available in a given
   GUI backend; eg, there's no obvious way to do something like
   Gouraud shading for pcolor in the gtk backend.  potentially very
   important

 * narrow focus for addition of new features.

The latter one is important to me since I maintain 3 of the backends,
and find myself having to make improvements in more than one place
when I find something amiss, which is usually a sign that you are
doing something wrong.

The main negative is the one you pointed to: additional complexity
makes the package more difficult to install and maintain.

JDH

[matplotlib-devel] matplotlib-0.50 alpha release

[John H. <jdh...@ac...>]

I've spent the last couple of days refactoring the matplotlib
backends, fixing bugs and adding some functionality.  Here's a
synopsis of what's new.  I encourage everyone to try it out so
complaints and bugs can be handled before the major release.

** Note there are some API changes so please read about this below **

** Note, GD users, GD rendering is significantly improved in my
   opinion.  However, some of new functionality requires a recent
   version of gd and a patch of the latest gdmodule, see below **

What's new in matplotlib 0.50e

  GD supports clipping and antialiased line drawing.  The line object
  has a new 'antialiased' property, that if true, the backend will
  render the line antialiased if supported.  **You will need to
  upgrade to gd-2.0.15 or later and gdmodule-0.51.  You will also need
  to replace _gdmodule.c with the code as described at
  http://matplotlib.sourceforge.net/backends.html#GD.

wild and wonderful bar charts

  You can provide an optional argument 'bottom' to the bar command to
  determine where the bottom of each bar is, default 0 for all. This
  enables stacked bar plots and candelstick plots --
  examples/bar_stacked.py. Thanks to David Moore and John Gill for
  suggestions and code.

Bugfixes (by backend)

    * All : the yticks on the right hand side were placed incorrectly,
      now fixed

    * All : ticklabels now make a more intelligent choice about how
      many significant digits to display

    * GD : An int truncation bug was causing the dotted lines to
      disappear

    * GD and GTK : Fixed line width to scale with DPI

    * GD : Fixed small text layout bug

    * GD : Fixed the constant for GD which maps pixels per inch - this
      should give better agreement with other backends witht he
      relative sizes of objects

    * GTK : Dash spacing was not properly scaling with DPI


Figure backend refactored

  The figure functionality was split into a backend independent
  component Figure and a backend dependent component
  FigureCanvasBase. This completes the transition to a totally
  abstract figure interface and improves the ability the switch
  backends.  See the file
  http://matplotlib.sourceforge.net/API_CHANGES that comes with the
  src distro for information on migrating applications to the new API.
  All the backend specific examples have been updated to the new API.


Enjoy,
John Hunter

[matplotlib-devel] draw_text

[John H. <jdh...@ac...>]

As we talked about a few weeks ago on matplotlib dev, I finally got
around to refactoring the AxisText handling to make it backend
independent.  In a nutshell, the renderer now implements draw_text,
which takes a matplotlib.text.Text instance.  Thus text behaves more
like the other artists in the figure (lines, patches, etc...).  The
renderer also has to implement a get_text_extent(text) method which
does the work formerly done by AxisTextWX.get_window_extent.

This clears the way for easier and better backend switching (eg,
saving PS from GTK or WX backends).  I've ported the GTK and PS
backends to the new API, and will start on GD after this email.  CVS
is updated.  I've uploaded a snapshot to
http://nitace.bsd.uchicago.edu:8080/files/share/matplotlib-0.42c.tar.gz
in case sf mirrors are lagging.

Jeremy, this probably won't be hard for you; basically just move all
the relevant AxisTextWX methods to RendererWX, with minor rewrites.
Take a look at the GTK backend for inspiration -- I do some cacheing
of font properties for efficiency with layoutd, which caches pango
layouts using text properties as keys.

I've also added a file to CVS called API_CHANGES, where I've
documented all the API changes.  You may want to take a look at this
too.

Thanks!
John Hunter

Re: [matplotlib-devel] text drawing from the renderer

[Jeremy O'D. <je...@o-...>]

John Hunter said:
>
> I'd like to refactor text so that backends will no longer need to
> implement classes derived from AxisTextBase, but rather provide a
> RendererBackend.draw_text method (as Jeremy suggested many moons ago).
> This will enable easy switching of backends in midstream, as discussed
> in the last couple of days on matplotlib-users.  One important
> limiting factor in the current implementation is the fact that axes,
> labels, etc.... instantiate derived AxisText classes.  Thus backend
> specific implementations inadvertently creep into (what should be)
> backend-independent interface classes, like Axes, Legend, Tick, Axis,
> etc...
>
> I think we should implement a Text(Artist) class which is totally
> backend independent (analogous to Line2D and Patch) with most of
> protected attribute data defined in AxistTextBase.  This class would
> store all the relevant text properties (fontsize, fontname,
> fontweight, etc) in a standardized way, and provides a few backend
> independent utility funcs.  The renderer would implement get_text_bbox
> and draw_text, each of which take a text instance as an argument;
> these two funcs, are the workhorses of text implementations.
>
> Jeremy, do you see any major problems with this proposal vis-a-vis wx?

The trickiest problem is that WX requires a device context to be able to
determine the size of a given piece of text, since in WX, GetTextExtent()
is a member of wxDC. This was a major pain when implementing text in
backend_wx, and is one of the messier pieces of code. Provided that the
implementation only requires the calculation of text extent when there is
a gc instantiated, we should not have much problem. From the sound of you=
r
proposal, this would be the case.

> On a related note, we should shoot for standardization of font names
> for the next major release.  Which fonts does WX provide, and which
> should be part of the core?

WX provides the following aliases (you'll see some of them used in
backend_wx):

wxSWISS      - a Sans-serif font
wxROMAN      - a Serif font
wxSCRIPT     - a cursive font
wxDECORATIVE - not sure - never used it!
wxMODERN     - a fixed pitch font
wxDEFAULT    - default - seems to be same as wxMODERN on Windows

The mapping to platform fonts depends on the WX platform, but there is a
further complication: non-TrueType fonts cannot be rotated on Windows
platforms, and some of the above are defined as non-TT fonts.

I suppose that we should have a dictionary which should be populated by
the backend. For backend_wx, I'm rather inclined to choose the some of th=
e
standard Windows fonts, rather than the WX defaults. On Linux, I really
need to look into anti-aliased text, but it probably makes sense to use
those nice new fonts supplied with Gnome 2 (as they are GPL). This means
that I may have to introduce a platform dependency into backend_wx to
ensure that a TT font is always chosen. We should also ensure that the
user can specify platform fonts if they really wish (again, backend_wx
allows for this by checking for a font name which is not in its
dictionary).

I should also remind you that (since the backend will no longer be
responsible for scaling), WX does not cope properly with scaling/rotation
of font sizes over 60 pts on all platforms. I currently simply clip the
maximum font size at 60 in backend_wx (which works fine in practice since
this is always readable) - it may be advisable to be able to cope with
this (e.g. let the backend clip point sizes it cannot handle, provided
that it correctly returns the text extent in such a case).

Overall, it's a very good idea. I can't see any major issues, and I think
I've outlined the minor ones.

Regards

Jeremy

> Thoughts?
>
> JDH
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Perforce Software.
> Perforce is the Fast Software Configuration Management System offering
> advanced branching capabilities and atomic changes on 50+ platforms.
> Free Eval! http://www.perforce.com/perforce/loadprog.html
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

[matplotlib-devel] text drawing from the renderer

[John H. <jdh...@ac...>]

I'd like to refactor text so that backends will no longer need to
implement classes derived from AxisTextBase, but rather provide a
RendererBackend.draw_text method (as Jeremy suggested many moons ago).
This will enable easy switching of backends in midstream, as discussed
in the last couple of days on matplotlib-users.  One important
limiting factor in the current implementation is the fact that axes,
labels, etc.... instantiate derived AxisText classes.  Thus backend
specific implementations inadvertently creep into (what should be)
backend-independent interface classes, like Axes, Legend, Tick, Axis,
etc...

I think we should implement a Text(Artist) class which is totally
backend independent (analogous to Line2D and Patch) with most of
protected attribute data defined in AxistTextBase.  This class would
store all the relevant text properties (fontsize, fontname,
fontweight, etc) in a standardized way, and provides a few backend
independent utility funcs.  The renderer would implement get_text_bbox
and draw_text, each of which take a text instance as an argument;
these two funcs, are the workhorses of text implementations.

Jeremy, do you see any major problems with this proposal vis-a-vis wx?

On a related note, we should shoot for standardization of font names
for the next major release.  Which fonts does WX provide, and which
should be part of the core?  

Thoughts?

JDH

Re: [matplotlib-devel] Re: [Matplotlib-users] Troubles with WX backend

[John H. <jdh...@ac...>]

>>>>> "Jeremy" == Jeremy O'Donoghue <je...@o-...> writes:

    Jeremy> Hi John, Thanks for your reply - and the patch! I am about
    Jeremy> to check in an updated backend_wx which basically contains
    Jeremy> your changes, with one difference: I have refactored the
    Jeremy> code:

    Jeremy>         drawDC=wxClientDC(self) drawDC.BeginDrawing()
    Jeremy> drawDC.Clear() drawDC.DrawBitmap(self.bitmap, 0, 0)
    Jeremy> drawDC.EndDrawing()

Excellent, I was going to suggest the same.

I have rewritten the interactive interface so that you can call

import matplotlib
matplotlib.use('WX')
matplotlib.interactive(True)

(and I've removed the ShowOn abomination)

Perhaps you should just send me the latest version of you code in case
the mirror lags behind as usual and I'll then apply my patch to the
wx_backend for the new interface, and commit.

JDH

[matplotlib-devel] Re: [Matplotlib-users] Troubles with WX backend

[Jeremy O'D. <je...@o-...>]

Hi John,

Thanks for your reply - and the patch! I am about to check in an updated 
backend_wx which basically contains your changes, with one difference: I have 
refactored the code:

        drawDC=wxClientDC(self)
        drawDC.BeginDrawing()
        drawDC.Clear()
        drawDC.DrawBitmap(self.bitmap, 0, 0)
        drawDC.EndDrawing()        

into a new function in FigureWX: gui_repaint(). I have done this for several 
reasons, but mainly because the same code appears in several places 
(FigureWX._onSize() as well as draw_if_interactive(), and it turns out to be 
needed for the Wx variant of interactive_demo (which I am also about to check 
in)).

I have also made a change so that if _DEBUG is set to a value less than 5 
(i.e. you are interested in debugging), the system exception hook is replaced 
with one which performs a traceback and enters pdb. This seems to work 
correctly for me on Linux, and I'll verify Win32 when I get back to work next 
week. It's a bit of an ugly hack, and I may rethink it later, but it's very 
handy for now, and doesn't intrude when not needed.

On Monday 29 December 2003 4:15 pm, John Hunter wrote:
> >>>>> "Jeremy" == Jeremy O'Donoghue <je...@o-...> writes:
>
>     Jeremy> A few questions for John Hunter: I think I need to do
>     Jeremy> something like the following: - show() must now
>     Jeremy> instantiate any figures already defined and enter the the
>     Jeremy> main event loop. ShowOn needs to keep track of this.  - I
>     Jeremy> need to keep track of the number of figures
>     Jeremy> instantiated. I assume that Gcf.destroy() does this.  - I
>     Jeremy> need to ensure that I do not exit when the last figure is
>     Jeremy> destroyed, and therefore need to manage that I may need to
>     Jeremy> create a new figure manager if there is none.
>
> Hi Jeremy,
>
> I think we should consider redoing this whole segment of the code from
> the ground up to make for a cleaner / cross GUI implementation.  It
> may be that Pearu Peteson <pe...@ce...> gui_thread code is the
> way to go for this since that is what is was designed to do (enable
> interactive control of WX plots (chaco) from the shell).  He has
> indicated a willingness to port it to pygtk provided we're willing to
> help test his code with the WX and GTK matplotlib backends.  However,
> this would create a scipy dependency...

I followed the thread form Pearu a few weeks back, although have not gotten 
around to trying the gui_thread code. I think there is something similar in 
the Python Cookbook for PyGtk (chapter 9.12), which might assist in doing a 
Gtk port.

I'm not keen on adding a dependency on scipy, but perhaps Pearu would consider 
making gui_thread its own module (or allowing us to do so). Checking back 
over the mail you sent to me, there are only three files involved. One of the 
things which drew me to working on Matplotlib was the small set of 
dependencies, and it would be a shame to loose this.

I'm very willing to play with the code he has with the Wx backend - I doubt 
that it will require much work on my behalf.

[snip]

> I don't think the current architecture for show, draw_if_interactive,
> ShowOn etc, is very elegant or easily understandable, and would be
> happy to refactor it for the next release.  Ideally we could handle
> these two cases across backends
>
>   1) defer all drawing until a call to show, which draws and realizes
>      all pending figures.  This should not hang the script, ie,
>      further drawing commands should be possible.

You'll notice in the code that I checked in that I have been playing with this 
in the backend_wx code. While it is possible to exit the mainloop, it seems 
that it is not possible to re-enter. It may be that I can do something akin 
to the Python Cookbook recepie I mentioned above for wx - it doesn't look too 
hard.

I have attached the code from the book example below, if you're interested.

>   2) do drawing with each matplotlib.matlab command for interactive
>      mode (current implementation in backend_gtk with ShowOn.set(1) at
>      start of script)

As you noteed, this now works with your fix.

> For the most part, I think we have this with the GTK backend, but it
> may be necessary to refactor in order to get something that works with
> both.  I'll think it over and take a look at the WX code to see if I
> get any ideas how to proceed.
>
> In the meantime, we should also see if we can get matplotlib to work
> with gui_thread -- I'll take a look at this too.
>
> JDH

Thanks again for the fix

Jeremy

====== interactive_gtk.py =====

import __builtin__, __main__
import codeop, keyword, gtk, os, re, readline, threading, traceback, signal, 
sys

def walk_class(klass):
    list = []
    for item in dir(klass):
        if item[0] != "_":
            list.append(item)

    for base in klass.__bases__:
        for item in walk_class(base):
            if item not in list: list.append(item)

    return list

class Completer:
    def __init__(self, lokals):
        self.locals = lokals
        self.completions = keyword.kwlist + \
                            __builtins__.__dict__.keys() + \
                            __main__.__dict__.keys()

    def complete(self, text, state):
        if state == 0:
            if "." in text:
                self.matches = self.attr_matches(text)
            else:
                self.matches = self.global_matches(text)
        try:
            return self.matches[state]
        except IndexError:
            return None

    def update(self, locs):
        self.locals = locs

        for key in self.locals.keys():
            if not key in self.completions:
                self.completions.append(key)

    def global_matches(self, text):
        matches = []
        n = len(text)
        for word in self.completions:
            if word[:n] == text:
                matches.append(word)
        return matches

    def attr_matches(self, text):
        m = re.match(r"(\w+(\.\w+)*)\.(\w*)", text)
        if not m:
            return
        expr, attr = m.group(1, 3)

        obj = eval(expr, self.locals)
        if str(obj)[1:4] == "gtk":
            words = walk_class(obj.__class__)
        else:
            words = dir(eval(expr, self.locals))

        matches = []
        n = len(attr)
        for word in words:
            if word[:n] == attr:
                matches.append("%s.%s" % (expr, word))
        return matches

class GtkInterpreter(threading.Thread):
    """ Run a GTK mainloop() in a separate thread. Python commands can be 
passed to the
    thread, where they will be executed. This is implemented by periodically 
checking for
    passed code using a GTK timeout callback. """
    TIMEOUT = 100 # interval in milliseconds between timeouts

    def __init__(self):
        threading.Thread.__init__ (self)
        self.ready = threading.Condition ()
        self.globs = globals ()
        self.locs = locals ()
        self._kill = 0
        self.cmd = ''       # current code block
        self.new_cmd = None # waiting line of code, or None if none waiting

        self.completer = Completer(self.locs)
        readline.set_completer(self.completer.complete)
        readline.parse_and_bind('tab: complete')

    def run(self):
        gtk.timeout_add(self.TIMEOUT, self.code_exec)
        gtk.mainloop()

    def code_exec(self):
        """ Execute waiting code. Called every timeout period. """
        self.ready.acquire()
        if self._kill: gtk.mainquit()
        if self.new_cmd != None:
            self.ready.notify()
            self.cmd = self.cmd + self.new_cmd
            self.new_cmd = None
            try:
                code = codeop.compile_command(self.cmd[:-1])
                if code:
                    self.cmd = ''
                    exec code, self.globs, self.locs
                    self.completer.update(self.locs)
            except:
                traceback.print_exc()
                self.cmd = ''

        self.ready.release()
        return 1

    def feed(self, code):
        """ Feed a line of code to the thread. This function will block until 
the code is
        checked by the GTK thread. Returns true if the thread has executed the 
code.
        Returns false if deferring execution until complete block is 
available. """
        if code[-1:]!='\n': code = code +'\n' # raw_input strips newline
        self.completer.update(self.locs)
        self.ready.acquire()
        self.new_cmd = code
        self.ready.wait()  # Wait until processed in timeout interval
        self.ready.release()

        return not self.cmd

    def kill(self):
        """ Kill the thread, returning when it has been shut down. """
        self.ready.acquire()
        self._kill=1
        self.ready.release()
        self.join()

# Read user input in a loop and send each line to the interpreter thread

def signal_handler(*args):
    print "SIGNAL:", args
    sys.exit()

if __name__=="__main__":
    signal.signal(signal.SIGINT, signal_handler)
    signal.signal(signal.SIGSEGV, signal_handler)

    prompt = '>>> '
    interpreter = GtkInterpreter()
    interpreter.start()
    interpreter.feed("from gtk import *")
    interpreter.feed("sys.path.append('.')")
    if len (sys.argv) > 1:
        for file in open(sys.argv[1]).readlines():
            interpreter.feed(file)
    print 'Interactive GTK Shell'

    try:
        while 1:
            command = raw_input(prompt) + '\n' # raw_input strips newlines
            prompt = interpreter.feed(command) and '>>> ' or '... '
    except (EOFError, KeyboardInterrupt): pass

    interpreter.kill()
    print

[matplotlib-devel] wx swallows exceptions

[John H. <jdh...@ac...>]

I find that when an assertion fails or another exception is raised in
the wx backend, python just exits silently without printing the
traceback or the exception that occurred.  This makes it difficult to
work with the wx backend.

Is there a setting so that I can get the traceback, or another
preferred method?

Thanks,
John Hunter

[matplotlib-devel] Re: [Matplotlib-users] Troubles with WX backend

[John H. <jdh...@ac...>]

>>>>> "Jeremy" == Jeremy O'Donoghue <je...@o-...> writes:


    Jeremy> I can say that in the original design, the intention was
    Jeremy> that show() would be the last line of any script. I know
    Jeremy> that John (author of virtually everything in Matplotlib
    Jeremy> except backend_wx) has recently made some changes to allow
    Jeremy> show() to be called more than once.

    Jeremy> Unfortunately, the code needed to do thisis quite specific
    Jeremy> to each GUI library, and I cannot simply port what has
    Jeremy> been done for GTK (I've just tried something very close to
    Jeremy> the GTK implementation, and it doesn't work).

    Jeremy> A few questions for John Hunter: I think I need to do
    Jeremy> something like the following: - show() must now
    Jeremy> instantiate any figures already defined and enter the the
    Jeremy> main event loop. ShowOn needs to keep track of this.  - I
    Jeremy> need to keep track of the number of figures
    Jeremy> instantiated. I assume that Gcf.destroy() does this.  - I
    Jeremy> need to ensure that I do not exit when the last figure is
    Jeremy> destroyed, and therefore need to manage that I may need to
    Jeremy> create a new figure manager if there is none.

Hi Jeremy,

I think we should consider redoing this whole segment of the code from
the ground up to make for a cleaner / cross GUI implementation.  It
may be that Pearu Peteson <pe...@ce...> gui_thread code is the
way to go for this since that is what is was designed to do (enable
interactive control of WX plots (chaco) from the shell).  He has
indicated a willingness to port it to pygtk provided we're willing to
help test his code with the WX and GTK matplotlib backends.  However,
this would create a scipy dependency...

But let me give a little overview of why the code is currently the way
it is.  When the use calls 'show' it realizes all the figures and sets

  self.show = on

This is a critical part, because the function draw_if_interactive
(which is called by every matplotlib.matlab command) only draws if
this variable is set

def draw_if_interactive():
    if ShowOn().get():
        figManager =  Gcf.get_active()
        if figManager is not None:
            fig = figManager.figure
            fig.draw()
            fig.queue_draw()

The point I want to emphasize is that the primary reason there is a
difference between interactive and batch mode is for efficiency.  In
interactive mode, the entire figure must be redrawn with each command
(eg, setting an xlabel, changing a ticklabel property, etc...).  This
can get quite expensive when you want to set a lot of properties.
Thus the default 'batch mode' defers all the drawing operations until
the end for efficiency, and just makes one draw.

I don't think the current architecture for show, draw_if_interactive,
ShowOn etc, is very elegant or easily understandable, and would be
happy to refactor it for the next release.  Ideally we could handle
these two cases across backends

  1) defer all drawing until a call to show, which draws and realizes
     all pending figures.  This should not hang the script, ie,
     further drawing commands should be possible.
        
  2) do drawing with each matplotlib.matlab command for interactive
     mode (current implementation in backend_gtk with ShowOn.set(1) at
     start of script)

For the most part, I think we have this with the GTK backend, but it
may be necessary to refactor in order to get something that works with
both.  I'll think it over and take a look at the WX code to see if I
get any ideas how to proceed.

In the meantime, we should also see if we can get matplotlib to work
with gui_thread -- I'll take a look at this too.

JDH

Re: [matplotlib-devel] next week

[<je...@o-...>]

Hi John,

jdh...@ac... wrote:
> 
> I'd like to do the 0.40 release early next week to the wxpython,
> python-list, etc, communities.  Do you think it's time?

I'm reasonably happy with the backend quality for now, and I think the only way we'll start to really squash the bugs is to get more people using matplotlib.

Yes, I think it's time.

> The only significant bug I'm aware of, and it's an important one, is
> the problem with vertical rendering of text in the wxpython with GTK2
> backend.  Unfortunately, since it appears to be a known bug, I'm not
> sure what we can do about it, except perhaps remind Robin Dunn and the
> other wx developers about the problem.
> 
> Your thoughts?

The best approach is probably as you suggest. If we release, perhaps we can find a friendly developer of wxPython with GTK2 who can perhaps help us. I'd agree that it's a significant issue, and I guess that most newer Linux distributions have probably given up on GTK 1.2. I may reconsider whether to try to do a wxPython build linked against GTK2. Debian has a 'fakeroot' utility, which may help me.

Regards

Jeremy

[matplotlib-devel] next week

[John H. <jdh...@ac...>]

I'd like to do the 0.40 release early next week to the wxpython,
python-list, etc, communities.  Do you think it's time?

The only significant bug I'm aware of, and it's an important one, is
the problem with vertical rendering of text in the wxpython with GTK2
backend.  Unfortunately, since it appears to be a known bug, I'm not
sure what we can do about it, except perhaps remind Robin Dunn and the
other wx developers about the problem.

Your thoughts?
JDH

Re: [matplotlib-devel] Bugs and patches posted

[John H. <jdh...@ac...>]

>>>>> "Zachary" == Zachary Pincus <zp...@st...> writes:

    Zachary> Hello - I just posted a few bugs (and patches to fix
    Zachary> them) to the sourceforge bugtracker page before I
    Zachary> realized that the email list is preferred.

More people see the list I think, but the SF mirror for the dev list
is generally woefully out of date, so it's a tossup.

    Zachary> Short story: I found and (hopefully) fixed a problem with
    Zachary> pcolor on non-square arrays, a startup error in the
    Zachary> interactive.py shell, and a problem with repeated output
    Zachary> when saving postscript backend figures. The details of
    Zachary> the bugs and patches are copied below.

Thanks for the detailed information and patches!

    Zachary> This happens because the shape of the X and Y arrays (the
    Zachary> output of a meshgrid call on line 1782) have the wrong
    Zachary> shape. In particular, the shape is backward.  e.g. if
    Zachary> C.shape = (x,y), then X.shape = Y.shape = (y, x).

The behavior of meshgrid certainly is a bit counter-intuitive.  Eg, in
matlab

    >> x = [1:7];
    >> y = [1:5];
    >> [X,Y] = meshgrid(x,y);
    >> Z = rand(length(x), length(y));
    >> pcolor(X,Y,Z);
    ??? Error using ==> surface
    Matrix dimensions must agree.

matplotlib fails in the same way.  I've updated the docs in meshgrid,
pcolor and fixed the meshgrid call in the case of pcolor(Z) for
nonsquare Z.

I added the following to the pcolor docs - let me know if you agree
with this.

"""
    Note, the behavior of meshgrid in matlab is a bit
    counterintuitive for x and y arrays.  For example,

      x = arange(7)
      y = arange(5)
      X, Y = meshgrid(x,y)

      Z = rand( len(x), len(y))
      pcolor(X, Y, Z)

    will fail in matlab and matplotlib.  You will probably be
    happy with

      pcolor(X, Y, transpose(Z))

    Likewise, for nonsquare Z,

      pcolor(transpose(Z))

    will make the x and y axes in the plot agree with the numrows
    and numcols of Z
"""

    Zachary> Patch: Change matplotlib/axes.py line 1782 from: X, Y =
    Zachary> meshgrid(range(numRows), range(numCols)) to: X, Y =
    Zachary> meshgrid(range(numCols), range(numRows))

Done

    Zachary> (*)
    Zachary> http://matplotlib.sourceforge.net/matplotlib.mlab.html#-meshgrid

Fixed doc bug.


    Zachary> Patch: Insert the command: interpreter.feed("from
    Zachary> matplotlib.backends.backend_gtk import ShowOn") somewhere
    Zachary> before line 206.

Fixed -- I don't know how this got removed.

    Zachary> Patch: Insert the following lines at the end of the
    Zachary> print_figure method of the FigurePS class in the file
    Zachary> matplotlib/ backends/backend_ps.py (line 189):
    Zachary> self._pswriter = StringIO()
    Zachary> self._pswriter.write(_psHeader)

Added a flush method, called by __init__ and after writing the figure.

Thanks again -- that was very helpful.

John Hunter

[matplotlib-devel] Bugs and patches posted

[Zachary P. <zp...@st...>]

Hello -

I just posted a few bugs (and patches to fix them) to the sourceforge 
bugtracker page before I realized that the email list is preferred.

Short story: I found and (hopefully) fixed a problem with pcolor on 
non-square arrays, a startup error in the interactive.py shell, and a 
problem with repeated output when saving postscript backend figures. 
The details of the bugs and patches are copied below.

Zach Pincus


Here is the relevant information:
----------------------------------
pcolor(array) fails when array is not square, on version
0.40i of matplotlib.

Details:
A non-square array causes an indexError on line 1798 of
matplotlib/axes.py:
c = C[i,j]
because iteration has continued past the edge of the array.

This happens because the shape of the X and Y arrays (the
output of a meshgrid call on line 1782) have the wrong
shape. In particular, the shape is backward.
e.g. if C.shape = (x,y), then X.shape = Y.shape = (y, x).

Patch:
Change matplotlib/axes.py line 1782 from:
X, Y = meshgrid(range(numRows), range(numCols))
to:
X, Y = meshgrid(range(numCols), range(numRows))

Rationale:
Meshgrid(range(x), range(y)) returns arrays with
a shape of (y, x), which is a bit counterintuitive.
The documentation(*) bears this out. It is confusing
because the convention therein is that a Nx x Ny entry
array has Nx columns and Ny rows, while a (Nx, Ny)-
shaped array (in Matlab as well as Numeric) has Nx rows
and Ny columns. I don't know why the original meshgrid in
matlab worked this way, but it does.

(*) http://matplotlib.sourceforge.net/matplotlib.mlab.html#-meshgrid

----------------------------------

Problem: examples/interactive.py (version 0.40i) does not
start up properly.

Details: A NameError is raised on startup when the
command on line 206 fails:
interpreter.feed ("ShowOn().set(1)")
This is because ShowOn is not in the global namespace.

Not running ShowOn().set(1) breaks interactive mode
badly.

Patch:
Insert the command:
interpreter.feed("from matplotlib.backends.backend_gtk import ShowOn")
somewhere before line 206.

----------------------------------

Problem:
If savefig() is called more than once on a postscript backend
figure, extra copies of the figure will be appended to the
output.

Details:
Regardless of the file name that the figure is saved under,
each time savefig() is called, an additional copy of the figure
appears at the end of the postscript output. So if savefig() is
called four times, the output from that fourth call will be
four copies of that figure on a multipage postscript file.

Patch:
Insert the following lines at the end of the print_figure
method of the FigurePS class in the file matplotlib/
backends/backend_ps.py (line 189):
self._pswriter = StringIO()
self._pswriter.write(_psHeader)

Rationale:
The postscript backend is not clearing the pswriter
accumulator buffer between repeated calls to print_figure.
So after each call, another copy of the figure gets added to
the buffer and dumped to the output. The patch re-
initializes the pswriter buffer.

Note that the patch is sort of bad form because it duplicates
the pswriter initialization code from the __init__ method of
that class. So if the init method is changed, the patched
lines need to be changed too. Better would be to move the
pswriter initialization to a separate function and call that
from both the init and print_figure methods.

Re: [matplotlib-devel] Significant update to backend_wx

[Jeremy O'D. <je...@o-...>]

On Tuesday 02 December 2003 8:29 pm, Jeremy O'Donoghue wrote:
> I'm seeing a failure with one of my asserts, so I'll start there. Should
> have more info shortly.

I have just run through all of the regression tests, and with

assert self.Ok()

in GraphicsContextWx.__init__() removed, all works correctly on my Debian 
platform. 

My suspicion is that the problem is that wxGTK does not like to see calls to 
Ok() in derived constructors. Please let me know if this fixes the problem on 
your machine, John, as I now have no problems on the two platform available 
to me.

A few other notes:

- Save to TIFF does not work on my Linux box. wxGTK generates a 'not 
supported' error message.
- Text is not anti-aliased on my Linux box. I suspect that this is probably a 
GTK configuration issue on my box (other GTK 1.2 apps aren't anti-aliased 
either), but I'd love confirmation.
- Performance of double buffering is vastly better on wxGTK than on Win32. The 
performance under Linux is subjectively very similar to the old GDI-style 
version, but (as previously noted) much worse under Win32. I am not sure why 
this should be the case (my Linux box is much lower spec than the Win32 box, 
as well!)

Regards

Jeremy