matplotlib-users

[Matplotlib-users] numerix on win32

[John H. <jdh...@ac...>]

The new transformation module I've been working on is implemented in
extension code.  It offers a couple of important benefits, namely
speed and generality (affines, nonseparable xy transformations).  It's
not in CVS yet because I'm still ironing out the details.

I have two methods for operating over sequences of x and y, seq_x_y
and numerix_x_y.  The first uses the python sequence API and the
second uses the numerix api.  I expect the latter will be a good bit
faster for long lines, since it doesn't have the python object type
coercion to deal with, but I haven't profiled it.

But there is a gotcha.  When I compile the code, I use flag in
setup.py to indicate whether to compile with Numeric or numarray.
This sets a flag and that is used in the transforms module (and the
image module) that will import arrayobject.h from either Numeric or
numarray.

A problem can arise if someone compiles with one and uses a different
setting in ~/.matplotlibrc.  I compiled the module with Numeric but
used numarray in my matplotlibrc.  The transformation went fine,
presumably thanks to the numarray compatibility layer.  I passed the
transformed arrays off to one of the backends, which does

from matplotlib.numerix import Int16

        y = int(self.height) - y.astype(Int16)

and got a 

  ValueError: type must be either a 1-length string, or python type
  object

clearly because of the difference between numpy and numarray type
args.

Is there a clever workaround for this problem?  It doesn't affect any
of the agg or PS backends.  GTK and GD both need to convert their
arrays to ints before passing them on to their respective renderers.
One solution is to do the conversion in list comps

 y = [ int(self.height-val) for val in y]

which obviously implies a performance hit but probably a bearable one
given the other speedups that the new transformation module implies
and that users have an option to use GTKAgg instead of GTK and Agg
instead of GD for optimal performance.

Another question is win32, where the user doesn't have the choice at
compile time.  If we compile in numpy for win32, the user's numarrays
will be transformed into numpys at render time, but this won't affect
the user arrays (unlike in scipy where the user is getting the return
values) so I don't see it as a big deal.  I think trying to compile in
both or supplying alternate binaries is doable but may not be worth
the maintenance and complexity.

In any case, there are a couple of readers here who know a bit more
about numarray and numeric than I do <wink> ; any thoughts?

JDH

RE: [Matplotlib-users] numerix on win32

[Perry G. <pe...@st...>]

John Hunter wrote:

> Is there a clever workaround for this problem?  It doesn't affect any
> of the agg or PS backends.  GTK and GD both need to convert their
> arrays to ints before passing them on to their respective renderers.
> One solution is to do the conversion in list comps
[...]
> Another question is win32, where the user doesn't have the choice at
> compile time.  If we compile in numpy for win32, the user's numarrays
> will be transformed into numpys at render time, but this won't affect
> the user arrays (unlike in scipy where the user is getting the return
> values) so I don't see it as a big deal.  I think trying to compile in
> both or supplying alternate binaries is doable but may not be worth
> the maintenance and complexity.
[...]
> 
> In any case, there are a couple of readers here who know a bit more
> about numarray and numeric than I do <wink> ; any thoughts?
>
Perhaps there is a clever workaround, but I wonder if the best solution
isn't to build dual versions of the extensions, one for Numeric and one 
for numarray if both are present (and this would also solve the win32
issue by distributing both). I think setup.py could be setup to do this,
and a mechanism for a module that uses these to use whichever 
is selected by the user to pick up the right sharable
(e.g., _numarray_mycext.so vs _Numeric_mycext.so). It's not
elegant, but it should be robust. Todd and I did talk  about if
there was a way to make numarray arrays compatable at the binary 
level; perhaps, but I suspect it involves a lot work and I'm not
sure it will eliminate all issues. So do you want us to try to
come up with a an example setup.py to do this? Perhaps Todd will
have some other ideas.

Perry

Re: [Matplotlib-users] numerix on win32

[Todd M. <jm...@st...>]

On Fri, 2004-05-14 at 08:35, John Hunter wrote:
> The new transformation module I've been working on is implemented in
> extension code.  It offers a couple of important benefits, namely
> speed and generality (affines, nonseparable xy transformations).  It's
> not in CVS yet because I'm still ironing out the details.
> 
> I have two methods for operating over sequences of x and y, seq_x_y
> and numerix_x_y.  The first uses the python sequence API and the
> second uses the numerix api.  I expect the latter will be a good bit
> faster for long lines, since it doesn't have the python object type
> coercion to deal with, but I haven't profiled it.
> 
> But there is a gotcha.  When I compile the code, I use flag in
> setup.py to indicate whether to compile with Numeric or numarray.
> This sets a flag and that is used in the transforms module (and the
> image module) that will import arrayobject.h from either Numeric or
> numarray.
> 
> A problem can arise if someone compiles with one and uses a different
> setting in ~/.matplotlibrc.  I compiled the module with Numeric but
> used numarray in my matplotlibrc.  The transformation went fine,
> presumably thanks to the numarray compatibility layer.  I passed the
> transformed arrays off to one of the backends, which does
> 
> from matplotlib.numerix import Int16
> 
>         y = int(self.height) - y.astype(Int16)
> 
> and got a 
> 
>   ValueError: type must be either a 1-length string, or python type
>   object
> 
> clearly because of the difference between numpy and numarray type
> args.

Yeah.  numarray has full blown type objects which form a hierarchy which
can be used to test for type properties such as signed, integral, etc. 

numarray uses names for its type objects that were used by Numeric to
make 1 character typecodes clearer and easier to remember.  Thus, in
numarray Int16 is an object, but in Numeric Int16 is an alias for 's'.

Over time,  our compatibility ambitions have grown so now numarray
generally works with Numeric typecodes wherever a type object is
expected.  

So,  while this feels like walking backward,  you can say:

        y = int(self.height) - y.astype('s')

and expect it to work with either.

> Another question is win32, where the user doesn't have the choice at
> compile time.  If we compile in numpy for win32, the user's numarrays
> will be transformed into numpys at render time, but this won't affect
> the user arrays (unlike in scipy where the user is getting the return
> values) so I don't see it as a big deal.  I think trying to compile in
> both or supplying alternate binaries is doable but may not be worth
> the maintenance and complexity.
>
> In any case, there are a couple of readers here who know a bit more
> about numarray and numeric than I do <wink> ; any thoughts?

(1) One thought I've had for handling this has been to improve the basic
binary compatibility of numarray with Numeric to the point that numerix
can be extended to the C-level.  In this scheme,  an extension would be
compiled against neither numarray nor Numeric but against numerix; 
which array package was actually being used would be figured out at run
time much as it works at the Python level now.   I'm still exploring the
idea.

(2) I could just build a numarray version of the matplotlob installer
with each release.  That would offload the work from you and would keep
the matplotlib code cleaner.  

Todd

RE: [Matplotlib-users] numerix on win32

[Perry G. <pe...@st...>]

Todd Miller wrote:
> 
> So,  while this feels like walking backward,  you can say:
> 
>         y = int(self.height) - y.astype('s')
> 
> and expect it to work with either.
> 
If this is used as the workaround, perhaps local (to the module)
versions of Int16 and such can be defined as 's' and so forth
so that it will be easier to update code rather than litter
it with such character codes.

Perry 

RE: [Matplotlib-users] numerix on win32

[Todd M. <jm...@st...>]

On Fri, 2004-05-14 at 11:27, Perry Greenfield wrote:
> Todd Miller wrote:
> > 
> > So,  while this feels like walking backward,  you can say:
> > 
> >         y = int(self.height) - y.astype('s')
> > 
> > and expect it to work with either.
> > 
> If this is used as the workaround, perhaps local (to the module)
> versions of Int16 and such can be defined as 's' and so forth
> so that it will be easier to update code rather than litter
> it with such character codes.
> 
> Perry 

Good idea.  I'll see if I can get numerix to take care of it.

Todd

[Matplotlib-users] using plotting contexts

[Perry G. <pe...@st...>]

I wonder what the recommended way of saving 
and using different plotting defaults is.

Let me give some background first. While many users 
like the current window color defaults (basically black
on white) some prefer white on black for interactive use
(for one thing, colors stand out more). And while they
may be morons for wanting to do so ;-), there is also
a very good reason to use such a color scheme for viewgraphs.
I don't know how many times I've suffered through a presentation
where the presenter showed the plots having a white background
and colored lines or symbols. In that setting, colors just don't
show well. ("well, you could see the yellow points in my version...")
On the other hand such color schemes don't print well (and
the current one does).

I could see someone wanting 2 or 3 different contexts. The 
current rcParams does allow for different save color faces, 
but that isn't quite enough for cases where text needs to be
white or something other than black to show up against black
bacgrounds.

Is there a means of using alternate rcParams dictionaries for 
use with different backends, say as a parameter to the print_figure
method? It doesn't look like it at the moment. I'd guess to do
that now means replacing rcParams temporarily, rendering the
plot to the file, and resetting rcParams to what it was (I haven't
actually tried it), but that seems clumsier than it should be.

Perry

Re: [Matplotlib-users] using plotting contexts

[John H. <jdh...@ac...>]

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:

    Perry> Is there a means of using alternate rcParams dictionaries
    Perry> for use with different backends, say as a parameter to the
    Perry> print_figure method? It doesn't look like it at the
    Perry> moment. I'd guess to do that now means replacing rcParams
    Perry> temporarily, rendering the plot to the file, and resetting
    Perry> rcParams to what it was (I haven't actually tried it), but
    Perry> that seems clumsier than it should be.

There are a few ways to go here.

Probably the best is based on the answer to "How do I customize the
default behavior of a single figure?"
http://matplotlib.sourceforge.net/faq.html#CUSTOM

Ie, you can set the rc params for a given session by updating the rc
dict before importing matplotlib.  You could set up a module like
plot_contexts 

plot_contexts.py::

from matplotlib import rcParams

def use(s):
    if s == 'powerpoint':
        rcParams['figure.facecolor']   = 'b'
        rcParams['text.fontname']   = 'cmr10'
        rcParams['lines.linewidth'] = 2.0
    elif s = 'some journal'
        and so on


And at the script level you could do

import plot_contexts
plot_contexts.use('powerpoint')
from matplotlib.matlab import *

The one down side of the rcParams approach is that currently all the
object defaults are evaluated at module load time, so you can only
switch at the start of your python session.  Eg, the current
constructors look like

class Line2D(Artist):
    def __init__(self, xdata, ydata,
                 linewidth=rcParams['lines.linewidth'],
                 linestyle=rcParams['lines.linestyle'],
                 )

By changing all the constructor defaults to 

    def __init__(self, xdata, ydata,
                 linewidth=None,
                 linestyle=None,
                 ):
                 if linewidth is None: linewidth = rcParams['lines.linewidth'],
                 if linestyle is None: linestyle = rcParams['lines.linestyle'],

we could defer those evaluations to construction time, so you could
switch contexts in the middle of an interactive session

  >>> plot_contexts.use('power point')
  >>> make_some_figs()
  >>> plot_contexts.use('some journal')
  >>> make_some_figs()  # again

I already did this for the FontProperties so you can have context
fonts, as in examples/font_properties_demo.py.  It wouldn't be much
work to do the same for the other properties controlled by rc.

As another approach, I have considered making the matplotlibrc finder
check the current dir first, and using that file if it finds one.
Then you could have dir based contexts.

JDH

RE: [Matplotlib-users] using plotting contexts

[Perry G. <pe...@st...>]

John Hunter wrote:
> >>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
> 
>     Perry> Is there a means of using alternate rcParams dictionaries
>     Perry> for use with different backends, say as a parameter to the
>     Perry> print_figure method? It doesn't look like it at the
>     Perry> moment. I'd guess to do that now means replacing rcParams
>     Perry> temporarily, rendering the plot to the file, and resetting
>     Perry> rcParams to what it was (I haven't actually tried it), but
>     Perry> that seems clumsier than it should be.
> 
> There are a few ways to go here.
> 
> Probably the best is based on the answer to "How do I customize the
> default behavior of a single figure?"
> http://matplotlib.sourceforge.net/faq.html#CUSTOM
> 
I had looked at that. I'm a little confused by the terminology though.
This sounds like a prescription for changing the defaults for some
parameters for a session rather than for a single figure. Do I
misread that? (Also, on my browser these entries are much wider
than the browser window for some reason; later in the faq that changes
as do some of the fonts. I wonder if something is screwed up there).

[details about setting up session context and object intialization
snipped]

I'm thinking of the case where someone tailors plots interactively
and then wants to render the same plot, but with different defaults
for a png file or postscript in the same session. In that case
a session change of default isn't very useful. They basically are
going to be force to write a script, and run it from a different
session to get a different version. So I think that is going
to be somewhat inconvenient for some users.

Changing the use of the rcParams to be immediate rather than locked
in at module initialization is more attractive to me (and may allow
one to assign differnt rc sets to different backends by default as
well; but that is not urgent). Do you mind changing when the 
parameters are picked up? (I'm asking that you do it now, just
whether that is acceptable).

Thanks, Perry

Re: [Matplotlib-users] using plotting contexts

[John H. <jdh...@ac...>]

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:

    Perry> I had looked at that. I'm a little confused by the
    Perry> terminology though.  This sounds like a prescription for
    Perry> changing the defaults for some parameters for a session
    Perry> rather than for a single figure. Do I misread that? (Also,
    Perry> on my browser these entries are much wider than the browser
    Perry> window for some reason; later in the faq that changes as do
    Perry> some of the fonts. I wonder if something is screwed up
    Perry> there).

You were rightfully confused.  It said figure when it should say
session.  I mainly work from scripts, with one figure per script, so I
was just munging them in my mind.  I made a brief update to the FAQ in
my own tree, but with the dynamic customization you discuss below,
this will need to be updated again.

As for the browser problem, I have the same.  I tried to spot an html
error that would explain it but haven't found it yet.

    Perry> Changing the use of the rcParams to be immediate rather
    Perry> than locked in at module initialization is more attractive
    Perry> to me (and may allow one to assign differnt rc sets to
    Perry> different backends by default as well; but that is not
    Perry> urgent). Do you mind changing when the parameters are
    Perry> picked up? (I'm asking that you do it now, just whether
    Perry> that is acceptable).

This seems like the way to go.  Nothing really speaks against it
except the slightly inelegant use of a bunch of Nones in the
constructors.  I'm happy to make the changes, or leave it to one of
you.

JDH

Re: [Matplotlib-users] numerix on win32

[John H. <jdh...@ac...>]

>>>>> "Todd" == Todd Miller <jm...@st...> writes:

    Todd> (2) I could just build a numarray version of the matplotlob
    Todd> installer with each release.  That would offload the work
    Todd> from you and would keep the matplotlib code cleaner.

If you want to do that I have no objections - I like this better than
maintaining both in a single build.  I think we need to make the
matplotlib code work with either build (eg using the nx.Type scheme
you proposed) so that if the user switches their numerix setting with
a given build everything still works.  But if there is interest in a
numarray build for win32, I see no reason why not.  We should probably
be careful to coordinate so that the two releases are as similar as
possible, with the only difference being the default numerix setting
in the rc file.

JDH