matplotlib-devel — matplotlib developers

Re: [matplotlib-devel] Several patches: kwargs cleanup, legend-kwargs, legend-fontsize

[Norbert N. <Nor...@gm...>]

Am Dienstag, 23. November 2004 22:07 schrieb John Hunter:
> >>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:
>
>     Fernando> Quick heads-up:
>
>     Fernando> Python 2.2 dicts do NOT have a pop() method, this
>     Fernando> appeared in python 2.3. It's fine if you decide to make
>     Fernando> matplotlib fully 2.3-dependent, but I just wanted to
>     Fernando> make you at least aware of this fact.  2.2 is still in
>     Fernando> reasonably wide use in the field, so if you are going to
>     Fernando> drop support for it, you might want to put a big fat
>     Fernando> warning about this fact, just to save users the hassle
>     Fernando> of weird bugs.
>
> Ahh, thanks for your vigilance.  I do plan to continue to support 2.2
> as long as possible, except for the known issues: datetime and
> mathtext.  I reverted all the pops to gets.  I also made some
> enhancements to backend_driver to better work with python2.2 - it now
> takes the python to run as a parameter and skips tests where 2.2 is
> already known to fail.
>
> Hopefully, I'll remember to be vigilant to test 2.2 before each
> release.

Ouch, sorry, I do not even have 2.2 installed. Guess with that information, 
the whole patch becomes bogus.

On the other hand: simpy reverting pop to get leaves you with the old problem: 
you have to drop the check for emptyness of kwargs since legal arguments are 
not removed after use. If, on the other hand, you remove this check, 
erraneous (p.e. misspelled) arguments are just silently ignored.

Maybe, a wrapper would solve the problem? How would one code a replacement for 
pop that works on 2.2 as well? Probably easiest by using get and del within a 
try..except block? The wrapper could then have a clear note how to replace it 
once 2.3 becomes mandatory sometimes in the future.

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Steve C. <ste...@ya...>]

On Tue, 2004-11-23 at 15:28 -0600, John Hunter wrote:
>     Steve>  - for some errors I needed to display a matplotlib message
>     Steve> rather than the default exception message, or to raise an
>     Steve> exception where error_msg () was used with no exception. I
>     Steve> added an 'MPLError' exception, its probably best to move it
>     Steve> into a central file if other people need to use it also.
> 
> Perhaps in matplotlib.__init__ ?  
OK, I've added "MPLError' a subclass of Exception to matplotlib.__init__

Steve

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:

    Steve> I've updated backend_gtk.py in cvs to use a default
    Steve> exception handler, and noticed a few things in the process:
    Steve> - sys.excepthook does not catch SystemExit, which is what
    Steve> we wanted anyway.

For some reason with matplotlib CVS using backend GTK on linux, I no
longer recover the linux shell when I close the figure by clicking on
the 'x' in the figure window

  > python somefile.py

I have to use CTRL-C.

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Jochen V. <vo...@se...>]

Hello,

On Tue, Nov 23, 2004 at 03:38:57PM -0600, John Hunter wrote:
> Perhaps I'm missing the obvious thing and not understanding your
> suggestion.  But form my end at the level of the matlab interface, the
> best we can do is get a helpful, descriptive error message to the
> user.  Did you have another approach in mind?
No, I am not very clear about anything here and was just
throwing ideas around.  Actually I am happy with the plans
as far as the PostScript backend is concerned (reporting everyting
as exceptions etc.)  So let's just try what you suggested.

All the best,
Jochen
--=20
http://seehuhn.de/

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Jochen" == Jochen Voss <vo...@se...> writes:

    Jochen> I think we should somehow explicitely catch all exceptions
    Jochen> we are expecting (maybe near the main loop) and the
    Jochen> exception handler could feed them to the backend.

    Jochen> Reasons:

    Jochen> 1) This looks like the most Pythonic solution to me:
    Jochen> exceptions are ment to by caught by "try: ... except: ..."
    Jochen> statements

    Jochen> 2) This might open a way to differentiate between expected
    Jochen> exections (opening a file with a user-supplied file name
    Jochen> might fail, if the user mistyped the name -> User should
    Jochen> be told and asked for another file name) and unexpected
    Jochen> ones which indicate programming errors.

    Jochen> 3) Easier to understand flow of control.  If you follow a
    Jochen> chain of callers through the source code you can see which
    Jochen> exections are caught at which place, whereas the
    Jochen> except_hook mechanism is more "magical".

    Jochen> What do you think?

I think in the backend and other matplotlib code where we know how to
handle the exception, this makes sense.  Eg, if backend_gtk gets a
IOError on some file, it can catch it and request a new filename.  We
should catch and handle exceptions where we can.

Buy I'm thinking about the matlab interface.  There are two problems
in matlab.py: 1) There are practically no exceptions that we can
handle at that level so the best we can do it forward it on the user
and 2) there is no single point to plug in a master exception handler.

Consider the canonical matlab interface function

def plot(*args, **kwargs):
    ret =  gca().plot(*args, **kwargs)
    draw_if_interactive()


gca().plot makes a series of deeply nested calls, as does
draw_if_interactive.  With these two calls, a substantial number of
the total methods defined in matplotlib are actually called (format
string parsers, line instantiation, transformations, clipping, gcs,
renderers, font finding, etc, etc....)  There are a lot of potential
exceptions to be thrown, and tracking down all of them would be a big
task.  And then once we catch them what could we do with them?  

The problem is compounded by the fact that the solution would have to
repeated for *every* matlab interface function separately, which looks
like it would lead to a big tangle of unmanageable code across the
whole module.

Perhaps I'm missing the obvious thing and not understanding your
suggestion.  But form my end at the level of the matlab interface, the
best we can do is get a helpful, descriptive error message to the
user.  Did you have another approach in mind?

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:

    Steve> I've updated backend_gtk.py in cvs to use a default
    Steve> exception handler, and noticed a few things in the process:
    Steve> - sys.excepthook does not catch SystemExit, which is what
    Steve> we wanted anyway.

I think the full exception should be printed.  It makes debugging very
hard, otherwise.  In fact, while recently debugging some code, I
commented out the exception handler for this reason.  matplotlib.cbook
provides a method exception_to_str to convert a traceback to a string.

    Steve>  - for some errors I needed to display a matplotlib message
    Steve> rather than the default exception message, or to raise an
    Steve> exception where error_msg () was used with no exception. I
    Steve> added an 'MPLError' exception, its probably best to move it
    Steve> into a central file if other people need to use it also.

Perhaps in matplotlib.__init__ ?  

    Steve>  - changing from error_msg() to raise exception means the
    Steve> rest of the method will not execute, which I hadn't thought
    Steve> about. This is no good for 'get_filename_from_user()' where
    Steve> I want to loop until a file (or Cancel) is selected. So I
    Steve> think GTK still has a need to use of a popup message dialog
    Steve> occasionally.  And for print_figure() it means the section
    Steve> of code to restore figure settings will not get executed
    Steve> after an error.

I agree that a general facility to popup a message is useful How
abouts we follow Jochen's suggestion and make is a FigureManager
method?

As for your get_filename / restore figure problems, can't these be
addressed by try/catching the exceptions locally, and restoring the
figure state before forwarding the exception onwards?

JDH

Re: [matplotlib-devel] Several patches: kwargs cleanup, legend-kwargs, legend-fontsize

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Quick heads-up:

    Fernando> Python 2.2 dicts do NOT have a pop() method, this
    Fernando> appeared in python 2.3. It's fine if you decide to make
    Fernando> matplotlib fully 2.3-dependent, but I just wanted to
    Fernando> make you at least aware of this fact.  2.2 is still in
    Fernando> reasonably wide use in the field, so if you are going to
    Fernando> drop support for it, you might want to put a big fat
    Fernando> warning about this fact, just to save users the hassle
    Fernando> of weird bugs.

Ahh, thanks for your vigilance.  I do plan to continue to support 2.2
as long as possible, except for the known issues: datetime and
mathtext.  I reverted all the pops to gets.  I also made some
enhancements to backend_driver to better work with python2.2 - it now
takes the python to run as a parameter and skips tests where 2.2 is
already known to fail.

Hopefully, I'll remember to be vigilant to test 2.2 before each
release.

JDH

Re: [matplotlib-devel] Several patches: kwargs cleanup, legend-kwargs, legend-fontsize

[Fernando P. <Fer...@co...>]

John Hunter schrieb:
>>>>>>"Norbert" == Norbert Nemec <Nor...@gm...> writes:
> 
> 
>     Norbert> Hi there, again a few patches, mostly independent but
>     Norbert> somewhat interconnected:
> 
> Hi Norbert, thanks for the patches.  I applied these, but used a
> somewhat different strategy for legend kwargs.  Details below.
> 
>     Norbert> axes-kwargs: a general cleanup of the code in axes.py -
>     Norbert> in several places, kwargs was handled inconsistently,
>     Norbert> ignoring surplus arguments. I went through all uses of
>     Norbert> kwargs and made sure that every argument is either used
>     Norbert> or reported as error
> 
> There was a bug in the pcolor patch of the axes kwargs
> 
>         alpha = kwargs.pop('alpha', 1.0)
>         norm = kwargs.pop('norm')
>         cmap = kwargs.pop('cmap')        
>         vmin = kwargs.pop('vmin')
>         vmax = kwargs.pop('vmax')        
> 
> pop doesn't return a default value like get does, so if you want to
> default to None (which we do here) you must explicitly return it
> 
>         norm = kwargs.pop('norm', None)
>         cmap = kwargs.pop('cmap', None)        
> 
>   ...etc...

Quick heads-up:

planck[~]> ip
Python 2.3.3 (#1, May  7 2004, 10:31:40)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.5.cvs -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: a={1:2,3:4}

In [2]: a.p
a.pop      a.popitem

BUT (note python version number below):

haar[~]> ip
Python 2.2.3 (#1, Oct 15 2003, 23:33:35)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.5.cvs -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: a={1:2,3:4}

In [2]: a.popitem


Python 2.2 dicts do NOT have a pop() method, this appeared in python 2.3. 
It's fine if you decide to make matplotlib fully 2.3-dependent, but I just 
wanted to make you at least aware of this fact.  2.2 is still in reasonably 
wide use in the field, so if you are going to drop support for it, you might 
want to put a big fat warning about this fact, just to save users the hassle 
of weird bugs.

Best,

f

Re: [matplotlib-devel] Several patches: kwargs cleanup, legend-kwargs, legend-fontsize

[John H. <jdh...@ac...>]

>>>>> "Norbert" == Norbert Nemec <Nor...@gm...> writes:

    Norbert> Hi there, again a few patches, mostly independent but
    Norbert> somewhat interconnected:

Hi Norbert, thanks for the patches.  I applied these, but used a
somewhat different strategy for legend kwargs.  Details below.

    Norbert> axes-kwargs: a general cleanup of the code in axes.py -
    Norbert> in several places, kwargs was handled inconsistently,
    Norbert> ignoring surplus arguments. I went through all uses of
    Norbert> kwargs and made sure that every argument is either used
    Norbert> or reported as error

There was a bug in the pcolor patch of the axes kwargs

        alpha = kwargs.pop('alpha', 1.0)
        norm = kwargs.pop('norm')
        cmap = kwargs.pop('cmap')        
        vmin = kwargs.pop('vmin')
        vmax = kwargs.pop('vmax')        

pop doesn't return a default value like get does, so if you want to
default to None (which we do here) you must explicitly return it

        norm = kwargs.pop('norm', None)
        cmap = kwargs.pop('cmap', None)        

  ...etc...

I only caught this bug when I ran examples/backend_driver.py, which
runs many test scripts across many backends, and it is good to make
sure you can run this w/o errors after any non-trivial patch.  Just
wanted to make you aware of it.

    Norbert> legend-fontsize: a tiny hack to make Legend respect the
    Norbert> FONTSIZE parameter correctly. (up to now, it was plainly
    Norbert> ignored)

    Norbert> legend-kwargs: have Legend.__init__ respect a bunch of
    Norbert> kwargs, overriding the global defaults. Also Axes.legend
    Norbert> passes these arguments through correctly

For kwargs, my general approach is to make them explicit in their
final target, eg in this case Legend.__init__.  So rather than do

    def __init__(self, parent, handles, labels, loc, isaxes=True, **kwargs):
        Artist.__init__(self)
        if is_string_like(loc) and not self.codes.has_key(loc):
            verbose.report_error('Unrecognized location %s. Falling back on upper right; valid locations are\n%s\t' %(loc, '\n\t'.join(self.codes.keys())))
        if is_string_like(loc): loc = self.codes.get(loc, 1)
        

        self.NUMPOINTS = kwargs.pop('numpoints',self.NUMPOINTS)
        self.FONTSIZE = kwargs.pop('fontsize',self.FONTSIZE)
        self.PAD = kwargs.pop('pad',self.PAD)
        self.LABELSEP = kwargs.pop('labelsep',self.LABELSEP)
        self.HANDLELEN = kwargs.pop('handlelen',self.HANDLELEN)
        self.HANDLETEXTSEP = kwargs.pop('handletextsep',self.HANDLETEXTSEP)
        self.AXESPAD = kwargs.pop('axespad',self.AXESPAD)
        if len(kwargs):
            raise TypeError, 'Unknown argument "%s"'%kwargs.keys()[0]

I think it's better to do make them explicit

    def __init__(self, parent, handles, labels, loc,
                 isaxes=True,
                 numpoints = 4,      # the number of points in the legend line
                 prop = FontProperties(size='smaller'),
                 pad = 0.2,          # the fractional whitespace inside the legend border
                 # the following dimensions are in axes coords
                 labelsep = 0.005,     # the vertical space between the legend entries
                 handlelen = 0.05,     # the length of the legend lines
                 handletextsep = 0.02, # the space between the legend line and legend text
                 axespad = 0.02,       # the border between the axes and legend edge

                 ):

because then it is self documenting in the python doc string and
clearer to someone trying to grok the code.

I use the **kwargs approach in methods that are simple thin wrappers
of the ultimate target, eg Axes.legend and matplotlib.matlab.legend.

So this is the approach I took in CVS.  I also think I fixed the
fontsize default property issue with the use of prop as a kwarg.

Give it a test drive, and thanks for the patches!

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Jochen V. <vo...@se...>]

Hello John,

On Mon, Nov 22, 2004 at 04:50:10PM -0600, John Hunter wrote:
> So what is the canonical way to funnel exceptions into GUI dialog
> boxes?  Isn't this what sys.except_hook is for?
I think we should somehow explicitely catch all exceptions
we are expecting (maybe near the main loop) and the exception handler
could feed them to the backend.

Reasons:

1) This looks like the most Pythonic solution to me: exceptions
  are ment to by caught by "try: ... except: ..." statements

2) This might open a way to differentiate between expected exections
  (opening a file with a user-supplied file name might fail, if the
  user mistyped the name -> User should be told and asked for another
  file name) and unexpected ones which indicate programming errors.

3) Easier to understand flow of control.  If you follow a chain of callers
  through the source code you can see which exections are caught at which
  place, whereas the except_hook mechanism is more "magical".

What do you think?

Jochen
--=20
http://seehuhn.de/

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Steve C. <ste...@ya...>]

I've updated backend_gtk.py in cvs to use a default exception handler,
and noticed a few things in the process:
 - sys.excepthook does not catch SystemExit, which is what we wanted
anyway.

 - for some errors I needed to display a matplotlib message rather than
the default exception message, or to raise an exception where error_msg
() was used with no exception. I added an 'MPLError' exception, its
probably best to move it into a central file if other people need to use
it also.

 - changing from error_msg() to raise exception means the rest of the
method will not execute, which I hadn't thought about. This is no good
for 'get_filename_from_user()' where I want to loop until a file (or
Cancel) is selected. So I think GTK still has a need to use of a popup
message dialog occasionally.
And for print_figure() it means the section of code to restore figure
settings will not get executed after an error.

Steve

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Fernando P. <Fer...@co...>]

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     >> overridden.  I don't see any elegant way to do this.  Any
>     >> ideas?
> 
>     Fernando> In [2]: sys.__excepthook__ is sys.excepthook Out[2]:
> 
> Excellent.  So if the backends define a function excepthook_backend
> and matplotlib.matlab does
> 
> if sys.__excepthook__ is sys.excepthook:
>     sys.excepthook = excepthook_backend
> 
> would we be in reasonably good standing with the python gods and
> others?

You'd have to ask the gods directly, but it looks pretty good to me, a mere 
mortal :)

Best,

f

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    >> overridden.  I don't see any elegant way to do this.  Any
    >> ideas?

    Fernando> In [2]: sys.__excepthook__ is sys.excepthook Out[2]:

Excellent.  So if the backends define a function excepthook_backend
and matplotlib.matlab does

if sys.__excepthook__ is sys.excepthook:
    sys.excepthook = excepthook_backend

would we be in reasonably good standing with the python gods and
others?

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Fernando P. <Fer...@co...>]

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     >> Actually, it would be fine if matplotlib overrode
>     >> sys.except_hook and ipython later came along and overrode that.
>     >> Basically, ipython would be saying, "I know I've got a shell to
>     >> display errors in, so we don't need to GUI method".  I don't
>     >> think matplotlib would have a problem with that.  Ditto for
>     >> envisage.  Basically, we would be providing a default method to
>     >> get the message to the GUI which could be overriden by other
>     >> applications that want to (ipython, envisage, what-have-you).
> 
>     Fernando> I guess this is a good heads-up for me.  I know ipython
>     Fernando> does some of this, I'll just add a bit more such
>     Fernando> control.  That way, if during the running of user code
>     Fernando> they need sys.ehook for something, they'll get it.  But
>     Fernando> ipython will keep it for when it needs it.  I'm pretty
>     Fernando> sure the embeddable ipython has such control in it, I
>     Fernando> may just need to put it in the general case as well.
> 
>     Fernando> Ultimately I'm not 100% sure what a good solution for
>     Fernando> matplotlib is, I just wanted to make you aware of these
>     Fernando> issues, so that at least they are on your radar.
> 
> 
> We do have some more options.  For one, we could use the excepthook
> only in the matlab interface -- in this case matlab is being used more
> as an application rather than a library.  Folks using matplotlib as a
> library, eg embedding in a GUI, would be advised to do their own
> trapping.
> 
> The only exceptional case I see is basically the ipython (and friends)
> case.  Ie, someone wants to write a shell or otherwise that embeds
> matplotlib.matlab.  In this case it would be fine to override
> matplotlib.matlab's excepthook, as discussed.
> 
> To play really nicely, matplotlib.matlab would like to be able to
> override excepthook only if it hadn't been otherwise overridden.  I
> don't see any elegant way to do this.  Any ideas?

Yup:

planck[mayavi]> ip
In [1]: import sys

In [2]: sys.__excepthook__ is sys.excepthook
Out[2]: False

planck[mayavi]> python
 >>> import sys
 >>> sys.__excepthook__ is sys.excepthook
True

Cheers,

f

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    >> Actually, it would be fine if matplotlib overrode
    >> sys.except_hook and ipython later came along and overrode that.
    >> Basically, ipython would be saying, "I know I've got a shell to
    >> display errors in, so we don't need to GUI method".  I don't
    >> think matplotlib would have a problem with that.  Ditto for
    >> envisage.  Basically, we would be providing a default method to
    >> get the message to the GUI which could be overriden by other
    >> applications that want to (ipython, envisage, what-have-you).

    Fernando> I guess this is a good heads-up for me.  I know ipython
    Fernando> does some of this, I'll just add a bit more such
    Fernando> control.  That way, if during the running of user code
    Fernando> they need sys.ehook for something, they'll get it.  But
    Fernando> ipython will keep it for when it needs it.  I'm pretty
    Fernando> sure the embeddable ipython has such control in it, I
    Fernando> may just need to put it in the general case as well.

    Fernando> Ultimately I'm not 100% sure what a good solution for
    Fernando> matplotlib is, I just wanted to make you aware of these
    Fernando> issues, so that at least they are on your radar.


We do have some more options.  For one, we could use the excepthook
only in the matlab interface -- in this case matlab is being used more
as an application rather than a library.  Folks using matplotlib as a
library, eg embedding in a GUI, would be advised to do their own
trapping.

The only exceptional case I see is basically the ipython (and friends)
case.  Ie, someone wants to write a shell or otherwise that embeds
matplotlib.matlab.  In this case it would be fine to override
matplotlib.matlab's excepthook, as discussed.

To play really nicely, matplotlib.matlab would like to be able to
override excepthook only if it hadn't been otherwise overridden.  I
don't see any elegant way to do this.  Any ideas?

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Fernando P. <Fer...@co...>]

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
> 
>     Fernando> I realize that excepthook is a tempting tool to use, but
>     Fernando> I hope you guys reconsider this.  I really think it
>     Fernando> would cause many more headaches down the road than those
>     Fernando> it initially appears to solve.
> 
> OK, good to know.  That was news to me.  Now why is it that ipython
> and envisage get to mess around with it and we don't :-) ?
> 
> So what is the canonical way to funnel exceptions into GUI dialog
> boxes?  Isn't this what sys.except_hook is for?

Mmh, I don't really know for sure.  But here's a quick test:

planck[mayavi]> egrep -r sys.except *
planck[mayavi]>

This is on the mayavi sources.  Mayavi DOES show all VTK exceptions into a GUI 
(such as you get when you try to open the volume rendering module with 
floating point data, for example).  I imagine it just traps them locally, but 
I'm not sure.  It is possible that mayavi has an easier job because it's done 
in Tk, with no threading issues to worry about.  Threads make this problem 
MUCH worse, since python has no sensible way for an exception raised in one 
thread to be handled by another: all go to the sys.excepthook bucket.  There 
have been long, complicated threads recently on c.l.py on the cousin topics of 
signals and exceptions in threads, and it doesn't look pretty.

> Actually, it would be fine if matplotlib overrode sys.except_hook and
> ipython later came along and overrode that.  Basically, ipython would
> be saying, "I know I've got a shell to display errors in, so we don't
> need to GUI method".  I don't think matplotlib would have a problem
> with that.  Ditto for envisage.  Basically, we would be providing a
> default method to get the message to the GUI which could be overriden
> by other applications that want to (ipython, envisage, what-have-you).

I guess this is a good heads-up for me.  I know ipython does some of this, 
I'll just add a bit more such control.  That way, if during the running of 
user code they need sys.ehook for something, they'll get it.  But ipython will 
keep it for when it needs it.  I'm pretty sure the embeddable ipython has such 
control in it, I may just need to put it in the general case as well.

Ultimately I'm not 100% sure what a good solution for matplotlib is, I just 
wanted to make you aware of these issues, so that at least they are on your radar.

Best,

f

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:


    Fernando> I realize that excepthook is a tempting tool to use, but
    Fernando> I hope you guys reconsider this.  I really think it
    Fernando> would cause many more headaches down the road than those
    Fernando> it initially appears to solve.

OK, good to know.  That was news to me.  Now why is it that ipython
and envisage get to mess around with it and we don't :-) ?

So what is the canonical way to funnel exceptions into GUI dialog
boxes?  Isn't this what sys.except_hook is for?

Actually, it would be fine if matplotlib overrode sys.except_hook and
ipython later came along and overrode that.  Basically, ipython would
be saying, "I know I've got a shell to display errors in, so we don't
need to GUI method".  I don't think matplotlib would have a problem
with that.  Ditto for envisage.  Basically, we would be providing a
default method to get the message to the GUI which could be overriden
by other applications that want to (ipython, envisage, what-have-you).

So I don't really see a danger here, but please educate me!

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Fernando P. <Fer...@co...>]

John Hunter schrieb:
>>>>>>"Steve" == Steve Chaplin <ste...@ya...> writes:
> 
> 
>     Steve> By subclassing Verbose you group all the functions that
>     Steve> display messages to the user together into one class.  I
>     Steve> may have misunderstood some of the discussion - I agree
>     Steve> with doing all error handling with exceptions, but when you
>     Steve> catch the exception in a GUI I'm assuming you still want to
>     Steve> popup a message to inform the user.  Are you describing
>     Steve> using exceptions and tracebacks without any error messages
>     Steve> for GUI backends? I don't think you can assume a GUI
>     Steve> backend user will see a traceback since the terminal window
>     Steve> may be obscured, iconified or even closed.
> 
> For the GUI error handling, I was assuming we would use the
> except_hook, and do away with error reporting in the verbose class.

I'd like to strongly plead that you stay away from sys.excepthook.

That is the 'last resort' tool to manipulate exceptions, and it's typically 
used by frameworks which need to completely control the python process.  For 
example, ipython puts its internal crash handler in sys.excepthook, so that if 
all else fails, the crash handler generates a very detailed crash report.  But 
all 'normal' exception handling is done by the internal user loop, with manual 
control.

I haven't had time to look in detail, but I even think that ipython pretty 
aggressively reclaims sys.excepthook if user code messes with it.  I wouldn't 
be surprised if other frameworks (like envisage) also used sys.excepthook 
themselves.  Matplotlib is 'only' a plotting library :), and it should IMHO 
play nicely with other code running along with it.  If it gets into a 
foodfight over who owns sys.excepthook, or if it crashes because 
sys.excepthook is not what it thinks it is, I expect serious interoperability 
problems to pop up down the road.

I realize that excepthook is a tempting tool to use, but I hope you guys 
reconsider this.  I really think it would cause many more headaches down the 
road than those it initially appears to solve.

Regards,

f

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:

    Steve> By subclassing Verbose you group all the functions that
    Steve> display messages to the user together into one class.  I
    Steve> may have misunderstood some of the discussion - I agree
    Steve> with doing all error handling with exceptions, but when you
    Steve> catch the exception in a GUI I'm assuming you still want to
    Steve> popup a message to inform the user.  Are you describing
    Steve> using exceptions and tracebacks without any error messages
    Steve> for GUI backends? I don't think you can assume a GUI
    Steve> backend user will see a traceback since the terminal window
    Steve> may be obscured, iconified or even closed.

For the GUI error handling, I was assuming we would use the
except_hook, and do away with error reporting in the verbose class.

For the regular verbose reporting, I'm not averse to plugging it into
a GUI dialog, but I'm not sure this is useful.  As I wrote before,
this will mainly be used in debug situations when we can probably
assume a user has access to a shell output.  They can always capture
it to a file using the regular rc mechanism of setting verbose.fileo.

If we did want to do it in a GUI, we would have to be fairly careful
about the implementation, so that the GUI cached sequential methods
and only displayed them if some time interval (eg 100ms) had lapsed
with no new messages.  This would be used to prevent the curse of 20
popups.  This could presumably be done in GTK with an idle handler and
a changed timestamp.

Alternatively, it could be done in the verbose base class itself,
implemented using threads, but I'm a little wary of the extra
complexity here.  

That said, I think that having the figure manager define a
popup_dialog method would be generally useful.  

    Steve> It does look like they are all non-fatal. I guess the fatal
    Steve> ones are the ones that matplotlib does not anticipate or
    Steve> catch, thinks like faulty installations, missing libraries
    Steve> etc. At the moment these would cause matplotlib to
    Steve> terminate but if we add a default exception handler we will
    Steve> start catching these also.  We could have a policy for
    Steve> matplotlib to catch all exceptions and always attempt to
    Steve> continue, and if it becomes unusable its up to the user to
    Steve> close the window. Also we could recognise some situations
    Steve> (if there are any) where we need to terminate, so we raise
    Steve> SystemExit and set the default exception handler to
    Steve> terminate on SystemExit and continue on all other cases.

Sounds right to me....  So in summary, if all agree and we've covered
all the bases, Verbose.report_error and error_msg disappear and are
replaced by regular exceptions.  matplotlib code can raise a
SystemExit for the relatively rare fatal errors.  GUIs define
sys.excepthook = exception_handler following the lead of Steve's
implementation in backend_gtk (in CVS).

verbose.report is left untouched for now but may be hooked into a GUI
reporting functionality if we can resolve the issues of whether this
is desirable and how to handle caching of many independent sequential
messages.

JDH

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Steve C. <ste...@ya...>]

On Sat, 2004-11-20 at 17:26 -0600, John Hunter wrote:
> As for report_error, subclassing Verbose, or using figure manager for
> this as Jochen has suggested, are both workable solutions, but what
> does it ultimately buy us?  I am inclined to the logically cleaner
> solution of doing all error handling with exceptions, using a hook
> like you've provided for GUIs.
By subclassing Verbose you group all the functions that display messages
to the user together into one class.
I may have misunderstood some of the discussion - I agree with doing all
error handling with exceptions, but when you catch the exception in a
GUI I'm assuming you still want to popup a message to inform the user.
Are you describing using exceptions and tracebacks without any error
messages for GUI backends? I don't think you can assume a GUI backend
user will see a traceback since the terminal window may be obscured,
iconified or even closed.

> I grepped for all the current uses of report error (included below) --
> on quick inspection none of these appear fatal for a GUI.  I think
> simply informing the user of the error may suffice.  Can you provide
> an example of where we may need to exit (and would it suffice for the
> raiser to simply raise a SystemExit for this case?)

It does look like they are all non-fatal. I guess the fatal ones are the
ones that matplotlib does not anticipate or catch, thinks like faulty
installations, missing libraries etc. At the moment these would cause
matplotlib to terminate but if we add a default exception handler we
will start catching these also.
We could have a policy for matplotlib to catch all exceptions and always
attempt to continue, and if it becomes unusable its up to the user to
close the window. Also we could recognise some situations (if there are
any) where we need to terminate, so we raise SystemExit and set the
default exception handler to terminate on SystemExit and continue on all
other cases.

Steve

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:

    Steve> I was thinking of something like:

    Steve> class VerboseGTK(Verbose): def report_error(self, s):
    Steve> dialog = gtk.MessageDialog( parent = None, type =
    Steve> gtk.MESSAGE_ERROR, buttons = gtk.BUTTONS_OK, message_format
    Steve> = msg) dialog.run() dialog.destroy()

    Steve> So that the matlab interface can call
    Steve> verbose.report_error() and for image backends it writes to
    Steve> stdout and for GUI backends it pops up a message dialog.

    Steve> You can hook a GTK dialog into unhandled Python exceptions
    Steve> with: import sys

    Steve> def exception_handler(type, value, tb): """Handle uncaught
    Steve> exceptions""" error_msg_gtk(value)

    Steve> sys.excepthook = exception_handler

    Steve> (I've added this to backend_gtk.py in cvs if you want to
    Steve> try it out)

As for report_error, subclassing Verbose, or using figure manager for
this as Jochen has suggested, are both workable solutions, but what
does it ultimately buy us?  I am inclined to the logically cleaner
solution of doing all error handling with exceptions, using a hook
like you've provided for GUIs.

The only lingering advantage I see for a report_error call w/o an
exception being raised is it presents a cleaner error message, which
is nice for newbies.  I have a python 3000-esque design philosophy for
matplotlib -- I want it to be accessible to newbies.  And a simple
message "function blah expects 1 or 2 arguments" is much more likely
to be read and parsed by a newbie, who in my experience will disregard
a traceback simply because it often appears unreadable, until you are
trained to read from the bottom up, which is counter intuitive to
some.

Are there other advantages to report_error that I'm missing, and if
not, does the readability issue justify circumventing the default
exception handling mechanism?  My inclination is that it doesn't.

    Steve> But you still need to decide how to handle the exceptions -
    Steve> with some you need to terminate the program, with others
    Steve> its safe to continue. It may mean you end up writing a
    Steve> complicated generic exception handler that tries to handle
    Steve> every possible exception. In that case handling exceptions
    Steve> individually, the usual way might be better, possibly using
    Steve> the sys.excepthook to handle the remaining uncaught
    Steve> exceptions, or using it when you want to terminate the
    Steve> program and want to popup a message saying "Fatal error..."

I grepped for all the current uses of report error (included below) --
on quick inspection none of these appear fatal for a GUI.  I think
simply informing the user of the error may suffice.  Can you provide
an example of where we may need to exit (and would it suffice for the
raiser to simply raise a SystemExit for this case?)

JDH

'Error: %s'%msg
'Unable to allocate color %1.3f, %1.3f, %1.3f; using nearest neighbor' % rgb
'Error: %s'% msg
'Error: %s' % msg
'Could not load font file "%s"'%fname
'Error: %s'% msg
'Could not load filename for text "%s"'%fname
msg
'Could not find bitmap file "%s"; dying'%bmpFilename
'backend_gtk could not import mathtext (build with ft2font')
'Error: %s' % exc
'The GTK backend cannot draw text at a %i degree angle, try GtkAgg instead' % angle
'mathtext not supported: %s' % exc
"Could not renderer vertical text", s
"cairo.numpy module required for draw_image(")
'Mathtext not implemented yet'
'Unrecognized cap style.  Found %s' % cs
'Unrecognized join style.  Found %s' % js
"%s: %s" % (exc.filename, exc.strerror)
'Format "%s" is not supported.\nSupported formats: %s.' %
./__init__.py:    def report_error(self, s:
'Could not find .matplotlibrc; using defaults'
message
'Illegal line #%d\n\t%s\n\tin file "%s"' % (cnt, line, fname)
'%s is deprecated in .matplotlibrc - use %s instead.' % (key, alt)
'Bad key "%s" on line %d in %s' % (key, cnt, fname)
'Bad val "%s" on line #%d\n\t"%s"\n\tin file "%s"\n\t%s' % (val, cnt, line, fname, msg)
'unrecognized backend %s.\n' % arg +\
./backend_bases.py:    verbose.report_error('Error: %s'% msg
"ColormapJet deprecated, please use cm.jet instead"
"Grayscale deprecated, please use cm.gray instead"
'urlopen( failure\n' + url + '\n' + exc.strerror[1])
"Could not open font file %s"%fpath
"Could not open font file %s"%fpath
 msg % name
'Could not match %s, %s, %s.  Returning %s' % (name, style, variant, self.defaultFont)
'Unrecognized location %s. Falling back on upper right; valid locations are\n%s\t' %(loc, '\n\t'.join(self.codes.keys()))
'Unrecognized line style %s' %( linestyle, type(linestyle))
'Unrecognized marker style %s'%( marker, type(marker))
'unrecognized symbol "%s"' % sym
'unrecognized symbol "%s, %d"' % (sym, num)
'Coherence is calculated by averaging over NFFT length segments.  Your signal is too short for your choice of NFFT'
'Dimension error'
'Second argument not permitted for matrices'
__doc__
'Unrecognized location %s. Falling back on bottom; valid locations are\n%s\t' %(loc, '\n\t'.join(self.codes.keys()))
'AutoLocator illegal dataInterval range %s; returning NullLocator'%d
'Unrecognized location %s. Falling back on upper right; valid locations are\n%s\t' %(loc, '\n\t'.join(self.codes.keys()))


    Steve> Steve





    Steve> -------------------------------------------------------
    Steve> This SF.Net email is sponsored by: InterSystems CACHE FREE
    Steve> OODBMS DOWNLOAD - A multidimensional database that combines
    Steve> robust object and relational technologies, making it a
    Steve> perfect match for Java, C++,COM, XML, ODBC and
    Steve> JDBC. www.intersystems.com/match8
    Steve> _______________________________________________
    Steve> Matplotlib-devel mailing list
    Steve> Mat...@li...
    Steve> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Jochen V. <vo...@se...>]

Hello,

On Sat, Nov 20, 2004 at 11:53:01AM +0800, Steve Chaplin wrote:
> I was thinking of something like:
>=20
> class VerboseGTK(Verbose):
>     def report_error(self, s):
>         dialog =3D gtk.MessageDialog(
>             parent         =3D None,
>             type           =3D gtk.MESSAGE_ERROR,
>             buttons        =3D gtk.BUTTONS_OK,
>             message_format =3D msg)
>         dialog.run()
>         dialog.destroy()
Alternatively we could make report_error a figure_manager method.
If could default to

class FigureManagerBase:
    def report_error(self, s):
	sys.stderr.write("error: %s\n"%s)

And FigureManagerGTK could overload it with the above code to generate
an error box.

Reasons why I would prefer this:

1) I do not like these global variables which are set on module import
  at all.  Using the VerboseGTK idea we would get another instance of this,
  namely something like "currentVerboseClass=3DVerboseBackend" or such.

  We already have something like this for figure managers, so no new instan=
ce
  of this would be created with my suggestion.

  Reporting errors would then work like this:

      manager =3D get_current_fig_manager()
      manager.canvas.report_error(message)

  which could be wrapped into a function.

2) The main functionality of the Verbose class seems to be,
  that the user can select how many messages he wants to see.
  Error messages (at least fatal ones) should be presented to
  the user in any case, so for me reporting errors does not
  look like an application of the Verbose class.

What do you think?

Jochen
--=20
http://seehuhn.de/

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Steve C. <ste...@ya...>]

On Fri, 2004-11-19 at 09:23 -0600, John Hunter wrote:
> I think you are right that the plethora of error reporting strategies
> is causing confusion, especially for me!  I like the idea of the GUI
> backends overriding placing a hook into the python exception handling
> process.  One possibility would be to do away with
> verbose.report_error and error_msg.  The GUIs hook into the exception
> message, and anywhere we want to report an error we raise a python
> exception.  And we continue to use verbose.report as before.
> 
> I just checked backend_ps and the only place is uses error_msg is
> 
>     error_msg_ps('Could not open %s for writing' % outfile)
> 
> which would be more naturally handled as an exception anyway.
> 
> Steve, could you look into hooking a GTK dialog into the python
> exception reporting mechanism to see if this is viable?  In summary,
> the thought is
> 
>   * use verbose only for non-error reporting
> 
>   * use exceptions for all error reporting
> 
>   * work some GUI magic to transparently get the errors forwarded to a
>     dialog box w/o using special functions
> 
> As for verbose.report, I'm not convinced it is a good idea to hook
> this into the GUI.  For one thing, some reporting occurs before the
> backend is determined.  For another, it would also require some
> caching of messages because if 30 messages generate 30 popups it will
> get annoying quick.  These things are manageable, but I think the main
> use for verbose.report is debugging a problem, in which case simply
> having the messages go to a stdout or a file may be the best place for
> them.
> 
> JDH
I was thinking of something like:

class VerboseGTK(Verbose):
    def report_error(self, s):
        dialog = gtk.MessageDialog(
            parent         = None,
            type           = gtk.MESSAGE_ERROR,
            buttons        = gtk.BUTTONS_OK,
            message_format = msg)
        dialog.run()
        dialog.destroy()

So that the matlab interface can call verbose.report_error() and for
image backends it writes to stdout and for GUI backends it pops up
a message dialog.

You can hook a GTK dialog into unhandled Python exceptions with:
import sys

def exception_handler(type, value, tb):
    """Handle uncaught exceptions"""
    error_msg_gtk(value)

sys.excepthook = exception_handler

(I've added this to backend_gtk.py in cvs if you want to try it out)

But you still need to decide how to handle the exceptions - with some
you need to terminate the program, with others its safe to continue. It
may mean you end up writing a complicated generic exception handler that
tries to handle every possible exception. In that case handling
exceptions individually, the usual way might be better, possibly using
the sys.excepthook to handle the remaining uncaught exceptions, or using
it when you want to terminate the program and want to popup a message
saying "Fatal error..."

Steve

Re: [matplotlib-devel] propagation of error messages in matplotlib

[John H. <jdh...@ac...>]

>>>>> "Jochen" == Jochen Voss <vo...@se...> writes:


    Jochen> Slight problem: it might now be a little bit more
    Jochen> difficult to include the name of the file which could not
    Jochen> be opened in the error message.  The IOError exception
    Jochen> will probably only have "permission denied" associated
    Jochen> with it.

Looks OK, at least on linux


>>> file('/sbin/ldconfig', 'w')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
IOError: [Errno 13] Permission denied: '/sbin/ldconfig'

Re: [matplotlib-devel] propagation of error messages in matplotlib

[Jochen V. <vo...@se...>]

Hello,

On Fri, Nov 19, 2004 at 09:23:36AM -0600, John Hunter wrote:
> In summary, the thought is
>=20
>   * use verbose only for non-error reporting
>=20
>   * use exceptions for all error reporting
I agree with this.

>   * work some GUI magic to transparently get the errors forwarded to a
>     dialog box w/o using special functions
Here I have no opinion until I understand the special kind
of magic which is going to be used here.

> For another, it would also require some
> caching of messages because if 30 messages generate 30 popups it will
> get annoying quick.
Yes, we have to be careful here.

As nobody objected to this part of the plan, I changed the PostScript
backend in CVS as follows:

    diff -u -r1.13 backend_ps.py
    --- backend_ps.py       13 Nov 2004 11:41:54 -0000      1.13
    +++ backend_ps.py       19 Nov 2004 16:23:13 -0000
    @@ -529,10 +529,7 @@
		 basename, ext =3D os.path.splitext(outfile)
		 if not ext: outfile +=3D '.ps'
		 isEPSF =3D ext.lower().startswith('.ep')
    -            try:
    -                fh =3D file(outfile, 'w')
    -            except IOError:
    -                error_msg_ps('Could not open %s for writing' % outfile)
    +            fh =3D file(outfile, 'w')
		 needsClose =3D True
		 title =3D outfile

Slight problem: it might now be a little bit more difficult to include
the name of the file which could not be opened in the error message.
The IOError exception will probably only have "permission denied" associated
with it.

All the best,
Jochen
--=20
http://seehuhn.de/