matplotlib

Jouni K. Seppänen wrote:
> The matplotlib.use() function was always a no-op after importing pylab,
> and there were occasional questions about it on the mailing list. It was
> changed fairly recently to raise an error in this case. Bug #1855454
> points out that this is not backward compatible: an application that
> calls use() too late didn't do anything before the change, but now gets
> stopped with a traceback. I suggest changing the error to a warning,
> which I think would still deliver the message but would be compatible
> with (broken) legacy code. What do you think?
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1855454&group_id=80706&atid=560720
> 

Good point.  I probably went overboard when making the original change 
in October, so I have now changed it to a warning as you suggest.

Eric

Porting this back to the trunk is non-trivial -- but I know John would 
like to do it, so I think it will happen once one of us has time.

As an aside, the unit stuff *should* be working on the branch.  Can you 
send me a minimal example that shows it failing?

Cheers,
Mike

James Evans wrote:
> I have taken the transforms branch and played with it a bit and my super simple ellipse test cases appeared to be working great.  I
> couldn't run our more intensive tests since they use unitized data and I was getting errors that looked like the transforms branch
> wasn't completely handling unitized data properly.  It would be great if we could see this fix in the main branch, then we can make
> use of it right away without having to wait for the transforms branch to be completed.
> 
> --James Evans
> 
> 
>> Date: Tue, 11 Dec 2007 15:47:45 -0500
>> From: Michael Droettboom <mdroe@...>
>> Subject: Re: [matplotlib-devel] Problem with Agg Ellipses
>> To: Ted Drain <ted.drain@...>
>> Cc: matplotlib development list
>> 	<matplotlib-devel@...>
>> Message-ID: <475EF771.3090805@...>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> And an actually interesting part of the plot... ;)
>>
>> Michael Droettboom wrote:
>>> Sorry -- correct attachment this time.
>>>
>>> Michael Droettboom wrote:
>>>> I have a working draft of something that may work for this problem on
>>>> the transforms branch.  I am happy to backport this to the trunk, but
>>>> that will require some effort, as the implementation relies on many of
>>>> the new geometric utilities on the branch that would also have to be
>>>> brought over.  It may be some time until the branch is ready for
>>>> production use, but if you are able to use it to experiment with this
>>>> approach to this specific problem, that would certainly make my life
>>>> easier, so I don't have to do the backporting work more than once.
>>>>
>>>> Attached is a plot comparing the new approach (above) vs. a 750-edge
>>>> polygonal approximation for the ellipses (based directly on James
>>>> Evans' example).  Here's a description of what it does:
>>>>
>>>>
>>>>         Ellipses are normally drawn using an approximation that uses
>>>>         eight cubic bezier splines.  The error of this approximation
>>>>         is 1.89818e-6, according to this unverified source:
>>>>
>>>>           Lancaster, Don.  Approximating a Circle or an Ellipse Using
>>>>           Four Bezier Cubic Splines.
>>>>
>>>>           http://www.tinaja.com/glib/ellipse4.pdf
>>>>
>>>>         There is a use case where very large ellipses must be drawn
>>>>         with very high accuracy, and it is too expensive to render the
>>>>         entire ellipse with enough segments (either splines or line
>>>>         segments).  Therefore, in the case where either radius of the
>>>>         ellipse is large enough that the error of the spline
>>>>         approximation will be visible (greater than one pixel offset
>>>>         from the ideal), a different technique is used.
>>>>
>>>>         In that case, only the visible parts of the ellipse are drawn,
>>>>         with each visible arc using a fixed number of spline segments
>>>>         (8).  The algorithm proceeds as follows:
>>>>
>>>>           1. The points where the ellipse intersects the axes bounding
>>>>           box are located.  (This is done be performing an inverse
>>>>           transformation on the axes bbox such that it is relative to
>>>>           the unit circle -- this makes the intersection calculation
>>>>           much easier than doing rotated ellipse intersection
>>>>           directly).
>>>>
>>>>           This uses the "line intersecting a circle" algorithm from:
>>>>
>>>>             Vince, John.  Geometry for Computer Graphics: Formulae,
>>>>             Examples & Proofs.  London: Springer-Verlag, 2005.
>>>>
>>>>           2. The angles of each of the intersection points are
>>>>           calculated.
>>>>
>>>>           3. Proceeding counterclockwise starting in the positive
>>>>           x-direction, each of the visible arc-segments between the
>>>>           pairs of vertices are drawn using the bezier arc
>>>>           approximation technique implemented in Path.arc().
>>>>
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>> Ted Drain wrote:
>>>>> All of these sound like good ideas to me.  Just for some history: the
>>>>> original reason we worked w/ John to get an Ellipse primitive in (vs
>>>>> a normal line plot of sampled points) were to:
>>>>> - improve ellipse plotting speeds (we plot a LOT of them at once)
>>>>> - improve post script output
>>>>>
>>>>> Ted
>>>>>
>>>>> At 08:53 AM 12/10/2007, Michael Droettboom wrote:
>>>>>> John Hunter wrote:
>>>>>>> On Dec 10, 2007 10:25 AM, Ted Drain <ted.drain@...> wrote:
>>>>>>>
>>>>>>>> I don't know if the current MPL architecture can support this but it
>>>>>>>> would be nice if it worked that way.  We have people making decisions
>>>>>>>> based on what these plots show that affect spacecraft worth hundreds
>>>>>>>> of millions of dollars so it's important that we're plotting
>>>>>> things accurately.
>>>>>>> We can support this, but I think we would do this with an arc class
>>>>>>> rather than an ellipse class, and write a special case class that is
>>>>>>> viewlim aware.
>>>>>> I agree -- I think there are two uses cases for ellipse that are in
>>>>>> conflict here.  One is these large ellipses, the other is for things
>>>>>> like scatter plots, where speed and file size is more important than
>>>>>> accuracy.  My mind was probably stuck on the latter as I've worked
>>>>>> along
>>>>>> the transforms branch.
>>>>>>
>>>>>>> A simple example of a line that has analogous
>>>>>>> behavior is examples/clippedline.py, which clips the points outside
>>>>>>> the viewport and draws in a different style according to the
>>>>>>> resolution of the viewlim.   The reason I think it would be preferable
>>>>>>> to use an arc here is because we won't have to worry about filling the
>>>>>>> thing when we only approximate a section of it.  You could feed in a
>>>>>>> 360 degree elliptical arc and then zoom into a portion of it.
>>>>>>>
>>>>>>> With the 8 point ellipse as is, and the addition of an arc class that
>>>>>>> does 4 or 8 point approximation within the zoom limits, should that
>>>>>>> serve your requirements?
>>>>>> As a possible starting point, the transforms branch already has
>>>>>> arc-approximation-by-cubic-bezier-spline code.  It determines the
>>>>>> number
>>>>>> of splines to use based on the radians included in the arc, which is
>>>>>> clearly not what we want here.  But it should be reasonably
>>>>>> straightforward to make that some fixed number and draw the arc between
>>>>>> the edges of the axes.  Or, alternatively, (and maybe this is what John
>>>>>> is suggesting), the arc could be approximated by line segments (with
>>>>>> the
>>>>>> number of segments something like the number of pixels across the
>>>>>> axes).
>>>>>>   To my naive mind, that seems more verifiable -- or at least it puts
>>>>>> the responsibility of getting this right all in one place.
>>>>>>
>>>>>> IMHO, these spline approximation tricks are all just with the aim of
>>>>>> pushing curve rendering deeper into the backends for speed and file
>>>>>> size
>>>>>> improvements.  But obviously there needs to be a way around it when it
>>>>>> matters.
>>>>>>
>>>>>> Cheers,
>>>>>> Mike
>>>>>>
>>>>>> --
>>>>>> Michael Droettboom
>>>>>> Science Software Branch
>>>>>> Operations and Engineering Division
>>>>>> Space Telescope Science Institute
>>>>>> Operated by AURA for NASA
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>>
>>>>>> SF.Net email is sponsored by:
>>>>>> Check out the new SourceForge.net Marketplace.
>>>>>> It's the best place to buy or sell services for
>>>>>> just about anything Open Source.
>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>> _______________________________________________
>>>>>> Matplotlib-devel mailing list
>>>>>> Matplotlib-devel@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>> Ted Drain     Jet Propulsion Laboratory   ted.drain@...
>>>>>
>>>>> -------------------------------------------------------------------------
>>>>>
>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>> Marketplace.
>>>>> It's the best place to buy or sell services for
>>>>> just about anything Open Source.
>>>>> http://sourceforge.net/services/buy/index.php
>>>>> _______________________________________________
>>>>> Matplotlib-devel mailing list
>>>>> Matplotlib-devel@...
>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>> Marketplace.
>>>> It's the best place to buy or sell services for
>>>> just about anything Open Source.
>>>> http://sourceforge.net/services/buy/index.php
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Matplotlib-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Matplotlib-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> --
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: ellipse.png
>> Type: image/png
>> Size: 49913 bytes
>> Desc: not available
>>
>> ------------------------------
>>
>> -------------------------------------------------------------------------
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>>
>> ------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel@...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>> End of Matplotlib-devel Digest, Vol 19, Issue 16
>> ************************************************
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

I have ported this arc stuff to the trunk.  It is in r4783.  Please let 
me know how that works, particularly wrt performance, for you.  Some 
things that are in C on the branch were done in Python on the trunk for 
convenience.

Cheers,
Mike

Michael Droettboom wrote:
> Porting this back to the trunk is non-trivial -- but I know John would 
> like to do it, so I think it will happen once one of us has time.
> 
> As an aside, the unit stuff *should* be working on the branch.  Can you 
> send me a minimal example that shows it failing?
> 
> Cheers,
> Mike
> 
> James Evans wrote:
>> I have taken the transforms branch and played with it a bit and my super simple ellipse test cases appeared to be working great.  I
>> couldn't run our more intensive tests since they use unitized data and I was getting errors that looked like the transforms branch
>> wasn't completely handling unitized data properly.  It would be great if we could see this fix in the main branch, then we can make
>> use of it right away without having to wait for the transforms branch to be completed.
>>
>> --James Evans
>>
>>
>>> Date: Tue, 11 Dec 2007 15:47:45 -0500
>>> From: Michael Droettboom <mdroe@...>
>>> Subject: Re: [matplotlib-devel] Problem with Agg Ellipses
>>> To: Ted Drain <ted.drain@...>
>>> Cc: matplotlib development list
>>> 	<matplotlib-devel@...>
>>> Message-ID: <475EF771.3090805@...>
>>> Content-Type: text/plain; charset="iso-8859-1"
>>>
>>> And an actually interesting part of the plot... ;)
>>>
>>> Michael Droettboom wrote:
>>>> Sorry -- correct attachment this time.
>>>>
>>>> Michael Droettboom wrote:
>>>>> I have a working draft of something that may work for this problem on
>>>>> the transforms branch.  I am happy to backport this to the trunk, but
>>>>> that will require some effort, as the implementation relies on many of
>>>>> the new geometric utilities on the branch that would also have to be
>>>>> brought over.  It may be some time until the branch is ready for
>>>>> production use, but if you are able to use it to experiment with this
>>>>> approach to this specific problem, that would certainly make my life
>>>>> easier, so I don't have to do the backporting work more than once.
>>>>>
>>>>> Attached is a plot comparing the new approach (above) vs. a 750-edge
>>>>> polygonal approximation for the ellipses (based directly on James
>>>>> Evans' example).  Here's a description of what it does:
>>>>>
>>>>>
>>>>>         Ellipses are normally drawn using an approximation that uses
>>>>>         eight cubic bezier splines.  The error of this approximation
>>>>>         is 1.89818e-6, according to this unverified source:
>>>>>
>>>>>           Lancaster, Don.  Approximating a Circle or an Ellipse Using
>>>>>           Four Bezier Cubic Splines.
>>>>>
>>>>>           http://www.tinaja.com/glib/ellipse4.pdf
>>>>>
>>>>>         There is a use case where very large ellipses must be drawn
>>>>>         with very high accuracy, and it is too expensive to render the
>>>>>         entire ellipse with enough segments (either splines or line
>>>>>         segments).  Therefore, in the case where either radius of the
>>>>>         ellipse is large enough that the error of the spline
>>>>>         approximation will be visible (greater than one pixel offset
>>>>>         from the ideal), a different technique is used.
>>>>>
>>>>>         In that case, only the visible parts of the ellipse are drawn,
>>>>>         with each visible arc using a fixed number of spline segments
>>>>>         (8).  The algorithm proceeds as follows:
>>>>>
>>>>>           1. The points where the ellipse intersects the axes bounding
>>>>>           box are located.  (This is done be performing an inverse
>>>>>           transformation on the axes bbox such that it is relative to
>>>>>           the unit circle -- this makes the intersection calculation
>>>>>           much easier than doing rotated ellipse intersection
>>>>>           directly).
>>>>>
>>>>>           This uses the "line intersecting a circle" algorithm from:
>>>>>
>>>>>             Vince, John.  Geometry for Computer Graphics: Formulae,
>>>>>             Examples & Proofs.  London: Springer-Verlag, 2005.
>>>>>
>>>>>           2. The angles of each of the intersection points are
>>>>>           calculated.
>>>>>
>>>>>           3. Proceeding counterclockwise starting in the positive
>>>>>           x-direction, each of the visible arc-segments between the
>>>>>           pairs of vertices are drawn using the bezier arc
>>>>>           approximation technique implemented in Path.arc().
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Mike
>>>>>
>>>>>
>>>>> Ted Drain wrote:
>>>>>> All of these sound like good ideas to me.  Just for some history: the
>>>>>> original reason we worked w/ John to get an Ellipse primitive in (vs
>>>>>> a normal line plot of sampled points) were to:
>>>>>> - improve ellipse plotting speeds (we plot a LOT of them at once)
>>>>>> - improve post script output
>>>>>>
>>>>>> Ted
>>>>>>
>>>>>> At 08:53 AM 12/10/2007, Michael Droettboom wrote:
>>>>>>> John Hunter wrote:
>>>>>>>> On Dec 10, 2007 10:25 AM, Ted Drain <ted.drain@...> wrote:
>>>>>>>>
>>>>>>>>> I don't know if the current MPL architecture can support this but it
>>>>>>>>> would be nice if it worked that way.  We have people making decisions
>>>>>>>>> based on what these plots show that affect spacecraft worth hundreds
>>>>>>>>> of millions of dollars so it's important that we're plotting
>>>>>>> things accurately.
>>>>>>>> We can support this, but I think we would do this with an arc class
>>>>>>>> rather than an ellipse class, and write a special case class that is
>>>>>>>> viewlim aware.
>>>>>>> I agree -- I think there are two uses cases for ellipse that are in
>>>>>>> conflict here.  One is these large ellipses, the other is for things
>>>>>>> like scatter plots, where speed and file size is more important than
>>>>>>> accuracy.  My mind was probably stuck on the latter as I've worked
>>>>>>> along
>>>>>>> the transforms branch.
>>>>>>>
>>>>>>>> A simple example of a line that has analogous
>>>>>>>> behavior is examples/clippedline.py, which clips the points outside
>>>>>>>> the viewport and draws in a different style according to the
>>>>>>>> resolution of the viewlim.   The reason I think it would be preferable
>>>>>>>> to use an arc here is because we won't have to worry about filling the
>>>>>>>> thing when we only approximate a section of it.  You could feed in a
>>>>>>>> 360 degree elliptical arc and then zoom into a portion of it.
>>>>>>>>
>>>>>>>> With the 8 point ellipse as is, and the addition of an arc class that
>>>>>>>> does 4 or 8 point approximation within the zoom limits, should that
>>>>>>>> serve your requirements?
>>>>>>> As a possible starting point, the transforms branch already has
>>>>>>> arc-approximation-by-cubic-bezier-spline code.  It determines the
>>>>>>> number
>>>>>>> of splines to use based on the radians included in the arc, which is
>>>>>>> clearly not what we want here.  But it should be reasonably
>>>>>>> straightforward to make that some fixed number and draw the arc between
>>>>>>> the edges of the axes.  Or, alternatively, (and maybe this is what John
>>>>>>> is suggesting), the arc could be approximated by line segments (with
>>>>>>> the
>>>>>>> number of segments something like the number of pixels across the
>>>>>>> axes).
>>>>>>>   To my naive mind, that seems more verifiable -- or at least it puts
>>>>>>> the responsibility of getting this right all in one place.
>>>>>>>
>>>>>>> IMHO, these spline approximation tricks are all just with the aim of
>>>>>>> pushing curve rendering deeper into the backends for speed and file
>>>>>>> size
>>>>>>> improvements.  But obviously there needs to be a way around it when it
>>>>>>> matters.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Mike
>>>>>>>
>>>>>>> --
>>>>>>> Michael Droettboom
>>>>>>> Science Software Branch
>>>>>>> Operations and Engineering Division
>>>>>>> Space Telescope Science Institute
>>>>>>> Operated by AURA for NASA
>>>>>>>
>>>>>>> -------------------------------------------------------------------------
>>>>>>>
>>>>>>> SF.Net email is sponsored by:
>>>>>>> Check out the new SourceForge.net Marketplace.
>>>>>>> It's the best place to buy or sell services for
>>>>>>> just about anything Open Source.
>>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>>> _______________________________________________
>>>>>>> Matplotlib-devel mailing list
>>>>>>> Matplotlib-devel@...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>>> Ted Drain     Jet Propulsion Laboratory   ted.drain@...
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>>
>>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>>> Marketplace.
>>>>>> It's the best place to buy or sell services for
>>>>>> just about anything Open Source.
>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>> _______________________________________________
>>>>>> Matplotlib-devel mailing list
>>>>>> Matplotlib-devel@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> -------------------------------------------------------------------------
>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>> Marketplace.
>>>>> It's the best place to buy or sell services for
>>>>> just about anything Open Source.
>>>>> http://sourceforge.net/services/buy/index.php
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Matplotlib-devel mailing list
>>>>> Matplotlib-devel@...
>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> SF.Net email is sponsored by:
>>>> Check out the new SourceForge.net Marketplace.
>>>> It's the best place to buy or sell services for
>>>> just about anything Open Source.
>>>> http://sourceforge.net/services/buy/index.php
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Matplotlib-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>> --
>>> Michael Droettboom
>>> Science Software Branch
>>> Operations and Engineering Division
>>> Space Telescope Science Institute
>>> Operated by AURA for NASA
>>> -------------- next part --------------
>>> A non-text attachment was scrubbed...
>>> Name: ellipse.png
>>> Type: image/png
>>> Size: 49913 bytes
>>> Desc: not available
>>>
>>> ------------------------------
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Matplotlib-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>
>>>
>>> End of Matplotlib-devel Digest, Vol 19, Issue 16
>>> ************************************************
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel@...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Dec 21, 2007 9:11 AM, Michael Droettboom <mdroe@...> wrote:

> > Porting this back to the trunk is non-trivial -- but I know John would
> > like to do it, so I think it will happen once one of us has time.

> I have ported this arc stuff to the trunk.  It is in r4783.  Please let
> me know how that works, particularly wrt performance, for you.  Some
> things that are in C on the branch were done in Python on the trunk for
> convenience.

Awesome -- thanks again Michael.  You keep stealing my holiday projects :-)

I added a check to make sure the 'axes' instance was present at draw
time, since patches can be added directly to the figure also, and
noted in the docstring that this is an axes only object.

I also added unit support, which is fairly trivial, and added Arc to
the unit/ellipse_large.py test case and
examples/unit/ellipse_with_units.py

Great work!
JDH

Everyone,
I just wanted to say thanks for tackling this problem so 
quickly.  It's great to see that problems can be worked out like this 
and we really appreciate it.

Here is some background on what we're using this for:  The current 
NASA mission to Mars (Phoenix) will be landing at the end of 
May.  They normally view their current trajectory and associated 
uncertainty as a plot of an ellipse projected on a plane that extends 
out from the center of the body (with time to impact the plane being 
another parameter).  The spacecraft has to hit a very narrow entry 
flight path angle corridor at the atmosphere - too steep and it burns 
up, too shallow and it skips out.  When this corridor is projected 
onto the plane mentioned above, it forms a large circle around 
Mars.  So those large circular plots we're making are the safe entry 
corridor that the trajectory needs to hit and the smaller ellipses 
are the possible trajectory entry points and uncertainties.  As you 
can imagine, it's fairly critical to get those ellipses plotted accurately :)

Thanks again,
Ted

At 08:23 AM 12/21/2007, John Hunter wrote:
>On Dec 21, 2007 9:11 AM, Michael Droettboom <mdroe@...> wrote:
>
> > > Porting this back to the trunk is non-trivial -- but I know John would
> > > like to do it, so I think it will happen once one of us has time.
>
> > I have ported this arc stuff to the trunk.  It is in r4783.  Please let
> > me know how that works, particularly wrt performance, for you.  Some
> > things that are in C on the branch were done in Python on the trunk for
> > convenience.
>
>Awesome -- thanks again Michael.  You keep stealing my holiday projects :-)
>
>I added a check to make sure the 'axes' instance was present at draw
>time, since patches can be added directly to the figure also, and
>noted in the docstring that this is an axes only object.
>
>I also added unit support, which is fairly trivial, and added Arc to
>the unit/ellipse_large.py test case and
>examples/unit/ellipse_with_units.py
>
>Great work!
>JDH
>
>-------------------------------------------------------------------------
>This SF.net email is sponsored by: Microsoft
>Defy all challenges. Microsoft(R) Visual Studio 2005.
>http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>_______________________________________________
>Matplotlib-devel mailing list
>Matplotlib-devel@...
>https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Ok -- this goes without saying, but...  I've only tested this with the 
toy examples you guys have sent etc.  Please verify the results on real 
data before relying on it.

Anyway, it's great to have contributed even such a small part to such a 
great undertaking!

Cheers,
Mike

Ted Drain wrote:
> Everyone,
> I just wanted to say thanks for tackling this problem so quickly.  It's 
> great to see that problems can be worked out like this and we really 
> appreciate it.
> 
> Here is some background on what we're using this for:  The current NASA 
> mission to Mars (Phoenix) will be landing at the end of May.  They 
> normally view their current trajectory and associated uncertainty as a 
> plot of an ellipse projected on a plane that extends out from the center 
> of the body (with time to impact the plane being another parameter).  
> The spacecraft has to hit a very narrow entry flight path angle corridor 
> at the atmosphere - too steep and it burns up, too shallow and it skips 
> out.  When this corridor is projected onto the plane mentioned above, it 
> forms a large circle around Mars.  So those large circular plots we're 
> making are the safe entry corridor that the trajectory needs to hit and 
> the smaller ellipses are the possible trajectory entry points and 
> uncertainties.  As you can imagine, it's fairly critical to get those 
> ellipses plotted accurately :)
> 
> Thanks again,
> Ted
> 
> At 08:23 AM 12/21/2007, John Hunter wrote:
>> On Dec 21, 2007 9:11 AM, Michael Droettboom <mdroe@...> wrote:
>>
>> > > Porting this back to the trunk is non-trivial -- but I know John would
>> > > like to do it, so I think it will happen once one of us has time.
>>
>> > I have ported this arc stuff to the trunk.  It is in r4783.  Please let
>> > me know how that works, particularly wrt performance, for you.  Some
>> > things that are in C on the branch were done in Python on the trunk for
>> > convenience.
>>
>> Awesome -- thanks again Michael.  You keep stealing my holiday 
>> projects :-)
>>
>> I added a check to make sure the 'axes' instance was present at draw
>> time, since patches can be added directly to the figure also, and
>> noted in the docstring that this is an axes only object.
>>
>> I also added unit support, which is fairly trivial, and added Arc to
>> the unit/ellipse_large.py test case and
>> examples/unit/ellipse_with_units.py
>>
>> Great work!
>> JDH
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel@...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel 

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Tedd, Michael -

Sorry to join this discussion late, but wouldn't it be easier to use a
representation of an ellipse in complex variables? That's what I have been
using and it seems pretty quick to me.

Have you guys tried this? Any experience you want to share?

Mark

On Dec 25, 2007 5:39 AM, Mark Bakker <markbak@...> wrote:
> Tedd, Michael -
>
> Sorry to join this discussion late, but wouldn't it be easier to use a
> representation of an ellipse in complex variables? That's what I have been
> using and it seems pretty quick to me.

The main point we are discussion is not how to represent the ellipse,
but what geometric primitives to use to render the ellipse.  Earlier
versions  of mpl used a polygon, then we added support for a 4 spline
approximation to reduce the error of the approximation.  This too did
not have sufficient accuracy, and so Michael added an 8 spline
approximation (Ellipse) as well as an adaptive approximation (Arc).

JDH

On Dec 16, 2007 5:26 PM, Andrew Straw <strawman@...> wrote:

> As a followup to the work on floats, I fixed rec2csv to deal with funky
> strings (strings with commas and quotes). I've checked this is as r4749,
> but I thought I'd announce here in case someone (John, in particular)
> was depending on some peculiar aspect of the old implementation. If
> there is something you depend on, can you update the unittests in
> unit/mlab_unit.py to check for the behavior you need?
>
> Also, John, I didn't see mlab.FormatString being used anywhere, but I
> didn't want to change it, either. So I made FormatString2 and used it.
> But if there's no known use of FormatString, lets kill the original and
> more FormatString2 into its place.

I am happy to leave it as is, and fix it if I bump into anything.
There was a use case at work that made me quote all the strings with "
but I am not sure what  it is right now so I will test with your
version tomorrow and see if I can find any problems.

There is one more thing I'd like to clean up in this code, and that is
the handling of date and datetime.  The current implementation will
return datetime regardless of whether the string looks like a date or
a datetime.  Thus it would fail a roundtrip tests if you had a date
field.  I may make some changes tomorrow to support date or datetime.

What do you think of making checkrows=0 the default for csv2rec?  It
is slower, but mostly guaranteed to do the right thing, whereas
checkrows=5 which is the current default can often fail if the first 5
rows match some type but later rows do  not, and may lead to confusion
among new users.

JDH

On Dec 16, 2007 5:26 PM, Andrew Straw <strawman@...> wrote:

> As a followup to the work on floats, I fixed rec2csv to deal with funky
> strings (strings with commas and quotes). I've checked this is as r4749,
> but I thought I'd announce here in case someone (John, in particular)
> was depending on some peculiar aspect of the old implementation. If
> there is something you depend on, can you update the unittests in
> unit/mlab_unit.py to check for the behavior you need?

OK, I found one problem with repr.  When adding support for date vs
datetime to the csv2rec roundtrip, and adding this to the unit test, I
notice that repr is probably not what we want for datetime; str makes
more sense here:

In [1]: import datetime

In [2]: d = datetime.date.today()

In [3]: repr(d)
Out[3]: 'datetime.date(2007, 12, 16)'

In [4]: str(d)
Out[4]: '2007-12-16'

The latter is more natural in the CSV file, and the repr version is
not supported by dateutil, at least not the one we are shipping:

In [5]: import dateutil.parser

In [6]: dateutil.parser.parse(repr(d))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)

/Users/jdhunter/python/svn/matplotlib/examples/<ipython console> in <module>()

...snip the rest of the traceback...

In [7]: dateutil.parser.parse(str(d))
Out[7]: datetime.datetime(2007, 12, 16, 0, 0)


So I changed FormatObject to use str, pending further discussion.  At
least for my common use cases, the only obj types I have in my record
arrays are dates and datetimes, and I find this to be a pretty
compelling use case since it is the type least likely to be supported
by other persistence methods (tostring and pickle both fail or do not
behave as expected with datetimes in the recarray).

But there is an oddity in the parsing of milliseconds which is causing
the updated unit test to fail; the code below illustrates the problem:

In [3]: import dateutil.parser

In [4]: import datetime

In [5]: s = '2007-12-18 22:29:34.924122'

In [6]: dateutil.parser.parse(s)
Out[6]: datetime.datetime(2007, 12, 18, 22, 29, 34, 924121)

Thoughts?

JDH

On Dec 16, 2007 9:10 PM, John Hunter <jdh2358@...> wrote:
> On Dec 16, 2007 5:26 PM, Andrew Straw <strawman@...> wrote:
>
> > As a followup to the work on floats, I fixed rec2csv to deal with funky
> > strings (strings with commas and quotes). I've checked this is as r4749,
> > but I thought I'd announce here in case someone (John, in particular)
> > was depending on some peculiar aspect of the old implementation. If
> > there is something you depend on, can you update the unittests in
> > unit/mlab_unit.py to check for the behavior you need?
> >
> > Also, John, I didn't see mlab.FormatString being used anywhere, but I
> > didn't want to change it, either. So I made FormatString2 and used it.
> > But if there's no known use of FormatString, lets kill the original and
> > more FormatString2 into its place.
>
> I am happy to leave it as is, and fix it if I bump into anything.
> There was a use case at work that made me quote all the strings with "
> but I am not sure what  it is right now so I will test with your
> version tomorrow and see if I can find any problems.
>
> There is one more thing I'd like to clean up in this code, and that is
> the handling of date and datetime.  The current implementation will
> return datetime regardless of whether the string looks like a date or
> a datetime.  Thus it would fail a roundtrip tests if you had a date
> field.  I may make some changes tomorrow to support date or datetime.

I'm CCing Chris Burns, from Berkeley here, but it would be great if
you post a note of these changes on the numpy list.  One of the things
to come out of the sprint was a lot of work on numpy i/o, and that
includes looking into integrating much of the MPL facilities.  So it
would be great if you keep the numpy team posted, to make sure they
get your latest code.

Cheers.

>
> What do you think of making checkrows=0 the default for csv2rec?  It
> is slower, but mostly guaranteed to do the right thing, whereas
> checkrows=5 which is the current default can often fail if the first 5
> rows match some type but later rows do  not, and may lead to confusion
> among new users.

Thanks for looking at this. From the unittest that you checked in, it 
appears you've succeeded in getting dates and datetimes to work, modulo 
the apparent datetime precision buglet you mention below.

My understanding with repr() is that, in general, it should attempt to 
support lossless copying of data with the eval() function. It does 
appear that dateutils does this -- repr() produces a string that looks 
like what you'd use to call the constructor. (Numpy scalars don't follow 
this idiom, and it's doubtful they could change without breaking a lot 
of code.) I also agree that for a CSV file, something like the str() on 
date and datetime objects makes most sense. This is all just to say that 
the date and datetime stuff looks good to my eye, but you're using that 
stuff a lot more than me. Finally, from your example, it does look like 
there's a bug somewhere in the datetime code. Since I don't use them, I 
leave that to you...

Finally, as far as the checkrows -- I must admit that I wasn't even 
aware of its existence until you asked the question. (That's the problem 
with writing code that just works - no one notices it!) Now that you've 
posed the question, I suppose I'm in favor a default of 0 (which 
actually means infinity). I suppose no one in their right mind is going 
to use CSV files to store gigabytes of data, so parsing the whole thing 
for consistency seems like a worthwhile expenditure of cycles. (And if 
they're thinking about it, maybe the slowness with dissuade them, or at 
least cause them to read the docstrings! :)

John Hunter wrote:
> On Dec 16, 2007 5:26 PM, Andrew Straw <strawman@...> wrote:
>
>   
>> As a followup to the work on floats, I fixed rec2csv to deal with funky
>> strings (strings with commas and quotes). I've checked this is as r4749,
>> but I thought I'd announce here in case someone (John, in particular)
>> was depending on some peculiar aspect of the old implementation. If
>> there is something you depend on, can you update the unittests in
>> unit/mlab_unit.py to check for the behavior you need?
>>     
>
> OK, I found one problem with repr.  When adding support for date vs
> datetime to the csv2rec roundtrip, and adding this to the unit test, I
> notice that repr is probably not what we want for datetime; str makes
> more sense here:
>
> In [1]: import datetime
>
> In [2]: d = datetime.date.today()
>
> In [3]: repr(d)
> Out[3]: 'datetime.date(2007, 12, 16)'
>
> In [4]: str(d)
> Out[4]: '2007-12-16'
>
> The latter is more natural in the CSV file, and the repr version is
> not supported by dateutil, at least not the one we are shipping:
>
> In [5]: import dateutil.parser
>
> In [6]: dateutil.parser.parse(repr(d))
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
>
> /Users/jdhunter/python/svn/matplotlib/examples/<ipython console> in <module>()
>
> ...snip the rest of the traceback...
>
> In [7]: dateutil.parser.parse(str(d))
> Out[7]: datetime.datetime(2007, 12, 16, 0, 0)
>
>
> So I changed FormatObject to use str, pending further discussion.  At
> least for my common use cases, the only obj types I have in my record
> arrays are dates and datetimes, and I find this to be a pretty
> compelling use case since it is the type least likely to be supported
> by other persistence methods (tostring and pickle both fail or do not
> behave as expected with datetimes in the recarray).
>
> But there is an oddity in the parsing of milliseconds which is causing
> the updated unit test to fail; the code below illustrates the problem:
>
> In [3]: import dateutil.parser
>
> In [4]: import datetime
>
> In [5]: s = '2007-12-18 22:29:34.924122'
>
> In [6]: dateutil.parser.parse(s)
> Out[6]: datetime.datetime(2007, 12, 18, 22, 29, 34, 924121)
>
> Thoughts?
>
> JDH
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Darren Dale wrote:
> On Thursday 13 December 2007 2:22:13 pm Robert Kern wrote:
>> John Hunter wrote:
>>> Do we need namespace packages in toolkits?  I recently added gtktools
>>> and exceltools to toolkits, and got a very hard to debug error:
>>>
>>> In [1]: import matplotlib
>>>
>>> In [2]: matplotlib.__file__
>>> Out[2]:
>>> '/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/__init__.py
>>> c'
>>>
>>> In [3]: matplotlib.rcParams['axes.axisbelow']
>>> Out[3]: False
>>>
>>> In [4]: import matplotlib.toolkits.gtktools
>>>
>>> In [5]: matplotlib.__file__
>>> Out[5]:
>>> '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__ini
>>> t__.pyc'
>>>
>>> In [6]: matplotlib.rcParams['axes.axisbelow']
>>> ------------------------------------------------------------
>>> Traceback (most recent call last):
>>>   File "<ipython console>", line 1, in ?
>>> KeyError: 'axes.axisbelow'
>>>
>>> Notice that the matplotlib module which was previously imported got
>>> changed by the import of the toolkit which declared itself a namespace
>>> package.  Now this may be the result of mpl not using namespace
>>> packages correctly (eg matplotlib/__init__.py should be empty save for
>>> the namespace package or something like that)
>> Probably. In setuptools 0.7, they can have stuff in them, but all of them
>> must be identical.
> 
> How would one learn about this new feature? According to setuptools.txt from 
> the 0.7 svn checkout, the namespace package's __init__.py must still be empty 
> save the namespace declaration.

It's a logical consequence of how namespace packages are loaded in 0.7 as has
been explained on the mailing list whenever the subject of namespace packages in
0.7 is brought up. It's still not recommended, but it can be made to work. The
remaining objection in the documentation is about installing the package using
--single-version-externally-managed for distro packagers. Using --svem, the
__init__.py files are not installed. Since a distro package would have a
mynamespace-common package (or something similar) that *just* provides this
__init__.py file for all of the mynamespace.foo packages, it can provide the
single copy of the meaningful code.

But it's still not a good idea.

>> If you are using an unmodified matplotlib/__init__.py in 
>> the main matplotlib package, then the "if 0:" is preventing the declaration
>> of the namespace package, and that may be contributing.
>>
>> Also, neither matplotlib nor matplotlib.toolkits is named as a
>> namespace_package in matplotlib's setup.py, and both have to be. Similarly,
>> basemap (for example) only lists matplotlib.toolkits but not matplotlib.
> 
> If I understand correctly, even after fixing these two problems, namespace 
> packages would not work since matplotlib/__init__.py contains lots of 
> meaningful code. Based on the remarks at 
> http://mail.python.org/pipermail/distutils-sig/2007-October/008346.html and 
> the setuptools svn commit log, it looks like development on version 0.7 is 
> not a priority and a release in the next year is unlikely. So is it fair to 
> say that for the foreseeable future, matplotlib is not compatible with 
> namespace packages unless we cleaned out __init__.py? 

Yes. Phillip explained this in February and suggested a way around the issue:

  http://www.mail-archive.com/distutils-sig@...>

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

OK, I created a test for numpy to isolate an issue: 
http://scipy.org/scipy/numpy/ticket/629

I think setting MPL's behavior to repr() is good, though. John, I see 
you did that r4745 -- thanks.

-Andrew

Fernando Perez wrote:
> On Dec 15, 2007 2:52 PM, Andrew Straw <strawman@...> wrote:
>> Hang on a minute, it looks like numpy.float64.__repr__() itself isn't
>> reproducing all significant digits... I'm writing up a test now and will
>> move this to the numpy list. I'm not sure how much is MPL and how much
>> is numpy at this point.
> 
> you may want to ping also at #scipy on irc.freenode.net, where the
> sprint participants are all hanging out.
> 
> cheers,
> 
> f

OK, I added unit/mlab_unit.py to svn. This checks that double precision 
floats can round-trip through rec2csv and csv2rec. To pass requires not 
only svn matplotlib (due to John's change), but also svn numpy (to be 
included with 1.0.5).

Also, it doesn't seem to me that rec2csv should close a file handle 
passed to it. This prevents use with StringIO, for example. So, I added 
a test for that, too. John, if it's not going to break anything for you, 
I'll go ahead and fix that.

John Hunter wrote:
> On Dec 15, 2007 3:13 PM, Andrew Straw <strawman@...> wrote:
>   
>> mlab.defaultformatd sets the default float formatter for rec2csv() to be
>> something that doesn't keep the full representation of a floating point
>> number. Obviously, I can pass in my own formatd argument to rec2csv(),
>> but I wonder if there's any reason why defaultformatd shouldn't use
>> repr() for numpy.float32 and numpy.float64?
>>     
>
> I'll be happy to make repr the default.  I was using %g because I
> mistakenly though this provided the appropriate number of significant
> digits.  I changed this to %r in the csvformat_factory in  svn.
>
> JDJ
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

On Dec 16, 2007 1:33 PM, Andrew Straw <strawman@...> wrote:
> OK, I added unit/mlab_unit.py to svn. This checks that double precision
> floats can round-trip through rec2csv and csv2rec. To pass requires not
> only svn matplotlib (due to John's change), but also svn numpy (to be
> included with 1.0.5).
>
> Also, it doesn't seem to me that rec2csv should close a file handle
> passed to it. This prevents use with StringIO, for example. So, I added
> a test for that, too. John, if it's not going to break anything for you,
> I'll go ahead and fix that.

Yes, please do.  You might want to modify cbook.to_filehandle to
optionally return a tuple specifying whether the returned fh is owned
by the caller.  Something like:

    fh, owner = cbook.to_filehandle(fname, returnowner=True)

and then later

    if owner:
        fh.close()

JDH

Done in r4748. I added the kwarg "return_opened" to cbook.to_filehandle().

John Hunter wrote:
> On Dec 16, 2007 1:33 PM, Andrew Straw <strawman@...> wrote:
>> OK, I added unit/mlab_unit.py to svn. This checks that double precision
>> floats can round-trip through rec2csv and csv2rec. To pass requires not
>> only svn matplotlib (due to John's change), but also svn numpy (to be
>> included with 1.0.5).
>>
>> Also, it doesn't seem to me that rec2csv should close a file handle
>> passed to it. This prevents use with StringIO, for example. So, I added
>> a test for that, too. John, if it's not going to break anything for you,
>> I'll go ahead and fix that.
> 
> Yes, please do.  You might want to modify cbook.to_filehandle to
> optionally return a tuple specifying whether the returned fh is owned
> by the caller.  Something like:
> 
>     fh, owner = cbook.to_filehandle(fname, returnowner=True)
> 
> and then later
> 
>     if owner:
>         fh.close()
> 
> JDH
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel@...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel