matplotlib-devel — matplotlib developers

Re: [matplotlib-devel] Matplotlib property support

[Benjamin R. <ben...@ou...>]

On Wed, Jan 16, 2013 at 2:42 PM, Todd <tod...@gm...> wrote:

> Currently matplotlib uses set_ and get_ functions for reading and writing
> values.  However, since 2.6 python supports properties, which allow access
> to such values as attributes in addition to using the functions directly.
> Would it be worthwhile implementing property support in matplotlib?
>
>
This was actually discussed during a Birds of a Feather (BOF) meeting back
in SciPy2012 (which John Hunter attended).  I am definitely for the idea,
but it is going to be a very painful and long deprecation process.  A bit
of background (at least, my understanding of it)...

In matlab, there is the setp and getp functions.  With this, you can set
and get properties of various graphing objects.  It is this feature that
John duplicated, and it is the organic growth from there is why we are
where we are now.

Furthermore, for those who haven't noticed, we have a strange "alias"
feature.  For example, one could set edgecolor, but also ec, to the same
effect.  In a lark, I decided to trace down this feature and, to my horror,
discovered how it works.  Believe it or not, the docstring for
set_edgecolor (or get_edgecolor, I forget) mentions that "ec" is an alias,
and there is code elsewhere that processes the docstrings, looking for
mentions of aliases, and uses that to guide the setting of properties.  Of
course, don't forget that sometimes we do the plural of a property name as
well...


Rather than keeping these attributes as a part of the artist objects, I
would rather encapsulate the artist properties using a new Properties class
(maybe it subclasses from dict?).  This would make it easier to apply/share
properties between many objects, and be more future-proof.

My 2 cents,
Ben Root

Re: [matplotlib-devel] Matplotlib property support

[Michael D. <md...@st...>]

This seems like a good candidate for a MEP.  We'd want to take a 
graceful approach to transitioning to properties.

See here for information about MEPs:

https://github.com/matplotlib/matplotlib/wiki

Mike

On 01/16/2013 02:42 PM, Todd wrote:
> Currently matplotlib uses set_ and get_ functions for reading and 
> writing values.  However, since 2.6 python supports properties, which 
> allow access to such values as attributes in addition to using the 
> functions directly.  Would it be worthwhile implementing property 
> support in matplotlib?
>
>
> ------------------------------------------------------------------------------
> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
> and much more. Keep your Java skills current with LearnJavaNow -
> 200+ hours of step-by-step video tutorials by Java experts.
> SALE $49.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122612
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

[matplotlib-devel] Matplotlib property support

[Todd <tod...@gm...>]

Currently matplotlib uses set_ and get_ functions for reading and writing
values.  However, since 2.6 python supports properties, which allow access
to such values as attributes in addition to using the functions directly.
Would it be worthwhile implementing property support in matplotlib?

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Paulo C. P. de A. <pau...@gm...>]

2013/1/16 Michael Droettboom <md...@st...>:

[...]
>> $ ls /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/
>> LICENSE_STIX           STIXNonUniBol.ttf      STIXSizOneSymBol.ttf
>> STIXGeneralBolIta.ttf  STIXNonUniIta.ttf      STIXSizOneSymReg.ttf
>> STIXGeneralBol.ttf     STIXNonUni.ttf         STIXSizThreeSymBol.ttf
>> STIXGeneralItalic.ttf  STIXSizFiveSymReg.ttf  STIXSizThreeSymReg.ttf
>> STIXGeneral.ttf        STIXSizFourSymBol.ttf  STIXSizTwoSymBol.ttf
>> STIXNonUniBolIta.ttf   STIXSizFourSymReg.ttf  STIXSizTwoSymReg.ttf
>>
>> This should require asking permission for bundling, but not duplicating
>> as ttf stix font are not installed. Before asking for it, you may want to
>> look at the test results:

  BTW, note that asking for such permissions may take a very long time,
and may be "denied" unless there is a very convincing reason to bundle it...

>> http://pcpa.fedorapeople.org/matplotlib-2.7-mpl-data-with-bundled-stix-ttf.txt
>> Because I did not see any difference on the results, so I probably missed
>> some step...
>
>
> Did you turn USE_FONTCONFIG back to False?  font-config won't find fonts
> here, but matplotlib's built-in font lookup mechanism will.

  Ops :-) but I think something is still messed, maybe it is getting confused
by finding system STIX 1.1 before the bundled ones somehow:

http://pcpa.fedorapeople.org/matplotlib-2.7-bundled-stix-ttf+fontconfig=false.txt

> Mike

Paulo

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Michael D. <md...@st...>]

On 01/16/2013 01:32 PM, Paulo César Pereira de Andrade wrote:
> 2013/1/16 Michael Droettboom <md...@st...>:
>
>> There was a very silly bug lurking in USE_FONTCONFIG=True mode, and I've
>> made a PR with a fix here:
>>
>> https://github.com/matplotlib/matplotlib/pull/1666
>    Updated test case with that patch:
> http://pcpa.fedorapeople.org/matplotlib-2.7+pull-1666.txt
>
>> However, I thought I'd investigate the issue with the STIX fonts -- I
>> actually just upgraded to F18 yesterday, so I thought I'd try what you
>> suggest and get matplotlib to use them.
>>
>> Thanks for taking on packaging for matplotlib, and thanks for understanding
>> the importance of running the test suite.
>>
>> It seems that the stix fonts packages in Fedora only come in .otf format --
>> I believe this is how they are shipped upstream as well.  Unfortunately,
>> matplotlib has no support for reading .otf files, so these are simply
>> unusable to matplotlib.
>>
>> There are some options:
>>
>> 1) Include the STIX ttf fonts included with matplotlib in the matplotlib
>> package and install them in `matplotlib/mpl-data/fonts/ttf` (as a vanilla
>> install would do) so as not to conflict with the stix-fonts package.  Maybe
>> these go in a python-matplotlib-stix-fonts package.
>    For the sake of correctness, I believe this should be the better option.
> I did:
> $ sudo mkdir /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf
> $ sudo cp -a matplotlib-1.2.0/lib/matplotlib/mpl-data/fonts/ttf/*STIX*
> /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fontsSave: No
> changes need to be saved, save again to override -- nothing saved.
> $ ls /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/
> LICENSE_STIX           STIXNonUniBol.ttf      STIXSizOneSymBol.ttf
> STIXGeneralBolIta.ttf  STIXNonUniIta.ttf      STIXSizOneSymReg.ttf
> STIXGeneralBol.ttf     STIXNonUni.ttf         STIXSizThreeSymBol.ttf
> STIXGeneralItalic.ttf  STIXSizFiveSymReg.ttf  STIXSizThreeSymReg.ttf
> STIXGeneral.ttf        STIXSizFourSymBol.ttf  STIXSizTwoSymBol.ttf
> STIXNonUniBolIta.ttf   STIXSizFourSymReg.ttf  STIXSizTwoSymReg.ttf
>
> This should require asking permission for bundling, but not duplicating
> as ttf stix font are not installed. Before asking for it, you may want to
> look at the test results:
> http://pcpa.fedorapeople.org/matplotlib-2.7-mpl-data-with-bundled-stix-ttf.txt
> Because I did not see any difference on the results, so I probably missed
> some step...

Did you turn USE_FONTCONFIG back to False?  font-config won't find fonts 
here, but matplotlib's built-in font lookup mechanism will.

Mike
>
>> 2) Include a version of the STIX fonts converted to ttf.  This will still
>> have the problem that the glyph tables in matplotlib need to be updated to
>> use them.
>>
>> 3) Update matplotlib's freetype wrapper to support .otf fonts.  Doable, but
>> considerable work.
>>
>> 4) Leave it as is but warn that STIX font support is broken with the Fedora
>> matplotlib package.
>>
>> Mike
> I actually did submit an update
> https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-7.fc18
> before you posted about
> https://github.com/matplotlib/matplotlib/pull/1666
> So would prefer a more complete review before overriding it.
> The update should be in queue for a week before going to
> mirrors, so it can be revoked and a better update submitted.
>
> Thanks,
> Paulo

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Paulo C. P. de A. <pau...@gm...>]

2013/1/16 Michael Droettboom <md...@st...>:

> There was a very silly bug lurking in USE_FONTCONFIG=True mode, and I've
> made a PR with a fix here:
>
> https://github.com/matplotlib/matplotlib/pull/1666

  Updated test case with that patch:
http://pcpa.fedorapeople.org/matplotlib-2.7+pull-1666.txt

> However, I thought I'd investigate the issue with the STIX fonts -- I
> actually just upgraded to F18 yesterday, so I thought I'd try what you
> suggest and get matplotlib to use them.
>
> Thanks for taking on packaging for matplotlib, and thanks for understanding
> the importance of running the test suite.
>
> It seems that the stix fonts packages in Fedora only come in .otf format --
> I believe this is how they are shipped upstream as well.  Unfortunately,
> matplotlib has no support for reading .otf files, so these are simply
> unusable to matplotlib.
>
> There are some options:
>
> 1) Include the STIX ttf fonts included with matplotlib in the matplotlib
> package and install them in `matplotlib/mpl-data/fonts/ttf` (as a vanilla
> install would do) so as not to conflict with the stix-fonts package.  Maybe
> these go in a python-matplotlib-stix-fonts package.

  For the sake of correctness, I believe this should be the better option.
I did:
$ sudo mkdir /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf
$ sudo cp -a matplotlib-1.2.0/lib/matplotlib/mpl-data/fonts/ttf/*STIX*
/usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fontsSave: No
changes need to be saved, save again to override -- nothing saved.
$ ls /usr/lib64/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/
LICENSE_STIX           STIXNonUniBol.ttf      STIXSizOneSymBol.ttf
STIXGeneralBolIta.ttf  STIXNonUniIta.ttf      STIXSizOneSymReg.ttf
STIXGeneralBol.ttf     STIXNonUni.ttf         STIXSizThreeSymBol.ttf
STIXGeneralItalic.ttf  STIXSizFiveSymReg.ttf  STIXSizThreeSymReg.ttf
STIXGeneral.ttf        STIXSizFourSymBol.ttf  STIXSizTwoSymBol.ttf
STIXNonUniBolIta.ttf   STIXSizFourSymReg.ttf  STIXSizTwoSymReg.ttf

This should require asking permission for bundling, but not duplicating
as ttf stix font are not installed. Before asking for it, you may want to
look at the test results:
http://pcpa.fedorapeople.org/matplotlib-2.7-mpl-data-with-bundled-stix-ttf.txt
Because I did not see any difference on the results, so I probably missed
some step...

> 2) Include a version of the STIX fonts converted to ttf.  This will still
> have the problem that the glyph tables in matplotlib need to be updated to
> use them.
>
> 3) Update matplotlib's freetype wrapper to support .otf fonts.  Doable, but
> considerable work.
>
> 4) Leave it as is but warn that STIX font support is broken with the Fedora
> matplotlib package.
>
> Mike

I actually did submit an update
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-7.fc18
before you posted about
https://github.com/matplotlib/matplotlib/pull/1666
So would prefer a more complete review before overriding it.
The update should be in queue for a week before going to
mirrors, so it can be revoked and a better update submitted.

Thanks,
Paulo

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Michael D. <md...@st...>]

On 01/16/2013 12:22 PM, Paulo César Pereira de Andrade wrote:
> 2013/1/16 Michael Droettboom <md...@st...>:
>
>>> -USE_FONTCONFIG = False
>>> +USE_FONTCONFIG = True
> [...]
>>> I think this is safe to be made a patch for Linux distros. I believe this could
>>> be the default for *Linux and *BSD in matplotlib.
>>>
>> This will silence things, and I agree it probably should be the default
>> on Linux, but I'm not sure if it's correct.  If F18 really does have
>    So far I managed to "apparently" solve my issues with my work in
> progress sagemath package for fedora.
>
>> STIX 1.1, then it's not going to work since the layout of the fonts has
>> changed so much.  That's going to require a great deal of effort.  Can
>    Yes, it has STIX 1.1.
>
>> you successfully run the test suite with this change?  That should catch
>    Sorry that I am not much experienced with matplotlib, but I am willing to
> help in whatever I can, given that now I can modify/update the matplotlib
> package in Fedora. After running it for the first time, and not looking
> much at the sources, it appears to have way too many test failures...
>
> $ nosetests-2.7 -v matplotlib.tests 2>&1 | tee ~/matplotlib-2.7.txt
> http://pcpa.fedorapeople.org/matplotlib-2.7.txt
>
> $ nosetests-3.3 -v matplotlib.tests 2>&1 | tee ~/matplotlib-3.3.txt
> http://pcpa.fedorapeople.org/matplotlib-3.3.txt
>
>> any glyph mismatch problems.  Is there a way to have the package depend
>> on 1.0 version of the fonts until matplotlib has a chance to update its
>> tables?
>    I am afraid this may not be an option (using stix fonts 1.0).
>
>
There was a very silly bug lurking in USE_FONTCONFIG=True mode, and I've 
made a PR with a fix here:

https://github.com/matplotlib/matplotlib/pull/1666

However, I thought I'd investigate the issue with the STIX fonts -- I 
actually just upgraded to F18 yesterday, so I thought I'd try what you 
suggest and get matplotlib to use them.

Thanks for taking on packaging for matplotlib, and thanks for 
understanding the importance of running the test suite.

It seems that the stix fonts packages in Fedora only come in .otf format 
-- I believe this is how they are shipped upstream as well. 
Unfortunately, matplotlib has no support for reading .otf files, so 
these are simply unusable to matplotlib.

There are some options:

1) Include the STIX ttf fonts included with matplotlib in the matplotlib 
package and install them in `matplotlib/mpl-data/fonts/ttf` (as a 
vanilla install would do) so as not to conflict with the stix-fonts 
package.  Maybe these go in a python-matplotlib-stix-fonts package.

2) Include a version of the STIX fonts converted to ttf.  This will 
still have the problem that the glyph tables in matplotlib need to be 
updated to use them.

3) Update matplotlib's freetype wrapper to support .otf fonts. Doable, 
but considerable work.

4) Leave it as is but warn that STIX font support is broken with the 
Fedora matplotlib package.

Mike

[matplotlib-devel] Upcoming release schedule thoughts

[Michael D. <md...@st...>]

There have been bits and pieces of discussion about future releases on 
PRs, and I thought I'd bring it over here to tie loose ends together.

There have been 83 commits since 1.2.0 -- it's probably time to start 
thinking about a 1.2.1.  The release candidate cycle shouldn't be as 
long for this, as it's only a bugfix release.  If you are aware of any 
issues or PRs in the tracker that make sense for 1.2.1 (i.e. simple 
bugfixes), please assign to the 1.2.x milestone.  Once that milestone is 
empty, sometime next week I will probably put out a 1.2.1rc1.

As for 1.3, we have MEP10 (documentation improvements) spearheaded by 
Nelle and MEP12 (gallery reorganization) spearheaded by Tony that I'd 
like to get in before that release.  Both of these changes require a lot 
of manual work, but it's fairly separable work. Perhaps we can divide 
the modules amongst a number of volunteers so the fearless leaders of 
both these projects shoulders less of the burden?  Given the size of 
these projects, that probably means holding off on 1.3 for some time, 
but we can always do another 1.2.x in the meantime.

Does this make sense?  Any additional thoughts?

Cheers,
Mike

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Paulo C. P. de A. <pau...@gm...>]

2013/1/16 Michael Droettboom <md...@st...>:

>> -USE_FONTCONFIG = False
>> +USE_FONTCONFIG = True
[...]
>> I think this is safe to be made a patch for Linux distros. I believe this could
>> be the default for *Linux and *BSD in matplotlib.
>>
> This will silence things, and I agree it probably should be the default
> on Linux, but I'm not sure if it's correct.  If F18 really does have

  So far I managed to "apparently" solve my issues with my work in
progress sagemath package for fedora.

> STIX 1.1, then it's not going to work since the layout of the fonts has
> changed so much.  That's going to require a great deal of effort.  Can

  Yes, it has STIX 1.1.

> you successfully run the test suite with this change?  That should catch

  Sorry that I am not much experienced with matplotlib, but I am willing to
help in whatever I can, given that now I can modify/update the matplotlib
package in Fedora. After running it for the first time, and not looking
much at the sources, it appears to have way too many test failures...

$ nosetests-2.7 -v matplotlib.tests 2>&1 | tee ~/matplotlib-2.7.txt
http://pcpa.fedorapeople.org/matplotlib-2.7.txt

$ nosetests-3.3 -v matplotlib.tests 2>&1 | tee ~/matplotlib-3.3.txt
http://pcpa.fedorapeople.org/matplotlib-3.3.txt

> any glyph mismatch problems.  Is there a way to have the package depend
> on 1.0 version of the fonts until matplotlib has a chance to update its
> tables?

  I am afraid this may not be an option (using stix fonts 1.0).

> Mike

Thanks,
Paulo

Re: [matplotlib-devel] Usefulness of Travis-CI

[Todd <tod...@gm...>]

On Jan 16, 2013 9:30 AM, "Fernando Perez" <fpe...@gm...> wrote:
>
> On Wed, Jan 16, 2013 at 12:10 AM, Nelle Varoquaux
> <nel...@gm...> wrote:
>
> > Last but not least, maybe we can see what numfocus has to offer.
>
> Absolutely!  I'll be offline for two weeks, but others on the list can
> certainly propose this to numfocus on the list and we can look into
> what can be done, esp. in a way that could also help other projects as
> well.
>
> Also, there's snakebite: http://www.snakebite.net.  The project seemed
> very dormant for a long time, but there's been some activity since:
> http://www.snakebite.net/network.  I'd ping Titus Brown on Twitter
> (@ctitusbrown) for info...
>
> Cheers,

There is also the open build service, which is more of a build farm but can
be set up pull, build, test, and publish git snapshots for most common
Linux distributions to your own online software repository simultaneously
with one click on a website or one commandline call.

https://build.opensuse.org/

They provide hosting, building, and distribution for free.  You can
probably set up a script to automatically rebuild master on a change, or
daily.  However, setting it up to test individual commits would be overly
difficult and probably be seen as an abuse of the system.  Using it to
always build, test, and release offer the latest master to most linux
distros, on the other hand, would be fine.  If someone contacts them they
can probably set up a repository just for you, or if this sort of thing is
useful a more general repository you can share with others (there is
already devel:languages:python, maybe devel:languages:python:unstable).

You can also use it to build release rpms and debs for various distros.  It
is already being used to build the packages discussed so far for openSUSE,
but if someone is willing to maintain them they can be built for other
distros as well.

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Michael D. <md...@st...>]

On 01/16/2013 10:38 AM, Paulo César Pereira de Andrade wrote:
> 2012/12/8 Paulo César Pereira de Andrade
> <pau...@gm...>:
>>    Hi,
>>
>>    Recently I asked to become  comaintainer of matplotlib in Fedora and
>> did update to 1.2.0
> [...]
>
>>      doctest:1214: UserWarning: findfont: Font family ['STIXGeneral']
>> not found. Falling back to Bitstream Vera Sans
> [...]
>
>>    I opened two bug reports about it at:
>> https://bugzilla.redhat.com/show_bug.cgi?id=885307 and
>> https://bugzilla.redhat.com/show_bug.cgi?id=885312
> [...]
>
> I just did look at it again a bit more, and found that the solution is quite
> simple. Just need the pseudo patch:
>
> ---%<---
> $ diff -u /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py{.orig,}
> --- /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py.orig
>   2013-01-16 13:25:07.922646400 -0200
> +++ /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py
>   2013-01-16 13:25:10.931646515 -0200
> @@ -62,7 +62,7 @@
>   except ImportError:
>       import pickle
>
> -USE_FONTCONFIG = False
> +USE_FONTCONFIG = True
>
>   verbose = matplotlib.verbose
>
> ---%<---
>
> I think this is safe to be made a patch for Linux distros. I believe this could
> be the default for *Linux and *BSD in matplotlib.
>
This will silence things, and I agree it probably should be the default 
on Linux, but I'm not sure if it's correct.  If F18 really does have 
STIX 1.1, then it's not going to work since the layout of the fonts has 
changed so much.  That's going to require a great deal of effort.  Can 
you successfully run the test suite with this change?  That should catch 
any glyph mismatch problems.  Is there a way to have the package depend 
on 1.0 version of the fonts until matplotlib has a chance to update its 
tables?

Mike

Re: [matplotlib-devel] Usefulness of Travis-CI

[Ryan M. <rm...@gm...>]

On Tue, Jan 15, 2013 at 6:20 PM, Damon McDougall
<dam...@gm...>wrote:

> Hi all,
>
> Recently, it has become more and more common that the travis builds
> fail.  They fail not because the test suite fails but because of some
> inherent issues with travis itself.  Usually, either the git clone
> fails or some other simple shell command fails.  I consider these
> 'failures' as being false negatives.  While a false negative once or
> twice a month would not be anything to worry about, I feel like this
> is happening often enough to render the pull request status extremely
> unhelpful.
>

I haven't been active enough lately to have a good feel overall on the
false negatives. What I can say right now is that it would help if we would
disable python 3.1 (and optionally replace it with 3.3). I believe (based
on numpy commit I saw the other day) that Travis no longer supports 3.1.
That would solve the failures I've seen with my PR's.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Re: [matplotlib-devel] Font issues in fedora 18 and rawhide

[Paulo C. P. de A. <pau...@gm...>]

2012/12/8 Paulo César Pereira de Andrade
<pau...@gm...>:
>   Hi,
>
>   Recently I asked to become  comaintainer of matplotlib in Fedora and
> did update to 1.2.0

[...]

>     doctest:1214: UserWarning: findfont: Font family ['STIXGeneral']
> not found. Falling back to Bitstream Vera Sans

[...]

>   I opened two bug reports about it at:
> https://bugzilla.redhat.com/show_bug.cgi?id=885307 and
> https://bugzilla.redhat.com/show_bug.cgi?id=885312

[...]

I just did look at it again a bit more, and found that the solution is quite
simple. Just need the pseudo patch:

---%<---
$ diff -u /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py{.orig,}
--- /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py.orig
 2013-01-16 13:25:07.922646400 -0200
+++ /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py
 2013-01-16 13:25:10.931646515 -0200
@@ -62,7 +62,7 @@
 except ImportError:
     import pickle

-USE_FONTCONFIG = False
+USE_FONTCONFIG = True

 verbose = matplotlib.verbose

---%<---

I think this is safe to be made a patch for Linux distros. I believe this could
be the default for *Linux and *BSD in matplotlib.

Thanks,
Paulo

Re: [matplotlib-devel] Usefulness of Travis-CI

[Michael D. <md...@st...>]

On 01/16/2013 07:15 AM, Thomas Kluyver wrote:
> On 16 January 2013 08:35, Nathaniel Smith <nj...@po... 
> <mailto:nj...@po...>> wrote:
>
>     When this has come up on the numfocus list before the status was,
>     sure, they can provide resources/funding if you can find someone
>     to do the actual work :-). If anyone is interested in putting in
>     the time to solve this problem properly then a lot of projects
>     would be grateful I think...
>
>
> We'd certainly be interested in a better solution for IPython - I 
> think we see more Travis builds failing because of Travis than because 
> of the actual code.
>
> Before Travis started testing pull requests, we wrote our own 
> test_pr.py script [1], which fetches a PR, merges it into master, 
> tests on several versions of Python, and (optionally) posts a comment 
> to Github with the results. This should be pretty easy to adapt to 
> other projects - NetworkX already decide to use it [2]. With a little 
> more code, I imagine we could configure a server to regularly scan 
> pull requests for changes and test them.

This definitely would be worth investigating.  At least in my 
institution, finding CPU time isn't terribly difficult.  Hosting our own 
network services is -- so something that does builds locally and then 
publishes the results somewhere else (i.e. Github) may make a lot of sense.

>
> Sympy also have a system for automatically testing pull requests [3]. 
> I think this is somewhat more advanced, with a queue on Google 
> Appengine, and workers which pull jobs from the queue.
>
> Finally, we're keen users of ShiningPanda [4]. It doesn't test pull 
> requests, but it's reliable, and can handle things like coverage 
> metrics nicely.

The matplotlib test suite is really CPU-intensive, and the last time I 
did the math, the ShiningPanda free plan was too tight -- it's about an 
hour of CPU time (on ShiningPanda's machines) per test for 4 
Numpy/Python combinations.  It's about $0.50/hr, and you get $6 free per 
month.

I really like ShiningPanda, mostly for how flexible the setup of 
prerequisites are, but it will cost some money to use.

Cheers,
Mike

>
> [1] https://github.com/ipython/ipython/blob/master/tools/test_pr.py
> [2] https://github.com/networkx/networkx/pull/752
> [3] http://reviews.sympy.org/
> [4] https://jenkins.shiningpanda.com/ipython/
>
> Best wishes,
> Thomas
>
>
> ------------------------------------------------------------------------------
> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
> and much more. Keep your Java skills current with LearnJavaNow -
> 200+ hours of step-by-step video tutorials by Java experts.
> SALE $49.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122612
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Usefulness of Travis-CI

[Thomas K. <th...@kl...>]

On 16 January 2013 08:35, Nathaniel Smith <nj...@po...> wrote:

> When this has come up on the numfocus list before the status was, sure,
> they can provide resources/funding if you can find someone to do the actual
> work :-). If anyone is interested in putting in the time to solve this
> problem properly then a lot of projects would be grateful I think...


We'd certainly be interested in a better solution for IPython - I think we
see more Travis builds failing because of Travis than because of the actual
code.

Before Travis started testing pull requests, we wrote our own test_pr.py
script [1], which fetches a PR, merges it into master, tests on several
versions of Python, and (optionally) posts a comment to Github with the
results. This should be pretty easy to adapt to other projects - NetworkX
already decide to use it [2]. With a little more code, I imagine we could
configure a server to regularly scan pull requests for changes and test
them.

Sympy also have a system for automatically testing pull requests [3]. I
think this is somewhat more advanced, with a queue on Google Appengine, and
workers which pull jobs from the queue.

Finally, we're keen users of ShiningPanda [4]. It doesn't test pull
requests, but it's reliable, and can handle things like coverage metrics
nicely.

[1] https://github.com/ipython/ipython/blob/master/tools/test_pr.py
[2] https://github.com/networkx/networkx/pull/752
[3] http://reviews.sympy.org/
[4] https://jenkins.shiningpanda.com/ipython/

Best wishes,
Thomas

Re: [matplotlib-devel] Usefulness of Travis-CI

[Nathaniel S. <nj...@po...>]

On 16 Jan 2013 09:30, "Fernando Perez" <fpe...@gm...> wrote:
>
> On Wed, Jan 16, 2013 at 12:10 AM, Nelle Varoquaux
> <nel...@gm...> wrote:
>
> > Last but not least, maybe we can see what numfocus has to offer.
>
> Absolutely!  I'll be offline for two weeks, but others on the list can
> certainly propose this to numfocus on the list and we can look into
> what can be done, esp. in a way that could also help other projects as
> well.

When this has come up on the numfocus list before the status was, sure,
they can provide resources/funding if you can find someone to do the actual
work :-). If anyone is interested in putting in the time to solve this
problem properly then a lot of projects would be grateful I think...

-n

Re: [matplotlib-devel] Usefulness of Travis-CI

[Fernando P. <fpe...@gm...>]

On Wed, Jan 16, 2013 at 12:10 AM, Nelle Varoquaux
<nel...@gm...> wrote:

> Last but not least, maybe we can see what numfocus has to offer.

Absolutely!  I'll be offline for two weeks, but others on the list can
certainly propose this to numfocus on the list and we can look into
what can be done, esp. in a way that could also help other projects as
well.

Also, there's snakebite: http://www.snakebite.net.  The project seemed
very dormant for a long time, but there's been some activity since:
http://www.snakebite.net/network.  I'd ping Titus Brown on Twitter
(@ctitusbrown) for info...

Cheers,

f

Re: [matplotlib-devel] Usefulness of Travis-CI

[Nelle V. <nel...@gm...>]

>> Hi all,
>>
>> Recently, it has become more and more common that the travis builds
>> fail.  They fail not because the test suite fails but because of some
>> inherent issues with travis itself.  Usually, either the git clone
>> fails or some other simple shell command fails.  I consider these
>> 'failures' as being false negatives.  While a false negative once or
>> twice a month would not be anything to worry about, I feel like this
>> is happening often enough to render the pull request status extremely
>> unhelpful.
>>
>> I realise the above is merely my opinion, but I feel that I would be
>> surprised if others did not find the Travis' false negative hit rate
>> tiresome.
>
> Agreed.
>
>>
>> If practical, I would like to move away from travis and have our test
>> suite run on a more reliable service.  How do others feel about this?
>>
>
> Good--but what is the superior alternative?

There's shiningpanda: this is what scikit-learn now uses (with
travis-ci): https://jenkins.shiningpanda.com/scikit-learn/
It's quite flexible, but I think you are limited to only a couple of
builds per day on master. It doesn't really replace travis, as travis
is supposed to run against PR but it is better than nothing.

Previously, scikit-learn used buildbot, hosted on one of the french
python association. I've got access to the server, and I can ask the
board members if we can set up a buildbot there. I'm sure that's not a
problem. But, once again, this will run the tests on master, not on
the PR.

Last but not least, maybe we can see what numfocus has to offer.

>
> I know nothing about who offers such services, why they do so, or the
> quality of the service.
>
> Eric
>
> ------------------------------------------------------------------------------
> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
> and much more. Keep your Java skills current with LearnJavaNow -
> 200+ hours of step-by-step video tutorials by Java experts.
> SALE $49.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122612
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Usefulness of Travis-CI

[Eric F. <ef...@ha...>]

On 2013/01/15 2:20 PM, Damon McDougall wrote:
> Hi all,
>
> Recently, it has become more and more common that the travis builds
> fail.  They fail not because the test suite fails but because of some
> inherent issues with travis itself.  Usually, either the git clone
> fails or some other simple shell command fails.  I consider these
> 'failures' as being false negatives.  While a false negative once or
> twice a month would not be anything to worry about, I feel like this
> is happening often enough to render the pull request status extremely
> unhelpful.
>
> I realise the above is merely my opinion, but I feel that I would be
> surprised if others did not find the Travis' false negative hit rate
> tiresome.

Agreed.

>
> If practical, I would like to move away from travis and have our test
> suite run on a more reliable service.  How do others feel about this?
>

Good--but what is the superior alternative?

I know nothing about who offers such services, why they do so, or the 
quality of the service.

Eric

[matplotlib-devel] Usefulness of Travis-CI

[Damon M. <dam...@gm...>]

Hi all,

Recently, it has become more and more common that the travis builds
fail.  They fail not because the test suite fails but because of some
inherent issues with travis itself.  Usually, either the git clone
fails or some other simple shell command fails.  I consider these
'failures' as being false negatives.  While a false negative once or
twice a month would not be anything to worry about, I feel like this
is happening often enough to render the pull request status extremely
unhelpful.

I realise the above is merely my opinion, but I feel that I would be
surprised if others did not find the Travis' false negative hit rate
tiresome.

If practical, I would like to move away from travis and have our test
suite run on a more reliable service.  How do others feel about this?

-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Re: [matplotlib-devel] PR #1599

[Damon M. <dam...@gm...>]

On Thu, Jan 10, 2013 at 7:40 AM, Michael Droettboom <md...@st...> wrote:
> Alternatively, maybe we could divide the open PRs into numerical ranges
> amongst some volunteers, who could triage and assign them to the attention
> of the appropriate experts?

I like this idea.

-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Re: [matplotlib-devel] Requiring numpy requirement to 1.5 on master?

[Phil E. <pel...@gm...>]

@mdboom - that sounds like roaring agreement to me.  :-)
Lets go for it.


On 14 January 2013 20:34, Damon McDougall <dam...@gm...> wrote:

> On Fri, Jan 11, 2013 at 12:33 PM, Eric Firing <ef...@ha...> wrote:
> > On 2013/01/11 5:38 AM, Michael Droettboom wrote:
> >> As pointed out in #1650, we have a bug on Python 3.x handling file-like
> >> objects that are UNIX FILEs, but not actual filesystem files, such as
> >> the sockets used for a urllib HTTP request.
> >>
> >> https://github.com/matplotlib/matplotlib/pull/1650
> >>
> >> As Christoph helpfully points out, Numpy has already solved this problem
> >> (since 1.5.0), so it would be easiest to just use their solution.
> >>
> >> For 1.2.x, I think we shouldn't update the Numpy requirement (which is
> >> currently 1.4) -- there, I'll have to port Numpy's compatibility
> >> functions to our code base.  But for master, I'd rather just use Numpy's
> >> stuff so all of the intricacies of this are handled in one place.
> >>
> >> (And as a detail, it isn't until Numpy 1.7 that a "CloseFile" function
> >> is provided, so even on master, we're stuck copying some code over).
> >>
> >> Any objections?
>
> Nope.
>
> --
> Damon McDougall
> http://www.damon-is-a-geek.com
> Institute for Computational Engineering Sciences
> 201 E. 24th St.
> Stop C0200
> The University of Texas at Austin
> Austin, TX 78712-1229
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122412
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Re: [matplotlib-devel] Requiring numpy requirement to 1.5 on master?

[Damon M. <dam...@gm...>]

On Fri, Jan 11, 2013 at 12:33 PM, Eric Firing <ef...@ha...> wrote:
> On 2013/01/11 5:38 AM, Michael Droettboom wrote:
>> As pointed out in #1650, we have a bug on Python 3.x handling file-like
>> objects that are UNIX FILEs, but not actual filesystem files, such as
>> the sockets used for a urllib HTTP request.
>>
>> https://github.com/matplotlib/matplotlib/pull/1650
>>
>> As Christoph helpfully points out, Numpy has already solved this problem
>> (since 1.5.0), so it would be easiest to just use their solution.
>>
>> For 1.2.x, I think we shouldn't update the Numpy requirement (which is
>> currently 1.4) -- there, I'll have to port Numpy's compatibility
>> functions to our code base.  But for master, I'd rather just use Numpy's
>> stuff so all of the intricacies of this are handled in one place.
>>
>> (And as a detail, it isn't until Numpy 1.7 that a "CloseFile" function
>> is provided, so even on master, we're stuck copying some code over).
>>
>> Any objections?

Nope.

-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Re: [matplotlib-devel] New plot type idea -- EventRaster

[Todd <tod...@gm...>]

On Mon, Jan 14, 2013 at 1:28 AM, Todd <tod...@gm...> wrote:

> On Mon, Nov 12, 2012 at 6:35 PM, Damon McDougall <
> dam...@gm...> wrote:
>
>> On Mon, Nov 12, 2012 at 2:17 PM, Paul Ivanov <piv...@gm...>
>> wrote:
>> > On Mon, Nov 12, 2012 at 7:36 AM, Michael Droettboom <md...@st...>
>> wrote:
>> >> On 11/11/2012 11:51 PM, Todd wrote:
>> >>> Now that 1.2 is out, can we revisit this?  I would like to get it
>> >>> implemented for the next feature release.
>> >>>
>> >>
>> >> Absolutely.  I think the next step, once you have an implementation,
>> >> would be to submit a pull request and we can all help with a review.
>> >
>> > This hasn't been mentioned yet, but Todd will hopefully find our
>> > developer docs useful:
>> > http://matplotlib.org/devel/index.html
>> >
>> > In particular, there's a section on writing a new pyplot function:
>> >
>> http://matplotlib.org/devel/coding_guide.html#writing-a-new-pyplot-function
>>
>> Thanks for that, Paul.
>>
>> Todd, there's also a section on writing tests for matplotlib on the
>> page Paul pointed out. For a new feature there should be a couple of
>> tests to go with it to make sure everything passes sanity checks.
>>
>> Thanks for spending your time contributing!
>>
>>
>>
> I have completed the plot type, including unit tests and examples (I am
> not an artist so someone else can probably make the examples prettier).
> I've confirmed pep8 compliance and run the code through pyflakes and pylint
> in addition to the unit tests.
>
> It is divided into two parts: an EventCollection class, which is a
> subclass of LineCollection in collections.py, and an eventplot method in
> axes.py (and pyplot, through boilerplate.py) which returns a list of
> EventCollection objects.
>
> I am ready to submit a git pull request now.  However, it depends on
> another git pull request I submitted, titled "add get_segments method to
> collections.LineCollection<https://github.com/matplotlib/matplotlib/pull/1655>".
> I can confirm the changes in this pull request work properly, I have been
> using the new method extensively in my new plot type.
>
> Would it be possible to get this accepted so I can submit the new plot
> type for review?  I am sure there will be lots of changes and cleanups that
> will be required before the new plots can be accepted, while the
> get_segments method is a relatively minor and non-intrusive change.
>
> If you wish to look at the code before the pull request it is in the
> toddrjen/matplotlib github fork, in the eventplot branch.  The main changes
> are to collections.py and axes.py.  The tests are in test_axes.py and a new
> test file test_collections.py. The examples are in eventcollection_demo.py
> and eventplot_demo.py
>
> Thanks a lot for your time.
>

Thanks for getting get_segments in so quickly.

The pull request has been submitted, see "Add EventCollection and
eventplot<https://github.com/matplotlib/matplotlib/pull/1657>
"

Re: [matplotlib-devel] New plot type idea -- EventRaster

[Todd <tod...@gm...>]

On Mon, Nov 12, 2012 at 6:35 PM, Damon McDougall
<dam...@gm...>wrote:

> On Mon, Nov 12, 2012 at 2:17 PM, Paul Ivanov <piv...@gm...> wrote:
> > On Mon, Nov 12, 2012 at 7:36 AM, Michael Droettboom <md...@st...>
> wrote:
> >> On 11/11/2012 11:51 PM, Todd wrote:
> >>> Now that 1.2 is out, can we revisit this?  I would like to get it
> >>> implemented for the next feature release.
> >>>
> >>
> >> Absolutely.  I think the next step, once you have an implementation,
> >> would be to submit a pull request and we can all help with a review.
> >
> > This hasn't been mentioned yet, but Todd will hopefully find our
> > developer docs useful:
> > http://matplotlib.org/devel/index.html
> >
> > In particular, there's a section on writing a new pyplot function:
> >
> http://matplotlib.org/devel/coding_guide.html#writing-a-new-pyplot-function
>
> Thanks for that, Paul.
>
> Todd, there's also a section on writing tests for matplotlib on the
> page Paul pointed out. For a new feature there should be a couple of
> tests to go with it to make sure everything passes sanity checks.
>
> Thanks for spending your time contributing!
>
>
>
I have completed the plot type, including unit tests and examples (I am not
an artist so someone else can probably make the examples prettier).  I've
confirmed pep8 compliance and run the code through pyflakes and pylint in
addition to the unit tests.

It is divided into two parts: an EventCollection class, which is a subclass
of LineCollection in collections.py, and an eventplot method in axes.py
(and pyplot, through boilerplate.py) which returns a list of
EventCollection objects.

I am ready to submit a git pull request now.  However, it depends on
another git pull request I submitted, titled "add get_segments method to
collections.LineCollection<https://github.com/matplotlib/matplotlib/pull/1655>".
I can confirm the changes in this pull request work properly, I have been
using the new method extensively in my new plot type.

Would it be possible to get this accepted so I can submit the new plot type
for review?  I am sure there will be lots of changes and cleanups that will
be required before the new plots can be accepted, while the get_segments
method is a relatively minor and non-intrusive change.

If you wish to look at the code before the pull request it is in the
toddrjen/matplotlib github fork, in the eventplot branch.  The main changes
are to collections.py and axes.py.  The tests are in test_axes.py and a new
test file test_collections.py. The examples are in eventcollection_demo.py
and eventplot_demo.py

Thanks a lot for your time.