matplotlib-devel — matplotlib developers

Re: [matplotlib-devel] RFC: boxplot_enhanced & paired_stats

[Yaroslav H. <sf...@on...>]

Thanks everyone for the feedback.

yeah -- scipy dependency is just a joke, as I said, since only sem is in
use, so would be trivial to 'fix'.

As for where to contribute unless to matplotlib -- I would have just
sticked it in our own PyMVPA ;)  but I think those would be generally
useful, so I will just try to cook up a proper PR against matplotlib
some time next week and then go from there.

Cheers!

On Fri, 16 Nov 2012, Paul Hobson wrote:
>      [4]http://nbviewer.ipython.org/url/www.onerussian.com/tmp/run_plots.ipynb

>      > I wonder if there is a need/place for it in matplotlib and what
>      changes would
>      > you advise. Sorry for the lack of documentation -- I guess I have not
>      finished
>      > it at that point (scipy dependency can easily be dropped, used only
>      for
>      > standard error function iirc):
>      Looks nice. We'd certainly be interesting in including it in
>      statsmodels/graphics if there isn't sufficient interest here and/or
>      you'd like to keep the scipy dependency. ;)
>    I was going to suggest either the same thing or adding it to pandas. I
>    think statsmodels if the better fit, though. I also noticed scipy is only
>    used for scipy.stats.sem -- so it might be easy enough to loose the scipy
>    dependency. Just a thought.
-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: [matplotlib-devel] OpenGL backend with Galry

[Cyrille R. <cyr...@gm...>]

>
> Yup, it's a bit of a hack right now b/c you need to merge several
>> branches and tools that are still in review, but it's not too bad.
>>
>> You need to start from this branch:
>>
>> https://github.com/ellisonbg/ipython/tree/jsonhandlers
>>
>> and then grab this repo:
>>
>> https://github.com/ipython/jsplugins
>>
>> I would start by testing the d3graph plugin and verify that you can do
>> what I show here (watch ~ 40 seconds):
>>
>> http://www.youtube.com/watch?v=F4rFuIb1Ie4&t=40m0s
>>
>> That should give you the basics.  Then the webgl visualizer example is
>> here:
>>
>> https://github.com/RishiRamraj/seepymol
>>
>
OK so I now have a very experimental proof of concept of how integrating
Galry in the IPython notebook. There's a short demo here:
http://www.youtube.com/watch?v=taN4TobRS-E

I'll put the code on github but there's of course much more to do.

I'll also work on a basic matplotlib-like high-level interface that will
work in both standard python/ipython consoles, and in the IPython notebook.

Cheers,
Cyrille

Re: [matplotlib-devel] OpenGL backend with Galry

[Cyrille R. <cyr...@gm...>]

>
> Yup, it's a bit of a hack right now b/c you need to merge several
> branches and tools that are still in review, but it's not too bad.
>
> You need to start from this branch:
>
> https://github.com/ellisonbg/ipython/tree/jsonhandlers
>
> and then grab this repo:
>
> https://github.com/ipython/jsplugins
>
> I would start by testing the d3graph plugin and verify that you can do
> what I show here (watch ~ 40 seconds):
>
> http://www.youtube.com/watch?v=F4rFuIb1Ie4&t=40m0s
>
> That should give you the basics.  Then the webgl visualizer example is
> here:
>
> https://github.com/RishiRamraj/seepymol
>
> Cheers,
>
> f
>

Great, thanks! I'll take a look to that and get back to you when I have
something.

Cheers,
Cyrille

Re: [matplotlib-devel] OpenGL backend with Galry

[Fernando P. <fpe...@gm...>]

Hi Cyrille,

On Fri, Nov 16, 2012 at 1:00 PM, Cyrille Rossant
<cyr...@gm...> wrote:
> Hi Fernando,
>
> It would be really great if galry could be integrated in the notebook
> indeed. Is the code of this demo available somewhere, so that I can get an
> idea about how this integration works?
>
> In theory, galry should be compatible with WebGL because one of the main
> components of galry is a shader code generator that can produce OpenGL
> ES-compatible GLSL code. Apart from that, I suppose you have some way of
> making Javascript and Python communicate? The interaction system of Galry,
> which is based on QT but with an abstraction layer, could then be plugged to
> Javascript somehow... Anyway, if I could take a look to the code of this
> demo, I should be able to evaluate how complicated this integration would
> be.

Yup, it's a bit of a hack right now b/c you need to merge several
branches and tools that are still in review, but it's not too bad.

You need to start from this branch:

https://github.com/ellisonbg/ipython/tree/jsonhandlers

and then grab this repo:

https://github.com/ipython/jsplugins

I would start by testing the d3graph plugin and verify that you can do
what I show here (watch ~ 40 seconds):

http://www.youtube.com/watch?v=F4rFuIb1Ie4&t=40m0s

That should give you the basics.  Then the webgl visualizer example is here:

https://github.com/RishiRamraj/seepymol

Cheers,

f

Re: [matplotlib-devel] OpenGL backend with Galry

[Cyrille R. <cyr...@gm...>]

Hi Fernando,

It would be really great if galry could be integrated in the
notebook indeed. Is the code of this demo available somewhere, so that I
can get an idea about how this integration works?

In theory, galry should be compatible with WebGL because one of the main
components of galry is a shader code generator that can produce OpenGL
ES-compatible GLSL code. Apart from that, I suppose you have some way of
making Javascript and Python communicate? The interaction system of Galry,
which is based on QT but with an abstraction layer, could then be plugged
to Javascript somehow... Anyway, if I could take a look to the code of this
demo, I should be able to evaluate how complicated this integration would
be.

Cyrille

2012/11/16 Fernando Perez <fpe...@gm...>

> Hi Cyrille,
>
> On Thu, Nov 15, 2012 at 11:24 AM, Cyrille Rossant
> <cyr...@gm...> wrote:
> > I am developing a high-performance interactive visualization package in
> > Python based on PyOpenGL (http://rossant.github.com/galry/). It is
> primarily
> > meant to be used as a framework for developing complex interactive GUIs
> (in
> > QT) that deal with very large amounts of data (tens of millions of
> points).
> > But it may also be used, like matplotlib, as a high-level interactive
> > library to plot and visualize data.
>
> quick question: how easy/feasible is WebGL integration?  I ask b/c
> we're starting to get the necessary machinery for easy WebGL
> visualization in the ipython notebook, see e.g.:
>
> http://www.flickr.com/photos/47156828@N06/8183294725
>
> so bringing galry to the notebook with minimal code duplication would
> be great.  I just mention it now in case it helps you make design
> decisions as you go along.
>
> Cheers,
>
> f
>

Re: [matplotlib-devel] OpenGL backend with Galry

[Fernando P. <fpe...@gm...>]

Hi Cyrille,

On Thu, Nov 15, 2012 at 11:24 AM, Cyrille Rossant
<cyr...@gm...> wrote:
> I am developing a high-performance interactive visualization package in
> Python based on PyOpenGL (http://rossant.github.com/galry/). It is primarily
> meant to be used as a framework for developing complex interactive GUIs (in
> QT) that deal with very large amounts of data (tens of millions of points).
> But it may also be used, like matplotlib, as a high-level interactive
> library to plot and visualize data.

quick question: how easy/feasible is WebGL integration?  I ask b/c
we're starting to get the necessary machinery for easy WebGL
visualization in the ipython notebook, see e.g.:

http://www.flickr.com/photos/47156828@N06/8183294725

so bringing galry to the notebook with minimal code duplication would
be great.  I just mention it now in case it helps you make design
decisions as you go along.

Cheers,

f

Re: [matplotlib-devel] RFC: boxplot_enhanced & paired_stats

[Paul H. <pmh...@gm...>]

On Fri, Nov 16, 2012 at 7:58 AM, Skipper Seabold <jss...@gm...>wrote:

> On Fri, Nov 16, 2012 at 10:19 AM, Yaroslav Halchenko <sf...@on...>
> wrote:
> > I just found some code (http://www.onerussian.com/tmp/plots.py and
> > pasted below for review/feedback) laying around which I wrote around
> > matplotlib for plotting primarily pair-wise stats results.  Here is a
> > demonstration:
> > http://nbviewer.ipython.org/url/www.onerussian.com/tmp/run_plots.ipynb
> >
> > I wonder if there is a need/place for it in matplotlib and what changes
> would
> > you advise. Sorry for the lack of documentation -- I guess I have not
> finished
> > it at that point (scipy dependency can easily be dropped, used only for
> > standard error function iirc):
> >
>
> Looks nice. We'd certainly be interesting in including it in
> statsmodels/graphics if there isn't sufficient interest here and/or
> you'd like to keep the scipy dependency. ;)
>
> Skipper


I was going to suggest either the same thing or adding it to pandas. I
think statsmodels if the better fit, though. I also noticed scipy is only
used for scipy.stats.sem -- so it might be easy enough to loose the scipy
dependency. Just a thought.
-paul

Re: [matplotlib-devel] RFC: boxplot_enhanced & paired_stats

[Skipper S. <jss...@gm...>]

On Fri, Nov 16, 2012 at 10:19 AM, Yaroslav Halchenko <sf...@on...> wrote:
> I just found some code (http://www.onerussian.com/tmp/plots.py and
> pasted below for review/feedback) laying around which I wrote around
> matplotlib for plotting primarily pair-wise stats results.  Here is a
> demonstration:
> http://nbviewer.ipython.org/url/www.onerussian.com/tmp/run_plots.ipynb
>
> I wonder if there is a need/place for it in matplotlib and what changes would
> you advise. Sorry for the lack of documentation -- I guess I have not finished
> it at that point (scipy dependency can easily be dropped, used only for
> standard error function iirc):
>

Looks nice. We'd certainly be interesting in including it in
statsmodels/graphics if there isn't sufficient interest here and/or
you'd like to keep the scipy dependency. ;)

Skipper

> #!/usr/bin/python
> #emacs: -*- mode: python-mode; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-
> #ex: set sts=4 ts=4 sw=4 noet:
> #------------------------- =+- Python script -+= -------------------------
> """
>  @file      paired-plots.py
>  @date      Fri Jan 13 11:48:00 2012
>  @brief
>
>
>   Yaroslav Halchenko                                            Dartmouth
>   web:     http://www.onerussian.com                              College
>   e-mail:  yo...@on...                              ICQ#: 60653192
>
>  DESCRIPTION (NOTES):
>
>  COPYRIGHT: Yaroslav Halchenko 2012
>
>  LICENSE: MIT
>
>   Permission is hereby granted, free of charge, to any person obtaining a copy
>   of this software and associated documentation files (the "Software"), to deal
>   in the Software without restriction, including without limitation the rights
>   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>   copies of the Software, and to permit persons to whom the Software is
>   furnished to do so, subject to the following conditions:
>
>   The above copyright notice and this permission notice shall be included in
>   all copies or substantial portions of the Software.
>
>   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>   THE SOFTWARE.
> """
> #-----------------\____________________________________/------------------
>
> __author__ = 'Yaroslav Halchenko'
> __revision__ = '$Revision: $'
> __date__ = '$Date:  $'
> __copyright__ = 'Copyright (c) 2012 Yaroslav Halchenko'
> __license__ = 'MIT'
>
>
> import numpy as np
> import pylab as pl
> import scipy.stats as ss
>
> def plot_boxplot_enhanced(
>     v,
>     contrast_labels=None,
>     condition_labels=None,
>     ccolors=['y', 'b'],
>     rand_offsets=None,
>     grid=True,
>     xticks_rotation=0,
>     **bp_kwargs):
>
>     width = bp_kwargs.get('width', 0.5)
>     pl.boxplot(v, **bp_kwargs)
>
>     if v.ndim < 2: v = v[:, None]
>     ncol = v.shape[1]
>
>     eff = np.mean(v, axis=0)          # effect sizes
>     sem = ss.sem(v, axis=0)
>
>     if rand_offsets is None:
>         rand_offsets = np.random.randn(len(v)) * 0.02
>
>     pl.plot((np.arange(ncol) + 1)[:, None] + rand_offsets,
>             v.T, '.', color='k', markerfacecolor='k')
>     for i in range(ncol):
>         lw = 2
>         pl.plot([1 - width/2. + i, 1+i],
>                 [0, 0],
>                 '--', color=ccolors[0], linewidth=lw) # first condition
>         pl.plot([1+i, 1 + width/2. +i],
>                 [eff[i]]*2,
>                 '--', color=ccolors[1], linewidth=lw)
>
>         # place ste
>         pl.errorbar(i+1 + 1.1*width/2.,
>                     eff[i],
>                     sem[i],
>                     elinewidth=2, linewidth=0,
>                     color='r', ecolor='r')
>
>         if contrast_labels and not i:                    # only for the first one
>             pl.text(1 - 1.1*width/2 + i, 0.1, contrast_labels[0],
>                     verticalalignment='bottom',
>                     horizontalalignment='right')
>             pl.text(1 + 1.2*width/2 + i, eff[i], contrast_labels[1],
>                     verticalalignment='bottom', horizontalalignment='left')
>     ax = pl.gca()
>     if condition_labels:
>         ax.set_xticklabels(condition_labels, rotation=xticks_rotation)
>     else:
>         # hide it
>         ax.axes.xaxis.set_visible(False)
>
>     if grid:
>         ax.grid()
>     return ax
>
>
> def plot_paired_stats(
>     v0, v1, contrast_labels,
>     condition_labels=None,
>     style=['barplot_effect',
>            'boxplot_raw',
>            'boxplot_effect'],
>     ccolors=['y', 'g'],
>     xticks_rotation=0,
>     grid=False,
>     fig=None,
>     bottom_adjust=None,
>     bp_kwargs={}):
>
>     if isinstance(style, str):
>         style = [style]
>
>     nplots = len(style)                 # how many subplots will be needed
>
>     # assure having 2nd dimension
>     if v0.ndim < 2: v0 = v0[:, None]
>     if v1.ndim < 2: v1 = v1[:, None]
>     assert(v0.shape == v1.shape)
>
>     ncol = v0.shape[1]
>     v10 = (v1 - v0)                     # differences
>     mv0 = np.mean(v0, axis=0)           # means
>     mv1 = np.mean(v1, axis=0)
>
>     eff = np.mean(v10, axis=0)          # effect sizes
>     sem = ss.sem(v10, axis=0)
>
>     # so that data points have are distinguishable
>     rand_offsets = np.random.randn(len(v10)) * 0.02
>
>     # interleaved combination for some plots
>     v_ = np.hstack((v0, v1))
>     v = np.zeros(v_.shape, dtype=v_.dtype)
>     v[:, np.hstack((np.arange(0, ncol*2, 2),
>                     np.arange(1, ncol*2, 2)))] = v_
>
>     #print v.shape
>     #print np.mean(v0, axis=0), np.mean(v1, axis=0)
>     #print np.min(v10, axis=0), np.max(v10, axis=0), \
>     #      np.mean(v10, axis=0), ss.sem(v10, axis=0)
>     #pl.boxplot(v10 + np.mean(v1), notch=1, widths=0.05)
>
>     #print v0.shape, v1.shape, np.hstack([v0, v1]).shape
>
>     if fig is None:
>         fig = pl.figure()
>
>     bwidth = 0.5
>     plot = 1
>
>     if condition_labels:
>         xlabels = [ '%s:%s' % (cond, contr)
>                     for cond in condition_labels
>                     for contr in contrast_labels ]
>     else:
>         xlabels = contrast_labels
>
>     bp_kwargs_ = {
>         #'bootstrap': 0,
>         'notch' : 1
>         }
>     bp_kwargs_.update(bp_kwargs)
>
>     def plot_grid(ax):
>         if grid:
>             ax.grid()
>
>     if 'barplot_effect' in style:
>         if len(style) > 1:
>             pl.subplot(1, nplots, plot)
>         plot += 1
>         # The simplest one
>         pl.bar(np.arange(1, ncol*2+1) - bwidth/2,
>                np.mean(v, axis=0),
>                color=ccolors*ncol,
>                edgecolor=ccolors*ncol,
>                alpha=0.8,
>                width=bwidth)
>         #pl.minorticks_off()
>         pl.tick_params('x', direction='out', length=6, width=1,
>                        top=False)
>         ax = pl.gca()
>         pl.xlim(0.5, ncol*2+0.5)
>         ax.set_xticks(np.arange(1, ncol*2+1))
>         ax.set_xticklabels(xlabels, rotation=xticks_rotation)
>         # place ste for effect size into the 2nd column
>         pl.errorbar(np.arange(ncol)*2+2,
>                     mv1,
>                     sem, elinewidth=2, linewidth=0,
>                     color='g', ecolor='r')
>
>         plot_grid(ax)
>
>     if 'boxplot_raw' in style:
>         if len(style) > 1:
>             pl.subplot(1, nplots, plot)
>         plot += 1
>
>         # Figure 1 -- "raw" data
>         # plot "connections" between boxplots
>         for i in range(ncol):
>             pargs = (np.array([i*2+1, i*2+2])[:, None] + rand_offsets,
>                      np.array([v0[:,i], v1[:,i]]))
>             pl.plot(*(pargs+('-',)), color='k', alpha=0.5, linewidth=0.25)
>             pl.plot(*(pargs+('.',)), color='k', alpha=0.9)
>         # boxplot of "raw" data
>         bp1 = pl.boxplot(v, widths=bwidth, **bp_kwargs_)
>         for i in range(ncol):
>             for c in xrange(2):
>                 b = bp1['boxes'][2*i+c]
>                 b.set_color(ccolors[c])
>                 b.set_linewidth(2)
>
>         ax = pl.gca()
>         ax.set_xticklabels(xlabels, rotation=xticks_rotation)
>         plot_grid(ax)
>
>     if 'boxplot_effect' in style:
>         if len(style) > 1:
>             pl.subplot(1, nplots, plot)
>         plot += 1
>         plot_boxplot_enhanced(v10,
>                               contrast_labels=contrast_labels,
>                               condition_labels=condition_labels,
>                               widths=bwidth,
>                               rand_offsets=rand_offsets, # reuse them
>                               grid=grid,
>                               **bp_kwargs_)
>
>     if bottom_adjust:
>         fig.subplots_adjust(bottom=bottom_adjust)
>     pl.draw_if_interactive()
>     return fig
>
> if __name__ == '__main__':
>
>     if True:
>         v = np.random.normal(size=(50,8)) * 20 + 120
>         if False:
>             v[:, 1] += 40
>             v[:, 3] -= 30
>             v[:, 5] += 60
>             v[:, 6] -= 60
>         else:
>             v -= np.arange(v.shape[1])*10
>         v /= 10
>
>     v0 = v[:, ::2]
>     v1 = v[:, 1::2]
>     d = v1 - v0
>     print np.mean(d, axis=0)
>     styles = ['barplot_effect',
>               'boxplot_raw',
>               'boxplot_effect'
>               ]
>     styles = styles + [styles]
>     pl.close('all')
>
>     if False:
>         f = plot_boxplot_enhanced((v1-v0)[:,0],
>                                   grid=True, xticks_rotation=30, notch=1)
>
>     for s in styles:
>         fig = pl.figure(figsize=(12,6))
>         f = plot_paired_stats(v0, v1, ['cont1', 'cont2'],
>                               style=s, fig=fig,
>                               condition_labels=['exp1', 'exp2', 'exp3', 'exp4'],
>                               grid=True, xticks_rotation=30)
>     pl.show()
>
>
> --
> Yaroslav O. Halchenko
> Postdoctoral Fellow,   Department of Psychological and Brain Sciences
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

[matplotlib-devel] RFC: boxplot_enhanced & paired_stats

[Yaroslav H. <sf...@on...>]

I just found some code (http://www.onerussian.com/tmp/plots.py and
pasted below for review/feedback) laying around which I wrote around
matplotlib for plotting primarily pair-wise stats results.  Here is a
demonstration:
http://nbviewer.ipython.org/url/www.onerussian.com/tmp/run_plots.ipynb

I wonder if there is a need/place for it in matplotlib and what changes would
you advise. Sorry for the lack of documentation -- I guess I have not finished
it at that point (scipy dependency can easily be dropped, used only for
standard error function iirc):

#!/usr/bin/python
#emacs: -*- mode: python-mode; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*- 
#ex: set sts=4 ts=4 sw=4 noet:
#------------------------- =+- Python script -+= -------------------------
"""
 @file      paired-plots.py
 @date      Fri Jan 13 11:48:00 2012
 @brief


  Yaroslav Halchenko                                            Dartmouth
  web:     http://www.onerussian.com                              College
  e-mail:  yo...@on...                              ICQ#: 60653192

 DESCRIPTION (NOTES):

 COPYRIGHT: Yaroslav Halchenko 2012

 LICENSE: MIT

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in
  all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.
"""
#-----------------\____________________________________/------------------

__author__ = 'Yaroslav Halchenko'
__revision__ = '$Revision: $'
__date__ = '$Date:  $'
__copyright__ = 'Copyright (c) 2012 Yaroslav Halchenko'
__license__ = 'MIT'


import numpy as np
import pylab as pl
import scipy.stats as ss

def plot_boxplot_enhanced(
    v,
    contrast_labels=None,
    condition_labels=None,
    ccolors=['y', 'b'],
    rand_offsets=None,
    grid=True,
    xticks_rotation=0,
    **bp_kwargs):

    width = bp_kwargs.get('width', 0.5)
    pl.boxplot(v, **bp_kwargs)

    if v.ndim < 2: v = v[:, None]
    ncol = v.shape[1]

    eff = np.mean(v, axis=0)          # effect sizes
    sem = ss.sem(v, axis=0)

    if rand_offsets is None:
        rand_offsets = np.random.randn(len(v)) * 0.02

    pl.plot((np.arange(ncol) + 1)[:, None] + rand_offsets,
            v.T, '.', color='k', markerfacecolor='k')
    for i in range(ncol):
        lw = 2
        pl.plot([1 - width/2. + i, 1+i],
                [0, 0],
                '--', color=ccolors[0], linewidth=lw) # first condition
        pl.plot([1+i, 1 + width/2. +i],
                [eff[i]]*2,
                '--', color=ccolors[1], linewidth=lw)

        # place ste
        pl.errorbar(i+1 + 1.1*width/2.,
                    eff[i],
                    sem[i],
                    elinewidth=2, linewidth=0,
                    color='r', ecolor='r')

        if contrast_labels and not i:                    # only for the first one
            pl.text(1 - 1.1*width/2 + i, 0.1, contrast_labels[0],
                    verticalalignment='bottom',
                    horizontalalignment='right')
            pl.text(1 + 1.2*width/2 + i, eff[i], contrast_labels[1],
                    verticalalignment='bottom', horizontalalignment='left')
    ax = pl.gca()
    if condition_labels:
        ax.set_xticklabels(condition_labels, rotation=xticks_rotation)
    else:
        # hide it
        ax.axes.xaxis.set_visible(False)

    if grid:
        ax.grid()
    return ax


def plot_paired_stats(
    v0, v1, contrast_labels,
    condition_labels=None,
    style=['barplot_effect',
           'boxplot_raw',
           'boxplot_effect'],
    ccolors=['y', 'g'],
    xticks_rotation=0,
    grid=False,
    fig=None,
    bottom_adjust=None,
    bp_kwargs={}):

    if isinstance(style, str):
        style = [style]

    nplots = len(style)                 # how many subplots will be needed

    # assure having 2nd dimension
    if v0.ndim < 2: v0 = v0[:, None]
    if v1.ndim < 2: v1 = v1[:, None]
    assert(v0.shape == v1.shape)

    ncol = v0.shape[1]
    v10 = (v1 - v0)                     # differences
    mv0 = np.mean(v0, axis=0)           # means
    mv1 = np.mean(v1, axis=0)

    eff = np.mean(v10, axis=0)          # effect sizes
    sem = ss.sem(v10, axis=0)

    # so that data points have are distinguishable
    rand_offsets = np.random.randn(len(v10)) * 0.02

    # interleaved combination for some plots
    v_ = np.hstack((v0, v1))
    v = np.zeros(v_.shape, dtype=v_.dtype)
    v[:, np.hstack((np.arange(0, ncol*2, 2),
                    np.arange(1, ncol*2, 2)))] = v_

    #print v.shape
    #print np.mean(v0, axis=0), np.mean(v1, axis=0)
    #print np.min(v10, axis=0), np.max(v10, axis=0), \
    #      np.mean(v10, axis=0), ss.sem(v10, axis=0)
    #pl.boxplot(v10 + np.mean(v1), notch=1, widths=0.05)

    #print v0.shape, v1.shape, np.hstack([v0, v1]).shape

    if fig is None:
        fig = pl.figure()

    bwidth = 0.5
    plot = 1

    if condition_labels:
        xlabels = [ '%s:%s' % (cond, contr)
                    for cond in condition_labels
                    for contr in contrast_labels ]
    else:
        xlabels = contrast_labels

    bp_kwargs_ = {
        #'bootstrap': 0,
        'notch' : 1
        }
    bp_kwargs_.update(bp_kwargs)

    def plot_grid(ax):
        if grid:
            ax.grid()

    if 'barplot_effect' in style:
        if len(style) > 1:
            pl.subplot(1, nplots, plot)
        plot += 1
        # The simplest one
        pl.bar(np.arange(1, ncol*2+1) - bwidth/2,
               np.mean(v, axis=0),
               color=ccolors*ncol,
               edgecolor=ccolors*ncol,
               alpha=0.8,
               width=bwidth)
        #pl.minorticks_off()
        pl.tick_params('x', direction='out', length=6, width=1,
                       top=False)
        ax = pl.gca()
        pl.xlim(0.5, ncol*2+0.5)
        ax.set_xticks(np.arange(1, ncol*2+1))
        ax.set_xticklabels(xlabels, rotation=xticks_rotation)
        # place ste for effect size into the 2nd column
        pl.errorbar(np.arange(ncol)*2+2,
                    mv1,
                    sem, elinewidth=2, linewidth=0,
                    color='g', ecolor='r')

        plot_grid(ax)

    if 'boxplot_raw' in style:
        if len(style) > 1:
            pl.subplot(1, nplots, plot)
        plot += 1

        # Figure 1 -- "raw" data
        # plot "connections" between boxplots
        for i in range(ncol):
            pargs = (np.array([i*2+1, i*2+2])[:, None] + rand_offsets,
                     np.array([v0[:,i], v1[:,i]]))
            pl.plot(*(pargs+('-',)), color='k', alpha=0.5, linewidth=0.25)
            pl.plot(*(pargs+('.',)), color='k', alpha=0.9)
        # boxplot of "raw" data
        bp1 = pl.boxplot(v, widths=bwidth, **bp_kwargs_)
        for i in range(ncol):
            for c in xrange(2):
                b = bp1['boxes'][2*i+c]
                b.set_color(ccolors[c])
                b.set_linewidth(2)

        ax = pl.gca()
        ax.set_xticklabels(xlabels, rotation=xticks_rotation)
        plot_grid(ax)

    if 'boxplot_effect' in style:
        if len(style) > 1:
            pl.subplot(1, nplots, plot)
        plot += 1
        plot_boxplot_enhanced(v10,
                              contrast_labels=contrast_labels,
                              condition_labels=condition_labels,
                              widths=bwidth,
                              rand_offsets=rand_offsets, # reuse them
                              grid=grid,
                              **bp_kwargs_)

    if bottom_adjust:
        fig.subplots_adjust(bottom=bottom_adjust)
    pl.draw_if_interactive()
    return fig

if __name__ == '__main__':

    if True:
        v = np.random.normal(size=(50,8)) * 20 + 120
        if False:
            v[:, 1] += 40
            v[:, 3] -= 30
            v[:, 5] += 60
            v[:, 6] -= 60
        else:
            v -= np.arange(v.shape[1])*10
        v /= 10

    v0 = v[:, ::2]
    v1 = v[:, 1::2]
    d = v1 - v0
    print np.mean(d, axis=0)
    styles = ['barplot_effect',
              'boxplot_raw',
              'boxplot_effect'
              ]
    styles = styles + [styles]
    pl.close('all')

    if False:
        f = plot_boxplot_enhanced((v1-v0)[:,0],
                                  grid=True, xticks_rotation=30, notch=1)

    for s in styles:
        fig = pl.figure(figsize=(12,6))
        f = plot_paired_stats(v0, v1, ['cont1', 'cont2'],
                              style=s, fig=fig,
                              condition_labels=['exp1', 'exp2', 'exp3', 'exp4'],
                              grid=True, xticks_rotation=30)
    pl.show()


-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: [matplotlib-devel] tick_params direction 'inout'

[Benjamin R. <ben...@ou...>]

On Fri, Nov 16, 2012 at 7:44 AM, Mike Kaufman <mc...@gm...> wrote:

> On 11/15/12 2:54 PM, Paul Ivanov wrote:
> > On Wed, Nov 14, 2012 at 7:37 PM, Mike Kaufman <mc...@gm...
>
> >
> >     I just found that *direction* accepts 'inout' as well, which
> >     does indeed place the tick on both sides of the spine. So the
> >     documentation should be updated to reflect this.
> >
> >
> > Thanks for the report, Mike, here's a PR for the patch:
> > https://github.com/matplotlib/matplotlib/pull/1503
> >
> >     If it were me, I'd allow 'both' to work as well.
> >
> >
> > I'm amenable to that, just not sure if that counts as a new feature and
> > should go into master, or a bugfix and go into v1.2.x.
> >
> > I can add this functionality to #1503 if that makes sense to go to v1.2.x
>
> Thanks Paul,
>
> One is a doc fix, the other requires changing code (inside axis.py?). So
> best practice probably argues that 'both', if acceptable, should really
> only go into master.
>
> That said, I think the 'both' change only requires a change to a pair of
> conditionals...
>
> M
>
>
I agree with the fact that implementing "both" to mean the same as "inout"
should go into master only.

Ben Root

Re: [matplotlib-devel] tick_params direction 'inout'

[Mike K. <mc...@gm...>]

On 11/15/12 2:54 PM, Paul Ivanov wrote:
> On Wed, Nov 14, 2012 at 7:37 PM, Mike Kaufman <mc...@gm...

>
>     I just found that *direction* accepts 'inout' as well, which
>     does indeed place the tick on both sides of the spine. So the
>     documentation should be updated to reflect this.
>
>
> Thanks for the report, Mike, here's a PR for the patch:
> https://github.com/matplotlib/matplotlib/pull/1503
>
>     If it were me, I'd allow 'both' to work as well.
>
>
> I'm amenable to that, just not sure if that counts as a new feature and
> should go into master, or a bugfix and go into v1.2.x.
>
> I can add this functionality to #1503 if that makes sense to go to v1.2.x

Thanks Paul,

One is a doc fix, the other requires changing code (inside axis.py?). So 
best practice probably argues that 'both', if acceptable, should really 
only go into master.

That said, I think the 'both' change only requires a change to a pair of 
conditionals...

M

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[Ian T. <ian...@gm...>]

On 16 November 2012 05:14, Damon McDougall <dam...@gm...>wrote:

> >> I have a C++ TriFinder class
> >> that I could modify to work within matplotlib, and it is O(log N) so
> should
> >> be faster than your version for typical use cases.
> >
> > What algorithm does this use? Is the code open source and/or availabel
> > for other projects?
>
> I'm pretty sure there is an O(log n) algorithm in the Numerical
> Recipes book. It requires you to construct the triangulation in a
> specific way (this allows one to set up a tree data structure of
> triangles nicely). There may be others that I am not aware of though.
>

I think this is the standard method used for point-in-triangulation tests
for delaunay triangulations, as the search structure is created as the
triangulation is built.  We need to support non-delaunay triangulations
that are specified by the user, which requires a different approach.

Ian

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[Ian T. <ian...@gm...>]

On 15 November 2012 21:25, Chris Barker <chr...@no...> wrote:

> On Wed, Nov 14, 2012 at 1:50 AM, Ian Thomas <ian...@gm...> wrote:
>
> > I think the code used to determine which triangle contains a certain
> point
> > should be factored out into its own TriFinder class,
>
> +1 -- this is a generally useful feature. In fact, it would be nice if
> a lot of this were in a pacakge that deals with triangular meshes,
> apart from MPL altogether (a scikit maybe?)
>

I hadn't considered any wider interest in it.  From a selfish matplotlib
point of view, even if it was in say a scikit, we wouldn't want to add an
external dependency to the scikit so we would still need a copy of the
source code within matplotlib anyway, just to do our interpolation for
plotting purposes.  The situation would be just like the delaunay code
which exists in a scikit but we include it in the mpl source code to avoid
the external dependency.

Once it is in mpl it is available for other projects to use if they wish,
subject to the BSD-style license.  There would be a question of who is
responsible for maintaining it as I don't particularly want to spread
myself thinly on other open source projects when there are a lot of
additions to mpl that I would like to do.  There would be the danger for
the other project of the reverse situation of our delaunay example, where
we have code that needs improvement but it is essentially unmaintained,
causing problems for users and embarrassment for developers.

> I have a C++ TriFinder class
> > that I could modify to work within matplotlib, and it is O(log N) so
> should
> > be faster than your version for typical use cases.
>
> What algorithm does this use? Is the code open source and/or availabel
> for other projects?
>
> I'm working on a package for working with unstructured grids in
> general, and also have a use for "what triangle is this point in" code
> for other purposes -- and I havne't found a fast, robust code for this
> yet.
>

It is code I have written recently based on the trapezoidal map algorithm
from the book 'Computational Geometry: Algorithms and Applications' by M.
de Berg et al.  It currently exists only on my main linux box, but it will
be open source within mpl for others to use as mentioned above (subject to
acceptance by the mpl developers of course).  It is an excellent book that
I wholeheartedly recommend to anyone with a passing interest in
computational geometry.  The algorithm itself looks painful initially (as
do many of the algorithms in the book) but it reduces to a surprisingly
small amount of code.

For well-formed triangulations (i.e. no duplicate points, no zero area
triangles and no overlapping triangles) the algorithm is 'sufficiently'
robust.  It is not completely robust in the sense of Shewchuk's robust
predicates, but is designed quite cleverly (or luckily) to avoid many of
the pitfalls that occur in naive point-in-triangle tests.  For example, a
naive implementation of a point-in-triangle test for a point on or very
near the common edge of two adjacent triangles might return true for both
triangles (which is usually fine), or false for both (which is
catastrophic).  The trapezoidal map avoids these problems by reducing the
test to a single point-line test which is therefore guaranteed to return
just one of the two triangles.  It is possible to think of a situation that
causes the algorithm to fail at a triangle which has such a small area that
the slopes of two of the sides are within the machine precision of each
other, but it is also possible to use the triangle connectivity to check
for and resolve this problem.  I haven't done so yet and need to consider
the tradeoff of effort required vs potential gain.

Someone who required guaranteed robustness could use Shewchuk's point-line
test with the algorithm.  I would not do this with mpl however, as the
correctness of the point-line test depends a lot on computer architecture
and compiler flags.  This would get out of date quickly and would need keen
monitoring by someone with knowledge of and interest in the area, plus
access to a lot of different computer architectures and OSes.  I fail on
all of those counts!


> >> particularly as only a few days ago I committed to writing a triangular
> grid
> >> interpolator for quad grids
>
> what is a triangular interpolator for quad grids? sounds useful, too.


That was poor English from me.  I meant interpolating from a triangular
grid *to* a quad grid, typically to make use of the wide range of quad grid
plotting functions like contour, pcolor, etc.

Ian

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[Damon M. <dam...@gm...>]

On Thu, Nov 15, 2012 at 3:25 PM, Chris Barker <chr...@no...> wrote:
> On Wed, Nov 14, 2012 at 1:50 AM, Ian Thomas <ian...@gm...> wrote:
>
>> I think the code used to determine which triangle contains a certain point
>> should be factored out into its own TriFinder class,
>
> +1 -- this is a generally useful feature. In fact, it would be nice if
> a lot of this were in a pacakge that deals with triangular meshes,
> apart from MPL altogether (a scikit maybe?)
>
>> I have a C++ TriFinder class
>> that I could modify to work within matplotlib, and it is O(log N) so should
>> be faster than your version for typical use cases.
>
> What algorithm does this use? Is the code open source and/or availabel
> for other projects?

I'm pretty sure there is an O(log n) algorithm in the Numerical
Recipes book. It requires you to construct the triangulation in a
specific way (this allows one to set up a tree data structure of
triangles nicely). There may be others that I am not aware of though.

>
> I'm working on a package for working with unstructured grids in
> general, and also have a use for "what triangle is this point in" code
> for other purposes -- and I havne't found a fast, robust code for this
> yet.
>
>>> particularly as only a few days ago I committed to writing a triangular grid
>>> interpolator for quad grids
>
> what is a triangular interpolator for quad grids? sounds useful, too.
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chr...@no...
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel



-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Re: [matplotlib-devel] OpenGL backend with Galry

[Cyrille R. <cyr...@gm...>]

OK so it seems that integrating any efficient OpenGL rendering code in
matplotlib as a backend is much more complicated than what I thought.

> I'm guessing with galry, you push the user-coordinates to the graphics
> card, then as the user is interacting, you're changing the transforms
> and re-rendering, but don't need to push the vertex data itself over
> and over again, and with with shaders, you could do arbitrary
> transforms. hence high performance.

You're absolutely right, that's the way high performance is achieved in
Galry. In fact, the low-level interface provides a way to write custom
shaders to implement really anything (interactive transformations or any
rendering effect). Whereas it is possible to push data on the GPU at any
time, the most efficient way of rendering stuff in Galry is to push
everything at the beginning, and then only update uniform shader variables
to implement transforms and dynamic effects.

That does not seem very compatible with the way backends appear to work
then. I fear that in order to have an efficient GL backend, either this
backend system would need to be updated so that it can be directly
connected to transformation events, or one would need to get around the
backend system.

Anyway, it looks like the difficulty would come more from the matplotlib
backend system than the OpenGL part (by the way, your rendering demos are
really cool Nicolas!). It would be great if a GSoC student could work on
this, and I would be happy to help if necessary. In the meantime, I might
write sometime an extremely basic high-level matplotlib-like interface for
Galry, with support in particular for scatter plots, continuous-time
signals, textures, maybe other plot types if anyone asks. It may be useful
until a fully working GL backend becomes available.

Cyrille

2012/11/15 Nicolas Rougier <Nic...@in...>

>
>
> Yep, I'm still developing some OpenGL technics to provide both nice and
> fast rendering and I hope to be able to help the writing of a GL backend
> for matplotlib next summer (provided we get a GSoC student for the project).
>
> So far, my main concern is that for efficient rendering using OpenGL, you
> need to consider that drawing means to create objects on the graphic card
> (line, curve, image, points, etc) that can be later manipulated
> (scaling/rotating/coloring/properties change, etc.). From what I remember
> in my early attempts at writing an OpenGL backend, I did not find the
> proper way to enforce such framework. Said differently, the backend is
> supposed to implement drawing operations while I would need to know if the
> drawing operations actually relates to something that is already on the
> graphic card or not. I'm not sure I'm very clear but I can develop the
> point if necessary. Having read the post by Michael (
> http://mdboom.github.com/blog/2012/08/06/matplotlib-client-side/) on
> client-side rendering, I think the proposed three-way split might be a
> solution but I do not know how advanced are the ideas.
>
> To date, I've been working on different things:
>
> Text/font :       http://code.google.com/p/freetype-gl/ (c code)
> Stroke/dash/paths: http://code.google.com/p/gl-agg/ (python)
> Images:            http://code.google.com/p/glumpy/ (python)
>
> If you want to get a feel of how nice and fast rendering could be, have a
> look at 'demo-lines.py' from the gl-agg repository (and play with mouse).
> From these experiments, I think it is possible to achieve AGG quality using
> OpenGL. What is really exciting is the perspective of having a opengl/webgl
> backend that could be used with ipython (there has been a recent post on
> ipython list that show such integration for a molecule viewer).
>
>
> Anyway, you're more than welcome to contribute to glumpy, but in the long
> run, I hope it will disappear in favor of a matplolib GL backend.
>
>
>
> Nicolas
>
>
>
>
>
>
> On Nov 15, 2012, at 22:03 , Benjamin Root wrote:
>
> >
> >
> > On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant <
> cyr...@gm...> wrote:
> > Hi all,
> >
> > I am developing a high-performance interactive visualization package in
> Python based on PyOpenGL (http://rossant.github.com/galry/). It is
> primarily meant to be used as a framework for developing complex
> interactive GUIs (in QT) that deal with very large amounts of data (tens of
> millions of points). But it may also be used, like matplotlib, as a
> high-level interactive library to plot and visualize data.
> >
> > The low-level interface is mostly done at this point (the code is still
> in an experimental stage though), and I'm now focusing on my current
> research project which is to write a scientific GUI based on this
> interface. However, I think people (including myself!) may be interested in
> a matplotlib-like high-level interface. I was first thinking about writing
> such an interface from scratch, by implementing a very small fraction of
> the matplotlib interface (basic commands like figure(), plot(), subplot(),
> show(), etc.). One could then quickly visualize huge datasets with the same
> commands than matplotlib.
> >
> > Another solution would be to write a matplotlib backend based on this
> library. I am not familar enough with the internals of matplotlib to know
> how complicated it could be. I may do it myself, but it would probably take
> a long time since it is currently not my highest priority. I would be glad
> if someone experienced in writing backends was interested in working on it.
> Actually I could do everything that is specific to my library, which
> already provides commands to plot points, lines, textures, etc. The canvas
> is based on QT and may be similar to what is already implemented in the QT
> backend.
> >
> > Of course, it would already be great if only the most basic plotting
> features were available in the backend. A first step could be for example
> to have a simplistic example "plot(x, sin(x))" working (with interactive
> navigation).
> >
> > I am looking forward to your feedback.
> >
> > Best,
> > Cyrille Rossant
> >
> >
> > Great to hear another person interested in bringing opengl to
> matplotlib!  Another project you might be interested in collaborating with
> is Glumpy: http://code.google.com/p/glumpy/
> >
> > From my limited knowledge of OpenGL, what my vision is that any of the
> existing backends have support for an OpenGL object, so we just need to be
> able to instantiate the opengl object in any figure object, and know how to
> send it the appropriate commands and data.  So, it is not exactly a
> backend, more of a "middling".  Anyway, I think the dev at Glumpy would be
> happy to have help, and probably have much more developed ideas on how to
> integrate with matplotlib.
> >
> > Cheers!
> > Ben Root
> >
> >
> ------------------------------------------------------------------------------
> > Monitor your physical, virtual and cloud infrastructure from a single
> > web console. Get in-depth insight into apps, servers, databases, vmware,
> > SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> > Pricing starts from $795 for 25 servers or applications!
> >
> http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>

Re: [matplotlib-devel] OpenGL backend with Galry

[Nicolas R. <Nic...@in...>]

Yep, I'm still developing some OpenGL technics to provide both nice and fast rendering and I hope to be able to help the writing of a GL backend for matplotlib next summer (provided we get a GSoC student for the project).

So far, my main concern is that for efficient rendering using OpenGL, you need to consider that drawing means to create objects on the graphic card (line, curve, image, points, etc) that can be later manipulated (scaling/rotating/coloring/properties change, etc.). From what I remember in my early attempts at writing an OpenGL backend, I did not find the proper way to enforce such framework. Said differently, the backend is supposed to implement drawing operations while I would need to know if the drawing operations actually relates to something that is already on the graphic card or not. I'm not sure I'm very clear but I can develop the point if necessary. Having read the post by Michael (http://mdboom.github.com/blog/2012/08/06/matplotlib-client-side/) on client-side rendering, I think the proposed three-way split might be a solution but I do not know how advanced are the ideas.

To date, I've been working on different things:

Text/font :       http://code.google.com/p/freetype-gl/ (c code)
Stroke/dash/paths: http://code.google.com/p/gl-agg/ (python)
Images:            http://code.google.com/p/glumpy/ (python)

If you want to get a feel of how nice and fast rendering could be, have a look at 'demo-lines.py' from the gl-agg repository (and play with mouse). From these experiments, I think it is possible to achieve AGG quality using OpenGL. What is really exciting is the perspective of having a opengl/webgl backend that could be used with ipython (there has been a recent post on ipython list that show such integration for a molecule viewer).


Anyway, you're more than welcome to contribute to glumpy, but in the long run, I hope it will disappear in favor of a matplolib GL backend.


 
Nicolas






On Nov 15, 2012, at 22:03 , Benjamin Root wrote:

> 
> 
> On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant <cyr...@gm...> wrote:
> Hi all,
> 
> I am developing a high-performance interactive visualization package in Python based on PyOpenGL (http://rossant.github.com/galry/). It is primarily meant to be used as a framework for developing complex interactive GUIs (in QT) that deal with very large amounts of data (tens of millions of points). But it may also be used, like matplotlib, as a high-level interactive library to plot and visualize data.
> 
> The low-level interface is mostly done at this point (the code is still in an experimental stage though), and I'm now focusing on my current research project which is to write a scientific GUI based on this interface. However, I think people (including myself!) may be interested in a matplotlib-like high-level interface. I was first thinking about writing such an interface from scratch, by implementing a very small fraction of the matplotlib interface (basic commands like figure(), plot(), subplot(), show(), etc.). One could then quickly visualize huge datasets with the same commands than matplotlib.
> 
> Another solution would be to write a matplotlib backend based on this library. I am not familar enough with the internals of matplotlib to know how complicated it could be. I may do it myself, but it would probably take a long time since it is currently not my highest priority. I would be glad if someone experienced in writing backends was interested in working on it. Actually I could do everything that is specific to my library, which already provides commands to plot points, lines, textures, etc. The canvas is based on QT and may be similar to what is already implemented in the QT backend.
> 
> Of course, it would already be great if only the most basic plotting features were available in the backend. A first step could be for example to have a simplistic example "plot(x, sin(x))" working (with interactive navigation).
> 
> I am looking forward to your feedback.
> 
> Best,
> Cyrille Rossant
> 
> 
> Great to hear another person interested in bringing opengl to matplotlib!  Another project you might be interested in collaborating with is Glumpy: http://code.google.com/p/glumpy/
> 
> From my limited knowledge of OpenGL, what my vision is that any of the existing backends have support for an OpenGL object, so we just need to be able to instantiate the opengl object in any figure object, and know how to send it the appropriate commands and data.  So, it is not exactly a backend, more of a "middling".  Anyway, I think the dev at Glumpy would be happy to have help, and probably have much more developed ideas on how to integrate with matplotlib.
> 
> Cheers!
> Ben Root
> 
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[Chris B. <chr...@no...>]

On Wed, Nov 14, 2012 at 1:50 AM, Ian Thomas <ian...@gm...> wrote:

> I think the code used to determine which triangle contains a certain point
> should be factored out into its own TriFinder class,

+1 -- this is a generally useful feature. In fact, it would be nice if
a lot of this were in a pacakge that deals with triangular meshes,
apart from MPL altogether (a scikit maybe?)

> I have a C++ TriFinder class
> that I could modify to work within matplotlib, and it is O(log N) so should
> be faster than your version for typical use cases.

What algorithm does this use? Is the code open source and/or availabel
for other projects?

I'm working on a package for working with unstructured grids in
general, and also have a use for "what triangle is this point in" code
for other purposes -- and I havne't found a fast, robust code for this
yet.

>> particularly as only a few days ago I committed to writing a triangular grid
>> interpolator for quad grids

what is a triangular interpolator for quad grids? sounds useful, too.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [matplotlib-devel] OpenGL backend with Galry

[Benjamin R. <ben...@ou...>]

On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant
<cyr...@gm...>wrote:

> Hi all,
>
> I am developing a high-performance interactive visualization package in
> Python based on PyOpenGL (http://rossant.github.com/galry/). It is
> primarily meant to be used as a framework for developing complex
> interactive GUIs (in QT) that deal with very large amounts of data (tens of
> millions of points). But it may also be used, like matplotlib, as a
> high-level interactive library to plot and visualize data.
>
> The low-level interface is mostly done at this point (the code is still in
> an experimental stage though), and I'm now focusing on my current research
> project which is to write a scientific GUI based on this interface.
> However, I think people (including myself!) may be interested in a
> matplotlib-like high-level interface. I was first thinking about writing
> such an interface from scratch, by implementing a very small fraction of
> the matplotlib interface (basic commands like figure(), plot(), subplot(),
> show(), etc.). One could then quickly visualize huge datasets with the same
> commands than matplotlib.
>
> Another solution would be to write a matplotlib backend based on this
> library. I am not familar enough with the internals of matplotlib to know
> how complicated it could be. I may do it myself, but it would probably take
> a long time since it is currently not my highest priority. I would be glad
> if someone experienced in writing backends was interested in working on it.
> Actually I could do everything that is specific to my library, which
> already provides commands to plot points, lines, textures, etc. The canvas
> is based on QT and may be similar to what is already implemented in the QT
> backend.
>
> Of course, it would already be great if only the most basic plotting
> features were available in the backend. A first step could be for example
> to have a simplistic example "plot(x, sin(x))" working (with interactive
> navigation).
>
> I am looking forward to your feedback.
>
> Best,
> Cyrille Rossant
>
>
Great to hear another person interested in bringing opengl to matplotlib!
Another project you might be interested in collaborating with is Glumpy:
http://code.google.com/p/glumpy/

>From my limited knowledge of OpenGL, what my vision is that any of the
existing backends have support for an OpenGL object, so we just need to be
able to instantiate the opengl object in any figure object, and know how to
send it the appropriate commands and data.  So, it is not exactly a
backend, more of a "middling".  Anyway, I think the dev at Glumpy would be
happy to have help, and probably have much more developed ideas on how to
integrate with matplotlib.

Cheers!
Ben Root

Re: [matplotlib-devel] tick_params direction 'inout'

[Paul I. <piv...@gm...>]

On Wed, Nov 14, 2012 at 7:37 PM, Mike Kaufman <mc...@gm...> wrote:

> Hi all,
>
> I don't have time to make a patch at the moment, but I thought I'd point
> it out for anyone to give it a go...
>
> tick_params(axis='both', **kwargs)
>      Change the appearance of ticks and tick labels.
>
>      Keyword arguments:
>
> ...
>
>      *direction* : ['in' | 'out']
>          Puts ticks inside or outside the axes.
>
> I just found that *direction* accepts 'inout' as well, which
> does indeed place the tick on both sides of the spine. So the
> documentation should be updated to reflect this.


Thanks for the report, Mike, here's a PR for the patch:
https://github.com/matplotlib/matplotlib/pull/1503


> If it were me, I'd allow 'both' to work as well.
>

I'm amenable to that, just not sure if that counts as a new feature and
should go into master, or a bugfix and go into v1.2.x.

I can add this functionality to #1503 if that makes sense to go to v1.2.x

-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[geoffroy b. <geo...@gm...>]

Hello Ian,

Thank you for your second proposition ; I find it very interesting in fact.

(But beware that, as I do not feel able to do this on my own, your code
exemple / guidance would be needed for sure... )
I will have a look at http://matplotlib.org/devel/index.html

I think your idea of having a separate TriFinder class is quite good. My
search algo. is not optimised, only avoiding a O(N) for interpolations
along a path or line.
Some other parts of the code (especially the loop over the (x,y) points in
__call__() ) may also be performance-critical, at least for the 'cubic'
interpolator. But I dont have a clear picture on how to embed C++ code in
python so I would need your example to figure out what is possible.

Regards
Geoffroy.


2012/11/14 Ian Thomas <ian...@gm...>

> Hi Geoffroy,
>
> I have had some time to look at your TriLinearInterpolator in some detail
> (the other two files only briefly).  I would indeed like to add something
> like this to matplotlib - the mesh refinement looks very nice and the
> interpolators would be useful to many people.
>
> As you suspected, the code does need significant changes before we can
> include it.  Some are merely cosmetic, as all code must adhere to PEP8 and
> the matplotlib coding guidelines, but there are also some functional and
> performance improvements.  For example, your wavefront method for finding
> the triangle containing a certain point must be able to deal with masked
> triangulations and indeed triangulations that are discontinuous, for
> example two islands in a masked-out ocean, which is unusual but must be
> supported.  In terms of performance, there is much explicit looping within
> numpy arrays that could be improved using other numpy array commands, and
> would also reduce the length of the source code.  There is an argument for
> some of the performance-critical code to be in C/C++.
>
> I think the code used to determine which triangle contains a certain point
> should be factored out into its own TriFinder class, so that (1) it does
> not need to be replicated in the two interpolator classes, and (2)
> different algorithms can be easily swapped if necessary.  I have a C++
> TriFinder class that I could modify to work within matplotlib, and it is
> O(log N) so should be faster than your version for typical use cases.
>
> I expect that this is probably more work than you anticipated when you
> asked if the code needed any improvement!  I propose the following: if you
> are happy to give matplotlib your source code as it stands and for us to
> include it under our BSD-style license, then I will take on the
> responsibility of getting it into a form that will be accepted by the other
> developers.  I will acknowledge your contribution in both the source code
> and on the web site, something like "based on code contributed by Geoffroy
> Billotey".
>
> Alternatively, if you would like to use this as an excuse to learn how to
> contribute to matplotlib more actively but don't want to take on
> everything, then we could divide up the work so that first I write my C++
> log(N) TriFinder class and the linear interpolator that uses it, and then
> you could modify the cubic interpolator following the format of the linear
> interpolator and using my guidance as and when you need it.
>
> Let me know your preference,
> Ian
>
> P.S.  Never apologise for not being a computer scientist!  Many of our
> developers, myself included, are proper scientists or engineers!!!
>
>
>
> On 29 October 2012 09:37, Ian Thomas <ian...@gm...> wrote:
>
>> Hi Geoffroy
>>
>> This will certainly be very useful.  I need to spend some time looking at
>> it and seeing how it would best fit within the matplotlib framework,
>> particularly as only a few days ago I committed to writing a triangular
>> grid interpolator for quad grids and it would be sensible to group these
>> interpolators together in some way.
>>
>> I'll get back to you when I've had time to look at it.
>>
>> Thanks for your efforts!
>> Ian
>>
>>
>>
>> On 28 October 2012 20:17, GBillotey <geo...@gm...> wrote:
>>
>>>    Hi!
>>>
>>>
>>> I had recently to develop interpolators for a function defined at the
>>> nodes
>>> of a user-specified triangular mesh.
>>> (Beside interpolation, it can help producing higher-quality tricontour
>>> plots, using interpolation on a refined mesh and matplotlib tricontour
>>> function.)
>>>
>>> Being a regular user of matplotlib, I would be happy if it can be useful
>>> to
>>> others...
>>> The code is hosted here:
>>> https://github.com/GBillotey/trimesh-interpolator.git
>>>
>>>
>>> Please let me know if it this dev. can be useful and if the code needs
>>> some
>>> cleaning (I am not a computer scientist, only a mechanical engineer)
>>>
>>>
>>> Cheers,
>>> Geoffroy.
>>>
>>
>

[matplotlib-devel] OpenGL backend with Galry

[Cyrille R. <cyr...@gm...>]

Hi all,

I am developing a high-performance interactive visualization package in
Python based on PyOpenGL (http://rossant.github.com/galry/). It is
primarily meant to be used as a framework for developing complex
interactive GUIs (in QT) that deal with very large amounts of data (tens of
millions of points). But it may also be used, like matplotlib, as a
high-level interactive library to plot and visualize data.

The low-level interface is mostly done at this point (the code is still in
an experimental stage though), and I'm now focusing on my current research
project which is to write a scientific GUI based on this interface.
However, I think people (including myself!) may be interested in a
matplotlib-like high-level interface. I was first thinking about writing
such an interface from scratch, by implementing a very small fraction of
the matplotlib interface (basic commands like figure(), plot(), subplot(),
show(), etc.). One could then quickly visualize huge datasets with the same
commands than matplotlib.

Another solution would be to write a matplotlib backend based on this
library. I am not familar enough with the internals of matplotlib to know
how complicated it could be. I may do it myself, but it would probably take
a long time since it is currently not my highest priority. I would be glad
if someone experienced in writing backends was interested in working on it.
Actually I could do everything that is specific to my library, which
already provides commands to plot points, lines, textures, etc. The canvas
is based on QT and may be similar to what is already implemented in the QT
backend.

Of course, it would already be great if only the most basic plotting
features were available in the backend. A first step could be for example
to have a simplistic example "plot(x, sin(x))" working (with interactive
navigation).

I am looking forward to your feedback.

Best,
Cyrille Rossant

[matplotlib-devel] tick_params direction 'inout'

[Mike K. <mc...@gm...>]

Hi all,

I don't have time to make a patch at the moment, but I thought I'd point 
it out for anyone to give it a go...

tick_params(axis='both', **kwargs)
     Change the appearance of ticks and tick labels.

     Keyword arguments:

...

     *direction* : ['in' | 'out']
         Puts ticks inside or outside the axes.

I just found that *direction* accepts 'inout' as well, which
does indeed place the tick on both sides of the spine. So the 
documentation should be updated to reflect this. If it were me, I'd 
allow 'both' to work as well.

M

Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)

[Ian T. <ian...@gm...>]

Hi Geoffroy,

I have had some time to look at your TriLinearInterpolator in some detail
(the other two files only briefly).  I would indeed like to add something
like this to matplotlib - the mesh refinement looks very nice and the
interpolators would be useful to many people.

As you suspected, the code does need significant changes before we can
include it.  Some are merely cosmetic, as all code must adhere to PEP8 and
the matplotlib coding guidelines, but there are also some functional and
performance improvements.  For example, your wavefront method for finding
the triangle containing a certain point must be able to deal with masked
triangulations and indeed triangulations that are discontinuous, for
example two islands in a masked-out ocean, which is unusual but must be
supported.  In terms of performance, there is much explicit looping within
numpy arrays that could be improved using other numpy array commands, and
would also reduce the length of the source code.  There is an argument for
some of the performance-critical code to be in C/C++.

I think the code used to determine which triangle contains a certain point
should be factored out into its own TriFinder class, so that (1) it does
not need to be replicated in the two interpolator classes, and (2)
different algorithms can be easily swapped if necessary.  I have a C++
TriFinder class that I could modify to work within matplotlib, and it is
O(log N) so should be faster than your version for typical use cases.

I expect that this is probably more work than you anticipated when you
asked if the code needed any improvement!  I propose the following: if you
are happy to give matplotlib your source code as it stands and for us to
include it under our BSD-style license, then I will take on the
responsibility of getting it into a form that will be accepted by the other
developers.  I will acknowledge your contribution in both the source code
and on the web site, something like "based on code contributed by Geoffroy
Billotey".

Alternatively, if you would like to use this as an excuse to learn how to
contribute to matplotlib more actively but don't want to take on
everything, then we could divide up the work so that first I write my C++
log(N) TriFinder class and the linear interpolator that uses it, and then
you could modify the cubic interpolator following the format of the linear
interpolator and using my guidance as and when you need it.

Let me know your preference,
Ian

P.S.  Never apologise for not being a computer scientist!  Many of our
developers, myself included, are proper scientists or engineers!!!


On 29 October 2012 09:37, Ian Thomas <ian...@gm...> wrote:

> Hi Geoffroy
>
> This will certainly be very useful.  I need to spend some time looking at
> it and seeing how it would best fit within the matplotlib framework,
> particularly as only a few days ago I committed to writing a triangular
> grid interpolator for quad grids and it would be sensible to group these
> interpolators together in some way.
>
> I'll get back to you when I've had time to look at it.
>
> Thanks for your efforts!
> Ian
>
>
>
> On 28 October 2012 20:17, GBillotey <geo...@gm...> wrote:
>
>>    Hi!
>>
>>
>> I had recently to develop interpolators for a function defined at the
>> nodes
>> of a user-specified triangular mesh.
>> (Beside interpolation, it can help producing higher-quality tricontour
>> plots, using interpolation on a refined mesh and matplotlib tricontour
>> function.)
>>
>> Being a regular user of matplotlib, I would be happy if it can be useful
>> to
>> others...
>> The code is hosted here:
>> https://github.com/GBillotey/trimesh-interpolator.git
>>
>>
>> Please let me know if it this dev. can be useful and if the code needs
>> some
>> cleaning (I am not a computer scientist, only a mechanical engineer)
>>
>>
>> Cheers,
>> Geoffroy.
>>
>

Re: [matplotlib-devel] New plot type idea -- EventRaster

[Marcel O. <m.o...@ja...>]

[...]
 > > On Mon, Sep 24, 2012 at 3:33 PM, Todd <toddrjen@...> wrote:
[...]
 > >> I tentatively am calling the plot type an "EventRaster" plot
 > >> (name suggestions, along with any other suggestions, are
 > >> welcome).  The plot is made up if horizontal rows of identical
 > >> vertical lines and/or markers.  Each line or marker represents a
 > >> discrete event, and each row represents a single sequence of
 > >> events (such as a trial).  The x-axis position of the line or
 > >> marker identifies the location of the event by some measure.  An
 > >> example of what such a plot often looks like is below.
 > >>
 > >> http://hebb.mit.edu/courses/9.29/2003/athena/dylanh/quad-rast.gif
 > >>
 > >> This sort of plot is used ubiquitously in neuroscience.  It is
 > >> used to show the time of discrete neural (brain cell) events
 > >> (called "spikes") over time in repeated trials, and is generally
 > >> called a spike raster, raster plot, or raster graph.  However,
 > >> it can be used in any situation where you are only concerned
 > >> with the position of events but not their amplitude, especially
 > >> if you want to look for patterns in those events or look for
 > >> differences between multiple sequences of events.
[...]
 > Assuming we go with the name, here is my proposed call signature:
 > 
 > EventRaster(x, offset=0, height=1, **kargs)
 > 
 > x is a 1D or 2D array.  If a 1D array, it create a single row of
 > lines.  If it is a 2D array, it creates one row of lines for each row
 > in the array.

Dear Todd,

I saw your posts on this idea with interest.  I recently had similar,
but more intricate plotting needs and came up with a brute-force
solution calling "barh" for each of the vertical bars.  It is
attached.  It is not "nice" code because it mixes program logic,
visual tweaks, and plotting, but maybe you can take it as an example
of what can already be done and should probably be wrapped in a clean
API.  This is already a simplified stand-along version, there is a
more involved production code behind.

It is a timing graph (that's what I think of, maybe "timing" could be
an appropriate name as I too agree that "raster" is a bit confusing)
of events with the following requirements:

- An event has a start and a finish (or, equivalently, a start and a
  duration), so the vertical bars must be drawn with varying widths.

- The bars may be colored differently (in the concrete application,
  the color scheme was green/yellow/red with green for tasks which are
  processed in their allocated time slice, yellow for the overrun into
  the next slice, and red (not show in example below) if allowable
  delay tolerances are violated; it is easy to imagine very different
  applications and coloring schemes).

- The bars may have different vertical heights (in the application
  representing channel capacities; in the attached example they are
  all the same, but in general they might be different); it is easy to
  imagine applications (maybe like the "rug plots" mentioned in the
  thread) where the vertical height of individual bars needs to be
  specified.

- There are discrete clock ticks which are indicated by vertical grid
  lines.

I am not sure if this gets too general, but maybe it makes sense to
fold some or all of the above into the API of what you are working
on.

With best regards,
Marcel

PS.: I initially sent a post like this one from a wrong email address,
so it got stuck in moderation.  It appears that sadly John Hunter is
still moderating the list, so it will probably never get approved.  If
someone else has access to moderation, please just cancel that post,
it's the same in different words.  I blew my one chance to cancel it
and now am stuck with an invalid confirmation URL for the list server.


---------------------------------------------------------------------
Marcel Oliver                                 Phone: +49-421-200-3212       
School of Engineering and Science               Fax: +49-421-200-3103
Jacobs University                       m.o...@ja...
Campus Ring 1                                   ol...@me... 
28759 Bremen, Germany         http://math.jacobs-university.de/oliver
---------------------------------------------------------------------

Re: [matplotlib-devel] master vs maintenance branches (current process and proposal for future workflow)

[Thomas K. <th...@kl...>]

On 12 November 2012 20:52, Benjamin Root <ben...@ou...> wrote:

> There are other git workflows, which are all perfectly valid.  If you can
> convince us to use another, feel free to propose one.  However, we have
> done a couple of releases with gitwash, and it has worked quite well for us
> given how small our maintenance overhead is.  Just have to remember to
> regularly merge the maintenance branch to master.  That's all.


For what it's worth, what Paul describes is essentially the process we use
in IPython. All pull requests target master, and critical fixes are
backported if we decide we're going to do another maintenance release. This
is easier for contributors, because all pull requests work the same way,
but I think it does take some time for someone to go through the issues
tagged for backporting and cherry-pick the commits. Min tackled that for
our recent maintenance releases.

We use pull request testing to keep master reasonably stable, although we
don't make any promises about it.

Best wishes,
Thomas