matplotlib

Jan Strube wrote:
> Dear matplotters,
> encouraged from the excellent response times to my last problem, I am 
> trying to explore more features of matplotlib.
> 
> My current problem is with hexbin.
> I have been using numpy.histogram2d and imshow so far for 2d histograms, 
> but I must admit that hexbin looks quite pretty.
> 
> In order to emphasize small differences over a large scale, I am using 
> hexbin with the bins='log' option.
> The result is very nice, but when I add a pyplot.colorbar(), I would 
> like to have the original number of entries on the ticks, not their log10.
> 
> Ideally, I'd change the yaxis of the colorbar to a logscale, and use 
> numpy.power(10, yaxis_values) on it.
> I think colorbar().ax.set_yscale('log') should do what I need for the 
> first part, but how do I get the original number of entries back, rather 
> than their log10.
> I could cheat and simply replace each ticklabel with 10^ticklabel, but I 
> don't even know how to do that.

Offhand, I suspect this would be the best solution at present, if the 
only problem now is that you want the ticklabels written differently. 
Try using the "format" kwarg of the colorbar, supplying either a format 
string or a formatter.

Something like this (untested):

from matplotlib.ticker import LogFormatter

class LogFormatterHB(LogFormatter):
     def __call__(self, v, pos=None):
         vv = self._base ** v
         return LogFormatter.__call__(self, vv, pos)

...

cbar = colorbar( ... ,  format=LogFormatterHB())


Eric

On Thu, Feb 18, 2010 at 10:25 AM, Jan Strube <curiousjan@...> wrote:
> Hi Eric,
> thanks for your response.
>
> Your solution looks like it's going to return the right numbers, but for
> some reason the tick labels are gone completely. (Except 0)
>
> The code I have is below. I do think the current behavior is a bug, but if I
> can get your workaround to go, I'll be happy.

What's the bug -- the fact that the tick labels are the log10 of the
number instead of the number?

I tried Eric's proposal with your code and it is working for me
(colorbar tick labels at 1, 10 and 100)

JDH


import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import LogFormatter

class LogFormatterHB(LogFormatter):
    def __call__(self, v, pos=None):
        vv = self._base ** v
        return LogFormatter.__call__(self, vv, pos)

data = np.load('deltaR_parton_jet_109371.npz')
ptcut = np.logical_and(data['jetMomentum'] < 300000, data['jetMomentum']>0)
deltaRCut = data['deltaR']>0
cut = np.logical_and(ptcut, deltaRCut)
fig = plt.figure()
plt.hexbin(data['jetMomentum'][cut] / 1000, data['deltaR'][cut],
gridsize=50, bins='log')
plt.title('deltaR between parton(eta<2.5) and jet(eta<2.5)')
plt.xlabel('jet pt (GeV)')
plt.ylabel('deltaR')
cb = plt.colorbar(format=LogFormatterHB())
cb.set_label('# entries')
plt.show()

On Thu, Feb 18, 2010 at 4:06 PM, Jan Strube <curiousjan@...> wrote:
> Hi John,
> thanks for trying this also. Yes, I think it's a bug that not the scale is
> log, but the data is.
> Unfortunately, the solution really doesn't work for me.
> Please see the attached screenshot. (Yes, it still says log_10 entries, but
> the code is otherwise the same)
> In [2]: matplotlib.__version__
> Out[2]: '1.0.svn'
> This is r8063, I think.
> Strange that I get different results. Could this be a backend problem? I use
> PyQT4.
> I'd be happy to also update from svn if you think that helps.

I'm running svn but not svn HEAD -- you should try updating to HEAD
and I will do the same later (unfortunately HEAD is broken on my work
machine (solaris, python2.4) because of the CXX upgrade I put in some
time ago.  I think I am on r8083.

I do not think this difference could be caused by a backend or GUI
version difference as all of the formatting logic happens in the
frontend.  If we are on the same version of svn, we should be getting
the same tick labels.

JDH

Hi John, Eric,

sorry to bug again, but was either of you able to reproduce my findings that
in svn head the tick labels don't get printed if the formatter changes them
to be outside the range of the axis?
Cheers,
    Jan


On Fri, Feb 19, 2010 at 10:40 AM, Jan Strube <curiousjan@...> wrote:

> Hi John,
>
> I am now at r8141.
> I don't see a difference. There's a one at the bottom of the scale, that's
> all.
> I am copying and pasting the code in your other email into a python
> console.
>
> It's a bit strange, because when I put in a print statement, I see that the
> method gets called a number of times...
> [edit] I see what's going on. If I replace vv with a large constant (like
> 100), it never gets printed.
> If I replace vv with a small constant (like 1), that is below the range of
> the scale, it does get printed at each tick.[/edit]
>
> My setup is kubuntu karmic, dependencies listed below.
> I have removed my matplotlibrc, but the result is the same. I am attaching
> it nevertheless...
>
> Cheers,
>     Jan
>
>
>
> ============================================================================
>
>     BUILDING
> MATPLOTLIB
>
>                 matplotlib:
> 1.0.svn
>
>                     python: 2.6.4 (r264:75706, Nov  2 2009, 14:44:17)
> [GCC
>
> 4.4.1]
>
>                   platform:
> linux2
>
>
>
>     REQUIRED
> DEPENDENCIES
>
>                      numpy:
> 1.3.0
>
>                  freetype2:
> 9.20.3
>
>
>
>     OPTIONAL BACKEND
> DEPENDENCIES
>
>                     libpng:
> 1.2.37
>
>                    Tkinter:
> no
>
>                             * TKAgg requires
> Tkinter
>
>                   wxPython:
> no
>
>                             * wxPython not
> found
>
>                 pkg-config: looking for pygtk-2.0
> gtk+-2.0
>
>                             * Package pygtk-2.0 was not found in the
> pkg-config
>                             * search path. Perhaps you should add the
> directory
>                             * containing `pygtk-2.0.pc' to the
> PKG_CONFIG_PATH
>
>                             * environment variable No package 'pygtk-2.0'
> found
>                             * Package gtk+-2.0 was not found in the
> pkg-config
>                             * search path. Perhaps you should add the
> directory
>                             * containing `gtk+-2.0.pc' to the
> PKG_CONFIG_PATH
>
>                             * environment variable No package 'gtk+-2.0'
> found
>                             * You may need to install 'dev' package(s)
> to
>                             * provide header
> files.
>
>                       Gtk+:
> no
>
>                             * Could not find Gtk+ headers in any
> of
>
>                             * '/usr/local/include', '/usr/include',
> '.'
>            Mac OS X native:
> no
>
>                         Qt:
> no
>
>                        Qt4: Qt: 4.5.2, PyQt4:
> 4.6
>
>                      Cairo:
> 1.8.6
>
>
>
>     OPTIONAL DATE/TIMEZONE
> DEPENDENCIES
>
>                   datetime: present, version
> unknown
>
>                   dateutil:
> 1.4.1
>
>                       pytz:
> 2009l
>
>
>
>     OPTIONAL USETEX
> DEPENDENCIES
>
>                     dvipng:
> 1.12
>
>                ghostscript:
> 8.70
>
>                      latex:
> 3.1415926
>
>                    pdftops:
> 0.12.0
>
>
>
>     [Edit setup.cfg to suppress the above
> messages]
>
>
> ============================================================================
>
>
> On Thu, Feb 18, 2010 at 10:29 PM, John Hunter <jdh2358@...> wrote:
>
>> On Thu, Feb 18, 2010 at 4:06 PM, Jan Strube <curiousjan@...> wrote:
>> > Hi John,
>> > thanks for trying this also. Yes, I think it's a bug that not the scale
>> is
>> > log, but the data is.
>> > Unfortunately, the solution really doesn't work for me.
>> > Please see the attached screenshot. (Yes, it still says log_10 entries,
>> but
>> > the code is otherwise the same)
>> > In [2]: matplotlib.__version__
>> > Out[2]: '1.0.svn'
>> > This is r8063, I think.
>> > Strange that I get different results. Could this be a backend problem? I
>> use
>> > PyQT4.
>> > I'd be happy to also update from svn if you think that helps.
>>
>> I'm running svn but not svn HEAD -- you should try updating to HEAD
>> and I will do the same later (unfortunately HEAD is broken on my work
>> machine (solaris, python2.4) because of the CXX upgrade I put in some
>> time ago.  I think I am on r8083.
>>
>> I do not think this difference could be caused by a backend or GUI
>> version difference as all of the formatting logic happens in the
>> frontend.  If we are on the same version of svn, we should be getting
>> the same tick labels.
>>
>> JDH
>>
>
>

On Mon, Feb 22, 2010 at 2:42 AM, Jan Strube <curiousjan@...> wrote:
> Hi John, Eric,
> sorry to bug again, but was either of you able to reproduce my findings that
> in svn head the tick labels don't get printed if the formatter changes them
> to be outside the range of the axis?

On svn HEAD (r8142)  (and the branch) I get the attached figure with
the attached code.  I'm running the code with

> python test.py

from the python shell.  Are you getting a different result?

JDH

On Mon, Feb 22, 2010 at 2:57 PM, Jan Strube <curiousjan@...> wrote:
> Hi John,
> thanks for the test case.
> Indeed I am getting different results.
> Labels that are beyond the range of the axis don't get printed.
> This is reproducible on ubuntu (test.png) and on mac os x (t.png) both have
> the svn head of mpl.


OK, I see we are getting several differences (eg the x and y viewlim
extent, pixel dimensions of saved figure) in addition to the tick
label problem you described.  I modified the test script to force the
use of the Agg backend.  Please run it with the verbose-debug flag and
post the image and output.  Mine are attached

  > python ~/test.py --verbose-debug > debug.out

On Mon, Feb 22, 2010 at 3:42 PM, Jan Strube <curiousjan@...> wrote:
> Hi John,
> the attachment may not make it to the list. However, please run the modified
> test.py that I have attached.
> It requires the attached input file.
> Then change it to read the original input file.
> In my case:
> The broken case:

OK, at least now we are running the sample example :-)

The problem is that the LogFormatter has a default which is
"decadeOnly=True" and in the first case which "worked" three of the
tick locations coincidentally came down on decades (0, 1, 2 -> 1, 10,
100).  In the case you were working with, only one of the ticks mapped
to the decade.

So for this case we want a locator that returns integers 0,1,2... that
will then get mapped via Eric's custom formatter to the 10^i formats.
Unfortunately, there is no easy way to set the locator for the
colorbar.  An easy workaround *for this case* is to simply set the
tick locations

  cb = plt.colorbar(format=LogFormatterHB(), ticks=[0,1,2])

but in general you may not know the decade span that you need.  It
does all feel a bit kludgy.  The problem as you noted in one of your
earlier posts is that the data is log scaled before being passed into
the PolyCollection and the fact that it is log scaled is then lost to
the colorbar.  It seems everything would fit together more naturally
if we passed in raw scalar data to the PolyCollection and set the norm
to be colors.LogNorm, and then also set norm=colors.LogNorm on the
colorbar

I tried:

  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
  cb = plt.colorbar(norm=colors.LogNorm())

but this appears to be broken:

msierig@...> python test.py
Traceback (most recent call last):
  File "test.py", line 29, in <module>
    cb = plt.colorbar()
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/pyplot.py",
line 1356, in colorbar
    ret = gcf().colorbar(mappable, cax = cax, ax=ax, **kw)
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/figure.py",
line 1103, in colorbar
    cb = cbar.Colorbar(cax, mappable, **kw)
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 690, in __init__
    ColorbarBase.__init__(self, ax, **kw)
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 242, in __init__
    self.draw_all()
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 260, in draw_all
    self._config_axes(X, Y)
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 332, in _config_axes
    self.update_ticks()
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 271, in update_ticks
    ticks, ticklabels, offset_string = self._ticker()
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/colorbar.py",
line 458, in _ticker
    b = np.array(locator())
  File "/home/msierig/dev/lib/python2.6/site-packages/matplotlib/ticker.py",
line 1173, in __call__
    vmin = self.axis.get_minpos()

On Mon, Feb 22, 2010 at 4:33 PM, John Hunter <jdh2358@...> wrote:
>  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
> data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
>  cb = plt.colorbar(norm=colors.LogNorm())
>
> but this appears to be broken:

I committed some changes to support this -- the following now works:

  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())

  cb = plt.colorbar(norm=polycol.norm)

Eric - I was surprised the colorbar does not use the mappable norm by
default (if passed norm=None).  Instead it uses ::

  norm = colors.Normalize()

is this a feature?

JDH

John Hunter wrote:
> On Mon, Feb 22, 2010 at 4:33 PM, John Hunter <jdh2358@...> wrote:
>>  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
>> data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
>>  cb = plt.colorbar(norm=colors.LogNorm())
>>
>> but this appears to be broken:
> 
> I committed some changes to support this -- the following now works:
> 
>   polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
> data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
> 
>   cb = plt.colorbar(norm=polycol.norm)
> 
> Eric - I was surprised the colorbar does not use the mappable norm by
> default (if passed norm=None).  Instead it uses ::
> 
>   norm = colors.Normalize()
> 
> is this a feature?

Yes.  You are looking at ColorbarBase, which does not have an associated 
mappable.  The derived Colorbar class does grab the cmap and norm from 
the mappable used in the initialization.  Is this somehow not working? 
Did you really need to specify the norm explicitly?

Eric
> 
> JDH

Hi John,

thanks for keeping at it. I have updated from svn

But this script
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import LogFormatter
from matplotlib import colors

class LogFormatterHB(LogFormatter):
   def __call__(self, v, pos=None):
       vv = self._base ** v
       print vv
       return LogFormatter.__call__(self, vv, pos)

data = np.load('deltaR_parton_jet_109370.npz')
ptcut = np.logical_and(data['jetMomentum'] < 300000, data['jetMomentum']>0)
deltaRCut = data['deltaR']>0
cut = np.logical_and(ptcut, deltaRCut)
fig = plt.figure()
polycol = plt.hexbin(data['jetMomentum'][cut] / 1000, data['deltaR'][cut],
gridsize=50, norm=colors.LogNorm())
plt.title('deltaR between parton(eta<2.5) and jet(eta<2.5)')
plt.xlabel('jet pt (GeV)')
plt.ylabel('deltaR')
cb = plt.colorbar(norm=polycol.norm)
cb.set_label('# entries')
fig.savefig('hexbin_demo.png', dpi=100)
#plt.show()

gives me an error:

> python test.py
Traceback (most recent call last):
  File "test.py", line 24, in <module>
    cb = plt.colorbar(norm=polycol.norm)
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/pyplot.py",
line 1356, in colorbar
    ret = gcf().colorbar(mappable, cax = cax, ax=ax, **kw)
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/figure.py",
line 1104, in colorbar
    cb = cbar.Colorbar(cax, mappable, **kw)
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
line 649, in __init__
    ColorbarBase.__init__(self, ax, **kw)
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
line 240, in __init__
    self.draw_all()
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
line 251, in draw_all
    self._config_axes(X, Y)
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
line 278, in _config_axes
    ticks, ticklabels, offset_string = self._ticker()
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
line 417, in _ticker
    b = np.array(locator())
  File
"/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/ticker.py",
line 1085, in __call__
    vmin = self.axis.get_minpos()
AttributeError: DummyAxis instance has no attribute 'get_minpos'

Cheers,
    Jan



On Tue, Feb 23, 2010 at 12:26 AM, John Hunter <jdh2358@...> wrote:

> On Mon, Feb 22, 2010 at 4:33 PM, John Hunter <jdh2358@...> wrote:
> >  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
> > data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
> >  cb = plt.colorbar(norm=colors.LogNorm())
> >
> > but this appears to be broken:
>
> I committed some changes to support this -- the following now works:
>
>  polycol = plt.hexbin(data['jetMomentum'][cut] / 1000,
> data['deltaR'][cut],gridsize=50, norm=colors.LogNorm())
>
>   cb = plt.colorbar(norm=polycol.norm)
>
> Eric - I was surprised the colorbar does not use the mappable norm by
> default (if passed norm=None).  Instead it uses ::
>
>  norm = colors.Normalize()
>
> is this a feature?
>
> JDH
>

On Tue, Feb 23, 2010 at 4:12 AM, Jan Strube <curiousjan@...> wrote:
> Hi John,
> thanks for keeping at it. I have updated from svn
> But this script
>   File
> "/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
> line 278, in _config_axes
>     ticks, ticklabels, offset_string = self._ticker()
>   File
> "/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/colorbar.py",
> line 417, in _ticker
>     b = np.array(locator())
>   File
> "/Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib-1.0.svn_r8037-py2.6-macosx-10.6-universal.egg/matplotlib/ticker.py",
> line 1085, in __call__
>     vmin = self.axis.get_minpos()
> AttributeError: DummyAxis instance has no attribute 'get_minpos'

Take a look at the file name in the traceback: matplotlib-1.0.svn_r8037

The r8037 is the svn revision number.  According the svn log, I made
my commit on r8149, so you are not running HEAD.  You may have
multiple matplotlib's installed and are not picking up the right egg.
I recommend

  > rm -rf /Users/Jan/PYTHON/lib/python2.6/site-packages/matplotlib*

and then doing a clean install from mpl svn r8149 or later.

JDH

On Mon, Feb 22, 2010 at 6:28 PM, Eric Firing <efiring@...> wrote:
> Yes.  You are looking at ColorbarBase, which does not have an associated
> mappable.  The derived Colorbar class does grab the cmap and norm from the
> mappable used in the initialization.  Is this somehow not working? Did you
> really need to specify the norm explicitly?

No, I didn't test this, I just read the code (apparently the wrong
code) and concluded I needed it.  I did just test w/o it and all is
well.  Sorry for the noise.

JDH

Gentlemen!

Thanks a lot for your help.
This works now for me (with and without the norm in the colorbar() call)

Best,
    Jan


On Tue, Feb 23, 2010 at 3:47 PM, John Hunter <jdh2358@...> wrote:

> On Mon, Feb 22, 2010 at 6:28 PM, Eric Firing <efiring@...> wrote:
> > Yes.  You are looking at ColorbarBase, which does not have an associated
> > mappable.  The derived Colorbar class does grab the cmap and norm from
> the
> > mappable used in the initialization.  Is this somehow not working? Did
> you
> > really need to specify the norm explicitly?
>
> No, I didn't test this, I just read the code (apparently the wrong
> code) and concluded I needed it.  I did just test w/o it and all is
> well.  Sorry for the noise.
>
> JDH
>