matplotlib-devel

Re: [matplotlib-devel] Matplotlib's new default colormap

[Lion K. <kri...@ge...>]

Hi all,

I was made aware of this thread and thought I’d share a notebook I recently made for a similar purpose:

http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5 <http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5> (takes a while to load…)

It attempts to “optimize" colormaps by defining optimality as having a linear lightness across the colormap in LAB color space. It is very simple and not a proper optimization procedure. It just goes to LAB space, sets the lightness to the target lightness, and goes back to sRGB space. This does not always work as the LAB color space is much bigger than the RGB one but in many cases it produces fairly good results.

The nice thing about this is that the lightness range can be chosen so it is does not always have to be stark white or black at the ends and some hue can be preserved.

I am not sure if some similar functionality is useful to include into matplotlib (I don’t really think so) but if yes, let me know and I’ll give it a try. I guess it could also be extended to optimize towards monotonic changes in hue.

Cheers and all the best!

Lion

Re: [matplotlib-devel] Matplotlib's new default colormap

[Thomas C. <tca...@gm...>]

That is super cool.  I was thinking about doing something similar, glad it
has already been so well done.

The example figures at the bottom bring up another point, we should have a
canonical set of test figures, both for the color map and the defaults in
general, I think that will really help with this discussion.

That example could also be reused as a standard show-case for style-files.

Tom


On Mon Nov 24 2014 at 11:32:41 AM Michael Droettboom <md...@st...>
wrote:

>  I, for one, would love to see a pull request for this if you're game.
>
> Mike
>
>
> On 11/24/2014 04:27 AM, Lion Krischer wrote:
>
> Hi all,
>
> I was made aware of this thread and thought I’d share a notebook I
> recently made for a similar purpose:
>
> http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5 (takes a
> while to load…)
>
> It attempts to “optimize" colormaps by defining optimality as having a
> linear lightness across the colormap in LAB color space. It is very simple
> and not a proper optimization procedure. It just goes to LAB space, sets
> the lightness to the target lightness, and goes back to sRGB space. This
> does not always work as the LAB color space is much bigger than the RGB one
> but in many cases it produces fairly good results.
>
> The nice thing about this is that the lightness range can be chosen so it
> is does not always have to be stark white or black at the ends and some hue
> can be preserved.
>
> I am not sure if some similar functionality is useful to include into
> matplotlib (I don’t really think so) but if yes, let me know and I’ll give
> it a try. I guess it could also be extended to optimize towards monotonic
> changes in hue.
>
> Cheers and all the best!
>
>  Lion
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREEhttp://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Matplotlib-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>
>
> --
> Michael Droettboom
> Science Software Branch
> Space Telescope Science Institute
> http://www.droettboom.com
>
>  ------------------------------------------------------------
> ------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/
> 4140/ostg.clktrk_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Re: [matplotlib-devel] Matplotlib's new default colormap

[Maximilian A. <max...@gm...>]

Hi all,

I had a discussion with Phil Elson about this last weekend during the
Bloomberg Open Source Day. I don't consider myself an expert on colormaps
by any means, but I started digging into them a while ago when I was
looking for a way of generating a perceptually linear *cyclic* colormap in
order to represent phase angle values. (I've been meaning to discuss this
issue on this list for a while but will do so in a different thread once I
get around to tidying up my results so far.) Phil encouraged me to reply to
this thread because he said that even non-expert views would be very
welcome, so here you go.

Basically, I agree with most of what Nathaniel Smith suggested in his email
from November 21. I'm going to comment on some of his points inline below
and will finally suggest a way of designing a new colormap at the end.


Nathaniel Smith wrote:

> it should be a sequential colormap [...]

Agreed.

> it should be perceptually uniform

Agreed.

> There's lots of research on how to measure perceptual distance --
> a colleague and I happen to have recently implemented a
> state-of-the-art model of this for another project, in case anyone
> wants to play with it [3].

I haven't had time to check this out in detail yet, but it looks pretty
interesting and will certainly be very useful to assess the quality of any
suggestions. However, can this help to actually *design* a new colormap?
The answer might be hidden in the referenced paper [Luo2006], but I haven't
read it yet.

> or just using good-old-L*a*b* is a reasonable quick-and-dirty
approximation.

Can you elaborate how "dirty" you think using L*a*b* would be? (See my
suggestion for finding a new colormap below.)

>- it should have a perceptually uniform luminance ramp, i.e. if you
> convert to greyscale it should still be uniform.

Agreed. What's unclear to me is how large this luminance ramp should be. We
certainly can't go all the way to full luminance because this won't be
visible on a white background. This probably needs experimenting (again see
below).

> - it should also have some kind of variation in hue, because
> hue variation is a really helpful additional cue to perception,
> having two cues is better than one, and there's no reason
> not to do it.

Agreed.

> - the hue variation should be chosen to produce reasonable results
> even for viewers with the more common types of colorblindness.
> (Which rules out things like red-to-green.)

Agreed. Are you aware of any simple ways of avoiding the most common
issues? Are there any blog posts or papers on designing colormaps that are
suitable for colorblind people?

> And, for bonus points, it would be nice to choose a hue ramp that
> still works if you throw away the luminance variation, because then we
> could use the version with varying luminance for 2d plots, and the
> version with just hue variation for 3d plots. (In 3d plots you really
> want to reserve the luminance channel for lighting/shading, because
> your brain is *really* good at extracting 3d shape from luminance
> variation. If the 3d surface itself has massively varying luminance
> then this screws up the ability to see shape.)

Just out of interest, is there currently an easy way in matplotlib of
producing a 3d plot where luminance is used for lighting/shading as you
suggest?


Now the question is: how do we actually *design* a colormap with these
requirements? Leon Krischer's notebook [1] looks totally awesome, but if I
understand correctly the optimisation he uses "only" takes care of
linearising the luminance value, but this does not necessarily guarantee
that the hue values are also linear, right? It also feels somewhat clumsy
to me to start out with a colormap that's "wrong" (w.r.t. our requirements
above) and then "fix" it. However, the notebook looks like a great guidance
for finding suitable candidates and assessing their quality.

It appears to me that a simple yet effective way of coming up with a good
colormap would be to pick two points in the L*a*b* color space that can be
represented by RGB values, connect them by a line and use the interpolated
values for the resulting colormap. Since L*a*b* space is (close to)
perceptually linear, this would pretty much guarantee all the requirements
above.

What's missing is an easy way of doing this. I'm envisaging a simply GUI
which allows the user to easily pick two points in L*a*b* space, generates
a colormap from them as described above and also generates a few sample
plots to evaluate the quality of the colormap (along the lines of [1] or
the numerous blog posts linked to in the discussion). I am close to having
a prototype for such a GUI which should allow to do this relatively
painlessly. I'll try to finish it up over the weekend and will post here
once it's ready. Btw, if anyone has suggestions for sample datasets that
can help in assessing the quality of colormaps they would be much
appreciated.

Any comments or clarifications of points that I misunderstood are very
welcome.

Best wishes,
Max

[1] http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5

Re: [matplotlib-devel] Matplotlib's new default colormap

[Phil E. <pel...@gm...>]

Thanks for all the contributions so far. Looks like matplotlib is blessed
with people who are far more knowledgeable than I on the subject, and I'd
say we were pretty much at a consensus on requirements.

Given these requirements, what we need is some proposed colormaps - Max's
approach of generating an optimal solution in LAB space sounds interesting,
as do the other approaches of minimising some parameters of existing
colormaps.

To facilitate this discussion, do we need a repository of proposed
colormaps so that we can compare like with like? I notice that Mike Bostock
has an interesting post to show various color spaces in d3.js which may be
of interest http://bl.ocks.org/mbostock/3014589.



On 4 December 2014 at 14:38, Maximilian Albert <max...@gm...>
wrote:

> Hi all,
>
> I had a discussion with Phil Elson about this last weekend during the
> Bloomberg Open Source Day. I don't consider myself an expert on colormaps
> by any means, but I started digging into them a while ago when I was
> looking for a way of generating a perceptually linear *cyclic* colormap in
> order to represent phase angle values. (I've been meaning to discuss this
> issue on this list for a while but will do so in a different thread once I
> get around to tidying up my results so far.) Phil encouraged me to reply to
> this thread because he said that even non-expert views would be very
> welcome, so here you go.
>
> Basically, I agree with most of what Nathaniel Smith suggested in his
> email from November 21. I'm going to comment on some of his points inline
> below and will finally suggest a way of designing a new colormap at the end.
>
>
> Nathaniel Smith wrote:
>
> > it should be a sequential colormap [...]
>
> Agreed.
>
> > it should be perceptually uniform
>
> Agreed.
>
> > There's lots of research on how to measure perceptual distance --
> > a colleague and I happen to have recently implemented a
> > state-of-the-art model of this for another project, in case anyone
> > wants to play with it [3].
>
> I haven't had time to check this out in detail yet, but it looks pretty
> interesting and will certainly be very useful to assess the quality of any
> suggestions. However, can this help to actually *design* a new colormap?
> The answer might be hidden in the referenced paper [Luo2006], but I haven't
> read it yet.
>
> > or just using good-old-L*a*b* is a reasonable quick-and-dirty
> approximation.
>
> Can you elaborate how "dirty" you think using L*a*b* would be? (See my
> suggestion for finding a new colormap below.)
>
> >- it should have a perceptually uniform luminance ramp, i.e. if you
> > convert to greyscale it should still be uniform.
>
> Agreed. What's unclear to me is how large this luminance ramp should be.
> We certainly can't go all the way to full luminance because this won't be
> visible on a white background. This probably needs experimenting (again see
> below).
>
> > - it should also have some kind of variation in hue, because
> > hue variation is a really helpful additional cue to perception,
> > having two cues is better than one, and there's no reason
> > not to do it.
>
> Agreed.
>
> > - the hue variation should be chosen to produce reasonable results
> > even for viewers with the more common types of colorblindness.
> > (Which rules out things like red-to-green.)
>
> Agreed. Are you aware of any simple ways of avoiding the most common
> issues? Are there any blog posts or papers on designing colormaps that are
> suitable for colorblind people?
>
> > And, for bonus points, it would be nice to choose a hue ramp that
> > still works if you throw away the luminance variation, because then we
> > could use the version with varying luminance for 2d plots, and the
> > version with just hue variation for 3d plots. (In 3d plots you really
> > want to reserve the luminance channel for lighting/shading, because
> > your brain is *really* good at extracting 3d shape from luminance
> > variation. If the 3d surface itself has massively varying luminance
> > then this screws up the ability to see shape.)
>
> Just out of interest, is there currently an easy way in matplotlib of
> producing a 3d plot where luminance is used for lighting/shading as you
> suggest?
>
>
> Now the question is: how do we actually *design* a colormap with these
> requirements? Leon Krischer's notebook [1] looks totally awesome, but if I
> understand correctly the optimisation he uses "only" takes care of
> linearising the luminance value, but this does not necessarily guarantee
> that the hue values are also linear, right? It also feels somewhat clumsy
> to me to start out with a colormap that's "wrong" (w.r.t. our requirements
> above) and then "fix" it. However, the notebook looks like a great guidance
> for finding suitable candidates and assessing their quality.
>
> It appears to me that a simple yet effective way of coming up with a good
> colormap would be to pick two points in the L*a*b* color space that can be
> represented by RGB values, connect them by a line and use the interpolated
> values for the resulting colormap. Since L*a*b* space is (close to)
> perceptually linear, this would pretty much guarantee all the requirements
> above.
>
> What's missing is an easy way of doing this. I'm envisaging a simply GUI
> which allows the user to easily pick two points in L*a*b* space, generates
> a colormap from them as described above and also generates a few sample
> plots to evaluate the quality of the colormap (along the lines of [1] or
> the numerous blog posts linked to in the discussion). I am close to having
> a prototype for such a GUI which should allow to do this relatively
> painlessly. I'll try to finish it up over the weekend and will post here
> once it's ready. Btw, if anyone has suggestions for sample datasets that
> can help in assessing the quality of colormaps they would be much
> appreciated.
>
> Any comments or clarifications of points that I misunderstood are very
> welcome.
>
> Best wishes,
> Max
>
> [1] http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>

Re: [matplotlib-devel] Matplotlib's new default colormap

[Phil E. <pel...@gm...>]

P.S. I just found
http://davidjohnstone.net/pages/lch-lab-colour-gradient-picker

On 22 December 2014 at 11:21, Phil Elson <pel...@gm...> wrote:

> Thanks for all the contributions so far. Looks like matplotlib is blessed
> with people who are far more knowledgeable than I on the subject, and I'd
> say we were pretty much at a consensus on requirements.
>
> Given these requirements, what we need is some proposed colormaps - Max's
> approach of generating an optimal solution in LAB space sounds interesting,
> as do the other approaches of minimising some parameters of existing
> colormaps.
>
> To facilitate this discussion, do we need a repository of proposed
> colormaps so that we can compare like with like? I notice that Mike Bostock
> has an interesting post to show various color spaces in d3.js which may be
> of interest http://bl.ocks.org/mbostock/3014589.
>
>
>
> On 4 December 2014 at 14:38, Maximilian Albert <
> max...@gm...> wrote:
>
>> Hi all,
>>
>> I had a discussion with Phil Elson about this last weekend during the
>> Bloomberg Open Source Day. I don't consider myself an expert on colormaps
>> by any means, but I started digging into them a while ago when I was
>> looking for a way of generating a perceptually linear *cyclic* colormap in
>> order to represent phase angle values. (I've been meaning to discuss this
>> issue on this list for a while but will do so in a different thread once I
>> get around to tidying up my results so far.) Phil encouraged me to reply to
>> this thread because he said that even non-expert views would be very
>> welcome, so here you go.
>>
>> Basically, I agree with most of what Nathaniel Smith suggested in his
>> email from November 21. I'm going to comment on some of his points inline
>> below and will finally suggest a way of designing a new colormap at the end.
>>
>>
>> Nathaniel Smith wrote:
>>
>> > it should be a sequential colormap [...]
>>
>> Agreed.
>>
>> > it should be perceptually uniform
>>
>> Agreed.
>>
>> > There's lots of research on how to measure perceptual distance --
>> > a colleague and I happen to have recently implemented a
>> > state-of-the-art model of this for another project, in case anyone
>> > wants to play with it [3].
>>
>> I haven't had time to check this out in detail yet, but it looks pretty
>> interesting and will certainly be very useful to assess the quality of any
>> suggestions. However, can this help to actually *design* a new colormap?
>> The answer might be hidden in the referenced paper [Luo2006], but I haven't
>> read it yet.
>>
>> > or just using good-old-L*a*b* is a reasonable quick-and-dirty
>> approximation.
>>
>> Can you elaborate how "dirty" you think using L*a*b* would be? (See my
>> suggestion for finding a new colormap below.)
>>
>> >- it should have a perceptually uniform luminance ramp, i.e. if you
>> > convert to greyscale it should still be uniform.
>>
>> Agreed. What's unclear to me is how large this luminance ramp should be.
>> We certainly can't go all the way to full luminance because this won't be
>> visible on a white background. This probably needs experimenting (again see
>> below).
>>
>> > - it should also have some kind of variation in hue, because
>> > hue variation is a really helpful additional cue to perception,
>> > having two cues is better than one, and there's no reason
>> > not to do it.
>>
>> Agreed.
>>
>> > - the hue variation should be chosen to produce reasonable results
>> > even for viewers with the more common types of colorblindness.
>> > (Which rules out things like red-to-green.)
>>
>> Agreed. Are you aware of any simple ways of avoiding the most common
>> issues? Are there any blog posts or papers on designing colormaps that are
>> suitable for colorblind people?
>>
>> > And, for bonus points, it would be nice to choose a hue ramp that
>> > still works if you throw away the luminance variation, because then we
>> > could use the version with varying luminance for 2d plots, and the
>> > version with just hue variation for 3d plots. (In 3d plots you really
>> > want to reserve the luminance channel for lighting/shading, because
>> > your brain is *really* good at extracting 3d shape from luminance
>> > variation. If the 3d surface itself has massively varying luminance
>> > then this screws up the ability to see shape.)
>>
>> Just out of interest, is there currently an easy way in matplotlib of
>> producing a 3d plot where luminance is used for lighting/shading as you
>> suggest?
>>
>>
>> Now the question is: how do we actually *design* a colormap with these
>> requirements? Leon Krischer's notebook [1] looks totally awesome, but if I
>> understand correctly the optimisation he uses "only" takes care of
>> linearising the luminance value, but this does not necessarily guarantee
>> that the hue values are also linear, right? It also feels somewhat clumsy
>> to me to start out with a colormap that's "wrong" (w.r.t. our requirements
>> above) and then "fix" it. However, the notebook looks like a great guidance
>> for finding suitable candidates and assessing their quality.
>>
>> It appears to me that a simple yet effective way of coming up with a good
>> colormap would be to pick two points in the L*a*b* color space that can be
>> represented by RGB values, connect them by a line and use the interpolated
>> values for the resulting colormap. Since L*a*b* space is (close to)
>> perceptually linear, this would pretty much guarantee all the requirements
>> above.
>>
>> What's missing is an easy way of doing this. I'm envisaging a simply GUI
>> which allows the user to easily pick two points in L*a*b* space, generates
>> a colormap from them as described above and also generates a few sample
>> plots to evaluate the quality of the colormap (along the lines of [1] or
>> the numerous blog posts linked to in the discussion). I am close to having
>> a prototype for such a GUI which should allow to do this relatively
>> painlessly. I'll try to finish it up over the weekend and will post here
>> once it's ready. Btw, if anyone has suggestions for sample datasets that
>> can help in assessing the quality of colormaps they would be much
>> appreciated.
>>
>> Any comments or clarifications of points that I misunderstood are very
>> welcome.
>>
>> Best wishes,
>> Max
>>
>> [1] http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>

Re: [matplotlib-devel] Matplotlib's new default colormap

[Michael D. <md...@st...>]

I, for one, would love to see a pull request for this if you're game.

Mike

On 11/24/2014 04:27 AM, Lion Krischer wrote:
> Hi all,
>
> I was made aware of this thread and thought I’d share a notebook I 
> recently made for a similar purpose:
>
> http://nbviewer.ipython.org/gist/krischer/d35096a9d3b6da5846a5 (takes 
> a while to load…)
>
> It attempts to “optimize" colormaps by defining optimality as having a 
> linear lightness across the colormap in LAB color space. It is very 
> simple and not a proper optimization procedure. It just goes to LAB 
> space, sets the lightness to the target lightness, and goes back to 
> sRGB space. This does not always work as the LAB color space is much 
> bigger than the RGB one but in many cases it produces fairly good results.
>
> The nice thing about this is that the lightness range can be chosen so 
> it is does not always have to be stark white or black at the ends and 
> some hue can be preserved.
>
> I am not sure if some similar functionality is useful to include into 
> matplotlib (I don’t really think so) but if yes, let me know and I’ll 
> give it a try. I guess it could also be extended to optimize towards 
> monotonic changes in hue.
>
> Cheers and all the best!
>
> Lion
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute

http://www.droettboom.com