From: Phil E. <pel...@gm...> - 2014-11-21 17:32:59
|
Please use this thread to discuss the best choice for a new *default* matplotlib colormap. This follows on from a discussion on the matplotlib-devel mailing list entitled "How to move beyond JET as the default matplotlib colormap". It is accepted that there can never be a *best* colormap for *all* data, so some documentation on choosing an appropriate colormap for specific data should always be sought. Nonetheless, matplotlib does need a default, and it has been shown just how damaging the Jet (matplotlib's current default) colormap really is, so we need to come up with a genuine alternative. To keep this thread as useful as possible please avoid posting "+1" type messages. If you'd like to suggest a colormap for consideration as matplotlib's new *default* please try to keep to reference-able/demonstrable fact. Thanks, Phil |
From: Darren D. <dsd...@gm...> - 2014-11-21 17:46:17
|
On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote: > Please use this thread to discuss the best choice for a new *default* > matplotlib colormap. > > This follows on from a discussion on the matplotlib-devel mailing list > entitled "How to move beyond JET as the default matplotlib colormap". > I remember reading a (peer-reviewed, I think) article about how "jet" was a very unfortunate choice of default. I can't find the exact article now, but I did find some other useful ones: http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html http://www.sandia.gov/~kmorel/documents/ColorMaps/ http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf Darren |
From: Nathaniel S. <nj...@po...> - 2014-11-21 21:42:47
|
On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote: > On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote: >> >> Please use this thread to discuss the best choice for a new default >> matplotlib colormap. >> >> This follows on from a discussion on the matplotlib-devel mailing list >> entitled "How to move beyond JET as the default matplotlib colormap". > > > I remember reading a (peer-reviewed, I think) article about how "jet" was a > very unfortunate choice of default. I can't find the exact article now, but > I did find some other useful ones: > > http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html > http://www.sandia.gov/~kmorel/documents/ColorMaps/ > http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf Those are good articles. There's a lot of literature on the problems with "jet", and lots of links in the matplotlib issue [1]. For those trying to get up to speed quickly, MathWorks recently put together a nice review of the literature [2]. One particularly striking paper they cite studied a group of medical students and found that (a) they were used to/practiced at using jet, (b) when given a choice of colormaps they said that they preferred jet, (c) they nonetheless made more *medical diagnostic errors* when using jet than with better designed colormaps (Borkin et al, 2011). I won't suggest a specific colormap, but I do propose that whatever we chose satisfy the following criteria: - it should be a sequential colormap, because diverging colormaps are really misleading unless you know where the "center" of the data is, and for a default colormap we generally won't. - it should be perceptually uniform, i.e., human subjective judgements of how far apart nearby colors are should correspond as linearly as possible to the difference between the numerical values they represent, at least locally. There's lots of research on how to measure perceptual distance -- a colleague and I happen to have recently implemented a state-of-the-art model of this for another project, in case anyone wants to play with it [3], or just using good-old-L*a*b* is a reasonable quick-and-dirty approximation. - it should have a perceptually uniform luminance ramp, i.e. if you convert to greyscale it should still be uniform. This is useful both in practical terms (greyscale printers are still a thing!) and because luminance is a very strong and natural cue to magnitude. - it should also have some kind of variation in hue, because hue variation is a really helpful additional cue to perception, having two cues is better than one, and there's no reason not to do it. - the hue variation should be chosen to produce reasonable results even for viewers with the more common types of colorblindness. (Which rules out things like red-to-green.) And, for bonus points, it would be nice to choose a hue ramp that still works if you throw away the luminance variation, because then we could use the version with varying luminance for 2d plots, and the version with just hue variation for 3d plots. (In 3d plots you really want to reserve the luminance channel for lighting/shading, because your brain is *really* good at extracting 3d shape from luminance variation. If the 3d surface itself has massively varying luminance then this screws up the ability to see shape.) Do these seem like good requirements? -n [1] https://github.com/matplotlib/matplotlib/issues/875 [2] http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html [3] https://github.com/njsmith/pycam02ucs ; install (or just run out of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute the perceptual distance between two RGB colors. It's also possible to use the underlying color model stuff to do things like generate colors with evenly spaced luminance and hues, or draw 3d plots of the shape of the human color space. It's pretty fun to play with :-) -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org |
From: Eric F. <ef...@ha...> - 2014-11-21 23:25:09
|
On 2014/11/21, 4:42 PM, Nathaniel Smith wrote: > On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote: >> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote: >>> >>> Please use this thread to discuss the best choice for a new default >>> matplotlib colormap. >>> >>> This follows on from a discussion on the matplotlib-devel mailing list >>> entitled "How to move beyond JET as the default matplotlib colormap". >> >> >> I remember reading a (peer-reviewed, I think) article about how "jet" was a >> very unfortunate choice of default. I can't find the exact article now, but >> I did find some other useful ones: >> >> http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf > > Those are good articles. There's a lot of literature on the problems > with "jet", and lots of links in the matplotlib issue [1]. For those > trying to get up to speed quickly, MathWorks recently put together a > nice review of the literature [2]. One particularly striking paper > they cite studied a group of medical students and found that (a) they > were used to/practiced at using jet, (b) when given a choice of > colormaps they said that they preferred jet, (c) they nonetheless made > more *medical diagnostic errors* when using jet than with better > designed colormaps (Borkin et al, 2011). > > I won't suggest a specific colormap, but I do propose that whatever we > chose satisfy the following criteria: > > - it should be a sequential colormap, because diverging colormaps are > really misleading unless you know where the "center" of the data is, > and for a default colormap we generally won't. > > - it should be perceptually uniform, i.e., human subjective judgements > of how far apart nearby colors are should correspond as linearly as > possible to the difference between the numerical values they > represent, at least locally. There's lots of research on how to > measure perceptual distance -- a colleague and I happen to have > recently implemented a state-of-the-art model of this for another > project, in case anyone wants to play with it [3], or just using > good-old-L*a*b* is a reasonable quick-and-dirty approximation. > > - it should have a perceptually uniform luminance ramp, i.e. if you > convert to greyscale it should still be uniform. This is useful both > in practical terms (greyscale printers are still a thing!) and because > luminance is a very strong and natural cue to magnitude. > > - it should also have some kind of variation in hue, because hue > variation is a really helpful additional cue to perception, having two > cues is better than one, and there's no reason not to do it. > > - the hue variation should be chosen to produce reasonable results > even for viewers with the more common types of colorblindness. (Which > rules out things like red-to-green.) > > And, for bonus points, it would be nice to choose a hue ramp that > still works if you throw away the luminance variation, because then we > could use the version with varying luminance for 2d plots, and the > version with just hue variation for 3d plots. (In 3d plots you really > want to reserve the luminance channel for lighting/shading, because > your brain is *really* good at extracting 3d shape from luminance > variation. If the 3d surface itself has massively varying luminance > then this screws up the ability to see shape.) > > Do these seem like good requirements? Goals, yes, though I wouldn't put much weight on the "bonus" criterion. I would add that it should be aesthetically pleasing, or at least comfortable, to most people. Perfection might not be attainable, and some tradeoffs may be required. Is anyone set up to produce test images and/or metrics for judging existing colormaps, or newly designed ones, on all of these criteria? Eric > > -n > > [1] https://github.com/matplotlib/matplotlib/issues/875 > [2] http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html > [3] https://github.com/njsmith/pycam02ucs ; install (or just run out > of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute > the perceptual distance between two RGB colors. It's also possible to > use the underlying color model stuff to do things like generate colors > with evenly spaced luminance and hues, or draw 3d plots of the shape > of the human color space. It's pretty fun to play with :-) > |
From: gary r. <gar...@gm...> - 2014-11-22 14:32:05
|
There was a talk by Kristen Thyng at scipy2014 that might be a good backgrounder for this: http://pyvideo.org/video/2769/perceptions-of-matplotlib-colormaps At the end she references this site http://mycarta.wordpress.com/ of Matteo Niccoli which is full of good content. I wonder if it's worth contacting Kristen or Matteo to let them know there's a discussion going on here that they might be interested in? On 22 November 2014 at 09:53, Eric Firing <ef...@ha...> wrote: > On 2014/11/21, 4:42 PM, Nathaniel Smith wrote: > > On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote: > >> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> > wrote: > >>> > >>> Please use this thread to discuss the best choice for a new default > >>> matplotlib colormap. > >>> > >>> This follows on from a discussion on the matplotlib-devel mailing list > >>> entitled "How to move beyond JET as the default matplotlib colormap". > >> > >> > >> I remember reading a (peer-reviewed, I think) article about how "jet" > was a > >> very unfortunate choice of default. I can't find the exact article now, > but > >> I did find some other useful ones: > >> > >> > http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html > >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ > >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf > > > > Those are good articles. There's a lot of literature on the problems > > with "jet", and lots of links in the matplotlib issue [1]. For those > > trying to get up to speed quickly, MathWorks recently put together a > > nice review of the literature [2]. One particularly striking paper > > they cite studied a group of medical students and found that (a) they > > were used to/practiced at using jet, (b) when given a choice of > > colormaps they said that they preferred jet, (c) they nonetheless made > > more *medical diagnostic errors* when using jet than with better > > designed colormaps (Borkin et al, 2011). > > > > I won't suggest a specific colormap, but I do propose that whatever we > > chose satisfy the following criteria: > > > > - it should be a sequential colormap, because diverging colormaps are > > really misleading unless you know where the "center" of the data is, > > and for a default colormap we generally won't. > > > > - it should be perceptually uniform, i.e., human subjective judgements > > of how far apart nearby colors are should correspond as linearly as > > possible to the difference between the numerical values they > > represent, at least locally. There's lots of research on how to > > measure perceptual distance -- a colleague and I happen to have > > recently implemented a state-of-the-art model of this for another > > project, in case anyone wants to play with it [3], or just using > > good-old-L*a*b* is a reasonable quick-and-dirty approximation. > > > > - it should have a perceptually uniform luminance ramp, i.e. if you > > convert to greyscale it should still be uniform. This is useful both > > in practical terms (greyscale printers are still a thing!) and because > > luminance is a very strong and natural cue to magnitude. > > > > - it should also have some kind of variation in hue, because hue > > variation is a really helpful additional cue to perception, having two > > cues is better than one, and there's no reason not to do it. > > > > - the hue variation should be chosen to produce reasonable results > > even for viewers with the more common types of colorblindness. (Which > > rules out things like red-to-green.) > > > > And, for bonus points, it would be nice to choose a hue ramp that > > still works if you throw away the luminance variation, because then we > > could use the version with varying luminance for 2d plots, and the > > version with just hue variation for 3d plots. (In 3d plots you really > > want to reserve the luminance channel for lighting/shading, because > > your brain is *really* good at extracting 3d shape from luminance > > variation. If the 3d surface itself has massively varying luminance > > then this screws up the ability to see shape.) > > > > Do these seem like good requirements? > > Goals, yes, though I wouldn't put much weight on the "bonus" criterion. > I would add that it should be aesthetically pleasing, or at least > comfortable, to most people. Perfection might not be attainable, and > some tradeoffs may be required. Is anyone set up to produce test images > and/or metrics for judging existing colormaps, or newly designed ones, > on all of these criteria? > > Eric > > > > > -n > > > > [1] https://github.com/matplotlib/matplotlib/issues/875 > > [2] > http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html > > [3] https://github.com/njsmith/pycam02ucs ; install (or just run out > > of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute > > the perceptual distance between two RGB colors. It's also possible to > > use the underlying color model stuff to do things like generate colors > > with evenly spaced luminance and hues, or draw 3d plots of the shape > > of the human color space. It's pretty fun to play with :-) > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > |
From: Thomas C. <tca...@gm...> - 2014-11-22 16:38:33
|
The contents of that talk are also in our documentation http://matplotlib.org/users/colormaps.html Tom On Sat Nov 22 2014 at 9:33:11 AM gary ruben <gar...@gm...> wrote: > There was a talk by Kristen Thyng at scipy2014 that might be a good > backgrounder for this: > http://pyvideo.org/video/2769/perceptions-of-matplotlib-colormaps > > At the end she references this site http://mycarta.wordpress.com/ of > Matteo Niccoli which is full of good content. I wonder if it's worth > contacting Kristen or Matteo to let them know there's a discussion going on > here that they might be interested in? > > > On 22 November 2014 at 09:53, Eric Firing <ef...@ha...> wrote: > >> On 2014/11/21, 4:42 PM, Nathaniel Smith wrote: >> > On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> >> wrote: >> >> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> >> wrote: >> >>> >> >>> Please use this thread to discuss the best choice for a new default >> >>> matplotlib colormap. >> >>> >> >>> This follows on from a discussion on the matplotlib-devel mailing list >> >>> entitled "How to move beyond JET as the default matplotlib colormap". >> >> >> >> >> >> I remember reading a (peer-reviewed, I think) article about how "jet" >> was a >> >> very unfortunate choice of default. I can't find the exact article >> now, but >> >> I did find some other useful ones: >> >> >> >> >> http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html >> >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ >> >> >> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf >> > >> > Those are good articles. There's a lot of literature on the problems >> > with "jet", and lots of links in the matplotlib issue [1]. For those >> > trying to get up to speed quickly, MathWorks recently put together a >> > nice review of the literature [2]. One particularly striking paper >> > they cite studied a group of medical students and found that (a) they >> > were used to/practiced at using jet, (b) when given a choice of >> > colormaps they said that they preferred jet, (c) they nonetheless made >> > more *medical diagnostic errors* when using jet than with better >> > designed colormaps (Borkin et al, 2011). >> > >> > I won't suggest a specific colormap, but I do propose that whatever we >> > chose satisfy the following criteria: >> > >> > - it should be a sequential colormap, because diverging colormaps are >> > really misleading unless you know where the "center" of the data is, >> > and for a default colormap we generally won't. >> > >> > - it should be perceptually uniform, i.e., human subjective judgements >> > of how far apart nearby colors are should correspond as linearly as >> > possible to the difference between the numerical values they >> > represent, at least locally. There's lots of research on how to >> > measure perceptual distance -- a colleague and I happen to have >> > recently implemented a state-of-the-art model of this for another >> > project, in case anyone wants to play with it [3], or just using >> > good-old-L*a*b* is a reasonable quick-and-dirty approximation. >> > >> > - it should have a perceptually uniform luminance ramp, i.e. if you >> > convert to greyscale it should still be uniform. This is useful both >> > in practical terms (greyscale printers are still a thing!) and because >> > luminance is a very strong and natural cue to magnitude. >> > >> > - it should also have some kind of variation in hue, because hue >> > variation is a really helpful additional cue to perception, having two >> > cues is better than one, and there's no reason not to do it. >> > >> > - the hue variation should be chosen to produce reasonable results >> > even for viewers with the more common types of colorblindness. (Which >> > rules out things like red-to-green.) >> > >> > And, for bonus points, it would be nice to choose a hue ramp that >> > still works if you throw away the luminance variation, because then we >> > could use the version with varying luminance for 2d plots, and the >> > version with just hue variation for 3d plots. (In 3d plots you really >> > want to reserve the luminance channel for lighting/shading, because >> > your brain is *really* good at extracting 3d shape from luminance >> > variation. If the 3d surface itself has massively varying luminance >> > then this screws up the ability to see shape.) >> > >> > Do these seem like good requirements? >> >> Goals, yes, though I wouldn't put much weight on the "bonus" criterion. >> I would add that it should be aesthetically pleasing, or at least >> comfortable, to most people. Perfection might not be attainable, and >> some tradeoffs may be required. Is anyone set up to produce test images >> and/or metrics for judging existing colormaps, or newly designed ones, >> on all of these criteria? >> >> Eric >> >> > >> > -n >> > >> > [1] https://github.com/matplotlib/matplotlib/issues/875 >> > [2] >> http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html >> > [3] https://github.com/njsmith/pycam02ucs ; install (or just run out >> > of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute >> > the perceptual distance between two RGB colors. It's also possible to >> > use the underlying color model stuff to do things like generate colors >> > with evenly spaced luminance and hues, or draw 3d plots of the shape >> > of the human color space. It's pretty fun to play with :-) >> > >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk >> _______________________________________________ >> Matplotlib-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> > > ------------------------------------------------------------ > ------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=157005751& > iu=/4140/ostg.clktrk_______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > |
From: Nathaniel S. <nj...@po...> - 2014-12-23 09:17:12
Attachments:
jet.png
YlGnBu_r.pdf
|
On Fri, Nov 21, 2014 at 10:53 PM, Eric Firing <ef...@ha...> wrote: > On 2014/11/21, 4:42 PM, Nathaniel Smith wrote: >> On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote: >>> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote: >>>> >>>> Please use this thread to discuss the best choice for a new default >>>> matplotlib colormap. >>>> >>>> This follows on from a discussion on the matplotlib-devel mailing list >>>> entitled "How to move beyond JET as the default matplotlib colormap". >>> >>> >>> I remember reading a (peer-reviewed, I think) article about how "jet" was a >>> very unfortunate choice of default. I can't find the exact article now, but >>> I did find some other useful ones: >>> >>> http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html >>> http://www.sandia.gov/~kmorel/documents/ColorMaps/ >>> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf >> >> Those are good articles. There's a lot of literature on the problems >> with "jet", and lots of links in the matplotlib issue [1]. For those >> trying to get up to speed quickly, MathWorks recently put together a >> nice review of the literature [2]. One particularly striking paper >> they cite studied a group of medical students and found that (a) they >> were used to/practiced at using jet, (b) when given a choice of >> colormaps they said that they preferred jet, (c) they nonetheless made >> more *medical diagnostic errors* when using jet than with better >> designed colormaps (Borkin et al, 2011). >> >> I won't suggest a specific colormap, but I do propose that whatever we >> chose satisfy the following criteria: >> >> - it should be a sequential colormap, because diverging colormaps are >> really misleading unless you know where the "center" of the data is, >> and for a default colormap we generally won't. >> >> - it should be perceptually uniform, i.e., human subjective judgements >> of how far apart nearby colors are should correspond as linearly as >> possible to the difference between the numerical values they >> represent, at least locally. There's lots of research on how to >> measure perceptual distance -- a colleague and I happen to have >> recently implemented a state-of-the-art model of this for another >> project, in case anyone wants to play with it [3], or just using >> good-old-L*a*b* is a reasonable quick-and-dirty approximation. >> >> - it should have a perceptually uniform luminance ramp, i.e. if you >> convert to greyscale it should still be uniform. This is useful both >> in practical terms (greyscale printers are still a thing!) and because >> luminance is a very strong and natural cue to magnitude. >> >> - it should also have some kind of variation in hue, because hue >> variation is a really helpful additional cue to perception, having two >> cues is better than one, and there's no reason not to do it. >> >> - the hue variation should be chosen to produce reasonable results >> even for viewers with the more common types of colorblindness. (Which >> rules out things like red-to-green.) >> >> And, for bonus points, it would be nice to choose a hue ramp that >> still works if you throw away the luminance variation, because then we >> could use the version with varying luminance for 2d plots, and the >> version with just hue variation for 3d plots. (In 3d plots you really >> want to reserve the luminance channel for lighting/shading, because >> your brain is *really* good at extracting 3d shape from luminance >> variation. If the 3d surface itself has massively varying luminance >> then this screws up the ability to see shape.) >> >> Do these seem like good requirements? > > Goals, yes, though I wouldn't put much weight on the "bonus" criterion. > I would add that it should be aesthetically pleasing, or at least > comfortable, to most people. Perfection might not be attainable, and > some tradeoffs may be required. Is anyone set up to produce test images > and/or metrics for judging existing colormaps, or newly designed ones, > on all of these criteria? I had some time on a plane today, so I wrote a little script for visualizing colormaps (esp. WRT perceptual uniformity and colorblindness). To try it: $ git clone https://github.com/njsmith/pycam02ucs.git $ cd pycam02ucs $ ipython In [1]: %matplotlib In [2]: from pycam02ucs.viscm import viscm In [3]: viscm("jet") (Or substitute your favorite built-in colormap, or pass a matplotlib colormap object, i.e. a callable that takes an array of values in the range [0, 1] and returns an array of RGBA values with shape (n, 4).) I'm attaching an example, plus an annotated example explaining what the different bits show. It's a bit crude, but has definitely reached the fun-to-play-around-with stage :-). If anyone makes improvements send me a PR! Hidden feature: you can pass show_gamut=True to get a crude approximation of the space of possible sRGB colors drawn onto the 3d plot at the bottom. The idea is if trying to design a better colormap it's useful to have a sense of what potential colors are available to use. It's pretty crude and somewhat distracting though so I left it off by default for now. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org |
From: Benjamin R. <ben...@ou...> - 2014-12-30 22:23:13
|
Neat stuff! Just a quick note about the 3D plot. By default, the scatter markers are shaded to give an illusion of depth. This is often what we want, but I think in this case, it might make sense to not do that. Add depthshade=False to the scatter call to turn it off. I think that was added for mpl version 1.3. Ben Root On Tue, Dec 23, 2014 at 4:16 AM, Nathaniel Smith <nj...@po...> wrote: > On Fri, Nov 21, 2014 at 10:53 PM, Eric Firing <ef...@ha...> wrote: > > On 2014/11/21, 4:42 PM, Nathaniel Smith wrote: > >> On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> > wrote: > >>> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> > wrote: > >>>> > >>>> Please use this thread to discuss the best choice for a new default > >>>> matplotlib colormap. > >>>> > >>>> This follows on from a discussion on the matplotlib-devel mailing list > >>>> entitled "How to move beyond JET as the default matplotlib colormap". > >>> > >>> > >>> I remember reading a (peer-reviewed, I think) article about how "jet" > was a > >>> very unfortunate choice of default. I can't find the exact article > now, but > >>> I did find some other useful ones: > >>> > >>> > http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html > >>> http://www.sandia.gov/~kmorel/documents/ColorMaps/ > >>> > http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf > >> > >> Those are good articles. There's a lot of literature on the problems > >> with "jet", and lots of links in the matplotlib issue [1]. For those > >> trying to get up to speed quickly, MathWorks recently put together a > >> nice review of the literature [2]. One particularly striking paper > >> they cite studied a group of medical students and found that (a) they > >> were used to/practiced at using jet, (b) when given a choice of > >> colormaps they said that they preferred jet, (c) they nonetheless made > >> more *medical diagnostic errors* when using jet than with better > >> designed colormaps (Borkin et al, 2011). > >> > >> I won't suggest a specific colormap, but I do propose that whatever we > >> chose satisfy the following criteria: > >> > >> - it should be a sequential colormap, because diverging colormaps are > >> really misleading unless you know where the "center" of the data is, > >> and for a default colormap we generally won't. > >> > >> - it should be perceptually uniform, i.e., human subjective judgements > >> of how far apart nearby colors are should correspond as linearly as > >> possible to the difference between the numerical values they > >> represent, at least locally. There's lots of research on how to > >> measure perceptual distance -- a colleague and I happen to have > >> recently implemented a state-of-the-art model of this for another > >> project, in case anyone wants to play with it [3], or just using > >> good-old-L*a*b* is a reasonable quick-and-dirty approximation. > >> > >> - it should have a perceptually uniform luminance ramp, i.e. if you > >> convert to greyscale it should still be uniform. This is useful both > >> in practical terms (greyscale printers are still a thing!) and because > >> luminance is a very strong and natural cue to magnitude. > >> > >> - it should also have some kind of variation in hue, because hue > >> variation is a really helpful additional cue to perception, having two > >> cues is better than one, and there's no reason not to do it. > >> > >> - the hue variation should be chosen to produce reasonable results > >> even for viewers with the more common types of colorblindness. (Which > >> rules out things like red-to-green.) > >> > >> And, for bonus points, it would be nice to choose a hue ramp that > >> still works if you throw away the luminance variation, because then we > >> could use the version with varying luminance for 2d plots, and the > >> version with just hue variation for 3d plots. (In 3d plots you really > >> want to reserve the luminance channel for lighting/shading, because > >> your brain is *really* good at extracting 3d shape from luminance > >> variation. If the 3d surface itself has massively varying luminance > >> then this screws up the ability to see shape.) > >> > >> Do these seem like good requirements? > > > > Goals, yes, though I wouldn't put much weight on the "bonus" criterion. > > I would add that it should be aesthetically pleasing, or at least > > comfortable, to most people. Perfection might not be attainable, and > > some tradeoffs may be required. Is anyone set up to produce test images > > and/or metrics for judging existing colormaps, or newly designed ones, > > on all of these criteria? > > I had some time on a plane today, so I wrote a little script for > visualizing colormaps (esp. WRT perceptual uniformity and > colorblindness). To try it: > > $ git clone https://github.com/njsmith/pycam02ucs.git > $ cd pycam02ucs > $ ipython > In [1]: %matplotlib > In [2]: from pycam02ucs.viscm import viscm > In [3]: viscm("jet") > > (Or substitute your favorite built-in colormap, or pass a matplotlib > colormap object, i.e. a callable that takes an array of values in the > range [0, 1] and returns an array of RGBA values with shape (n, 4).) > > I'm attaching an example, plus an annotated example explaining what > the different bits show. > > It's a bit crude, but has definitely reached the > fun-to-play-around-with stage :-). If anyone makes improvements send > me a PR! > > Hidden feature: you can pass show_gamut=True to get a crude > approximation of the space of possible sRGB colors drawn onto the 3d > plot at the bottom. The idea is if trying to design a better colormap > it's useful to have a sense of what potential colors are available to > use. It's pretty crude and somewhat distracting though so I left it > off by default for now. > > -n > > -- > Nathaniel J. Smith > Postdoctoral researcher - Informatics - University of Edinburgh > http://vorpus.org > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is > your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > > |
From: Maximilian A. <max...@gm...> - 2015-01-06 02:37:59
|
Happy new year everyone! Apologies for the long silence. I was snowed in with work before Christmas and then mostly cut off from the internet for the past two weeks. Fortunately, I had a chance over the holidays to flesh out the GUI which I mentioned in my previous email. You can find it here: https://github.com/maxalbert/colormap-selector Basically, it allows you to pick the start/end color of a colormap from two cross sections in CIELab space and interpolates those colors linearly (see the README file for more details). Currently there is one scatterplot to illustrate the resulting colormap but it can be trivially extended to show more interesting sample plots. There are still a few things missing that I'd like to add but at least it's in a state where it can be used and I'd be grateful for feedback, especially with regard to the colormaps generated with it (I do have some opinions myself but it would be interesting to hear others' first). Regarding our ongoing discussion, I had a very useful chat with two colleagues before Christmas which spurred more thoughts. But I guess it's best to discuss them in a separate email when I'm less tired. ;) Best wishes, Max |
From: Federico A. <ari...@gm...> - 2015-01-08 17:44:32
|
Nice job. I find your GUI a little bit confusing (new to colormap stuff) but I like the idea, basically I find it overkill, I would replace the gui by a plot and a couple of slider widgets something simpler to integrate without new dependencies. Do you really need the third 3d plot on the right? On Mon, Jan 5, 2015 at 9:37 PM, Maximilian Albert <max...@gm...> wrote: > Happy new year everyone! > > Apologies for the long silence. I was snowed in with work before Christmas > and then mostly cut off from the internet for the past two weeks. > Fortunately, I had a chance over the holidays to flesh out the GUI which I > mentioned in my previous email. You can find it here: > > https://github.com/maxalbert/colormap-selector > > Basically, it allows you to pick the start/end color of a colormap from two > cross sections in CIELab space and interpolates those colors linearly (see > the README file for more details). Currently there is one scatterplot to > illustrate the resulting colormap but it can be trivially extended to show > more interesting sample plots. There are still a few things missing that I'd > like to add but at least it's in a state where it can be used and I'd be > grateful for feedback, especially with regard to the colormaps generated > with it (I do have some opinions myself but it would be interesting to hear > others' first). > > Regarding our ongoing discussion, I had a very useful chat with two > colleagues before Christmas which spurred more thoughts. But I guess it's > best to discuss them in a separate email when I'm less tired. ;) > > Best wishes, > Max > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Y yo que culpa tengo de que ellas se crean todo lo que yo les digo? -- Antonio Alducin -- |
From: Maximilian A. <max...@gm...> - 2015-01-08 22:20:20
|
Hi Federico, Thanks for trying it out and for the feedback! Indeed, I started out writing a simple IPython notebook along the lines you suggested, with just a couple of sliders and plots, but it quickly became too slow and unwieldy for quick explorations, hence the slightly more elaborate GUI. I agree that the reason for the 3D plot on the right may not be obvious at the moment. Personally, I find it useful to get a feel for what the representable colors in CIELab space (and the cross sections for L=const) look like, but when simply using a linear interpolation between two colors (as I'm doing at the moment) it may not be needed to visualise it in 3D. The reason I added it is that while playing around with the GUI I got the impression that my initial suggestion of using a simple linear interpolation between two colors may not result in the best-looking colormaps (this is confirmed by Nathaniel's reply). I'm currently toying with the option to use curved interpolations, and for thee it would be very useful IMHO to see what they look like in 3D. Btw, I have refactored my code a bit and it should be easy to write a simpler UI (e.g. in an IPython notebook) which doesn't need the other dependencies (also, I could drop the wxpython dependency because some conflict with Vispy which I had experienced seems to have disappeared). If you like, feel free to give it a shot to write a UI the way you imagine it. It's always good to have more options for exploration. :) Best wishes, Max 2015-01-08 17:44 GMT+00:00 Federico Ariza <ari...@gm...>: > Nice job. > > I find your GUI a little bit confusing (new to colormap stuff) but I > like the idea, basically I find it overkill, I would replace the gui > by a plot and a couple of slider widgets something simpler to > integrate without new dependencies. > Do you really need the third 3d plot on the right? > > On Mon, Jan 5, 2015 at 9:37 PM, Maximilian Albert > <max...@gm...> wrote: > > Happy new year everyone! > > > > Apologies for the long silence. I was snowed in with work before > Christmas > > and then mostly cut off from the internet for the past two weeks. > > Fortunately, I had a chance over the holidays to flesh out the GUI which > I > > mentioned in my previous email. You can find it here: > > > > https://github.com/maxalbert/colormap-selector > > > > Basically, it allows you to pick the start/end color of a colormap from > two > > cross sections in CIELab space and interpolates those colors linearly > (see > > the README file for more details). Currently there is one scatterplot to > > illustrate the resulting colormap but it can be trivially extended to > show > > more interesting sample plots. There are still a few things missing that > I'd > > like to add but at least it's in a state where it can be used and I'd be > > grateful for feedback, especially with regard to the colormaps generated > > with it (I do have some opinions myself but it would be interesting to > hear > > others' first). > > > > Regarding our ongoing discussion, I had a very useful chat with two > > colleagues before Christmas which spurred more thoughts. But I guess it's > > best to discuss them in a separate email when I'm less tired. ;) > > > > Best wishes, > > Max > > > > > ------------------------------------------------------------------------------ > > Dive into the World of Parallel Programming! The Go Parallel Website, > > sponsored by Intel and developed in partnership with Slashdot Media, is > your > > hub for all things parallel software development, from weekly thought > > leadership blogs to news, videos, case studies, tutorials and more. Take > a > > look and join the conversation now. http://goparallel.sourceforge.net > > _______________________________________________ > > Matplotlib-devel mailing list > > Mat...@li... > > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > > > > > > -- > Y yo que culpa tengo de que ellas se crean todo lo que yo les digo? > > -- Antonio Alducin -- > |
From: Nathaniel S. <nj...@po...> - 2015-01-08 17:54:55
|
On Tue, Jan 6, 2015 at 2:37 AM, Maximilian Albert <max...@gm...> wrote: > Happy new year everyone! > > Apologies for the long silence. I was snowed in with work before Christmas > and then mostly cut off from the internet for the past two weeks. > Fortunately, I had a chance over the holidays to flesh out the GUI which I > mentioned in my previous email. You can find it here: > > https://github.com/maxalbert/colormap-selector > > Basically, it allows you to pick the start/end color of a colormap from two > cross sections in CIELab space and interpolates those colors linearly (see > the README file for more details). There's a downside to this approach for the kinds of colormaps we've been talking about in this thread, where we want both a large lightness range plus a colorful result. The problem is that the way color space is shaped, you can't simultaneously have both high saturation (colorfulness) *and* high/low lightness. So if you pick your extreme points to be near black and white, then they can only have a slight tinting of color, and then if you linearly interpolate between these, then you end up with slightly tinted greyscale. Colormaps like YlGnBu or cubehelix or parula are designed to start out with low saturation, then as they move into the middle of the lightness scale they arc outwards, then arc back in again. This is a lot easier to visualize (e.g. by playing with the script I posted upthread) than it is to explain in text :-). Like, if you do viscm(YlGnBu_r) and look at the plot in the lower-right then it's clear that it's not a simple straight line in (J'/K, a', b') space (which is a higher-tech analogue to L* a* b* space). -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org |
From: Maximilian A. <max...@gm...> - 2015-01-08 22:30:25
Attachments:
screenshot_colormap_blue_to_yellow.png
|
Hi Nathaniel, > > Basically, it allows you to pick the start/end color of a colormap from > two > > cross sections in CIELab space and interpolates those colors linearly > (see > > the README file for more details). > > There's a downside to this approach for the kinds of colormaps we've > been talking about in this thread, where we want both a large > lightness range plus a colorful result. The problem is that the way > color space is shaped, you can't simultaneously have both high > saturation (colorfulness) *and* high/low lightness. So if you pick > your extreme points to be near black and white, then they can only > have a slight tinting of color, and then if you linearly interpolate > between these, then you end up with slightly tinted greyscale. > You raise an excellent point here. It explains nicely what I have experienced while playing with my GUI. Indeed, I found that a simple linear interpolation did not result in totally satisfactory colormaps (see my previous reply to Federico). I couldn't quite explain why, but your explanation makes this clear. One exception I encountered is an interpolation between dark blue and yellow as in the attached screenshot (which I hope makes it through to the mailing list) - probably because it mostly avoids the low-saturation (grey-ish) region of the color space. But I agree with you that using a curved, rather than linear, interpolation can probably yield better results. It sounds like you have a good deal of experience with various color spaces and colormaps. Do you have an idea for a good "recipe" how to pick a curve in a given colorspace that leads to a satisfactory colormap? My first idea was to change the interpolating line to a circular arc passing through an "intermediate" color, but it's not clear to me whether there is any preferred "direction" for nudging the line into an arc. Also, most other colormaps, such as the examples "YlGnBu" and "cubehelix" which you mentioned, use more complicated curves than mere circular arcs (btw, kudos for your script - it's a great way of visualising these colormaps). I don't have enough knowledge yet to decide whether either approach is better. I've started toying with curved interpolations in my code but this is not quite ready to be pushed to Github yet. Anyway, if you have any suggestions I'd love to hear them. I also found a few more blog posts and papers which I hadn't seen before and which look extremely useful: (i) "Subtleties of color" http://earthobservatory.nasa.gov/blogs/elegantfigures/2013/08/05/subtleties-of-color-part-1-of-6/ A series of six blog posts with an excellent introduction to color theory and the issues around choosing colormaps. Well worth a read! It also suggests that CIE L*c*h* space (which uses the three variables lightness, chroma (saturation) and hue), may be a better choice than CIE L*a*b*, which I have been using so far. (ii) "How To Avoid Equidistant HSV Colors" http://vis4.net/blog/posts/avoid-equidistant-hsv-colors/ Blog post with some interactive tools to visualise sections of CIE L*a*b* space and HCL (Hue-Chroma-Lightness) space. Here is a nice standalone version of the second tool: http://tristen.ca/hcl-picker/#/hlc/6/1/1B2531/E5FC74 (iii) "Generating Color Palettes using Intuitive Parameters" http://magnaview.nl/documents/MagnaView-M_Wijffelaars-Generating_color_palettes_using_intuitive_parameters.pdf Excellent-looking paper on the subject. I haven't read it in full yet but it looks like a great resource which might answer some of my questions above. At this stage I'm wondering how best to proceed. There seems to be huge number of resources and information, but we don't really have a clear path forward. I agree with Phil Elson's assessment when I talked to him at the Open Source Day: what we need is for someone to make a suggestion for a colormap and list a number of reasons why this particular one should be chosen. Then we have a basis for discussion and can argue about it. If anybody has such a suggestion yet, it would be great to hear about it (even if it is not perfect). Otherwise I'll try to make one once I have explored various options a bit more (although it may take a little while as my spare time is rather limited at the moment). Best wishes, Max |
From: Nathaniel S. <nj...@po...> - 2015-01-09 01:17:29
|
On Thu, Jan 8, 2015 at 10:30 PM, Maximilian Albert <max...@gm...> wrote: > Hi Nathaniel, > >> >> > Basically, it allows you to pick the start/end color of a colormap from >> > two >> > cross sections in CIELab space and interpolates those colors linearly >> > (see >> > the README file for more details). >> >> There's a downside to this approach for the kinds of colormaps we've >> been talking about in this thread, where we want both a large >> lightness range plus a colorful result. The problem is that the way >> color space is shaped, you can't simultaneously have both high >> saturation (colorfulness) *and* high/low lightness. So if you pick >> your extreme points to be near black and white, then they can only >> have a slight tinting of color, and then if you linearly interpolate >> between these, then you end up with slightly tinted greyscale. > > > You raise an excellent point here. It explains nicely what I have > experienced while playing with my GUI. Indeed, I found that a simple linear > interpolation did not result in totally satisfactory colormaps (see my > previous reply to Federico). I couldn't quite explain why, but your > explanation makes this clear. > > One exception I encountered is an interpolation between dark blue and yellow > as in the attached screenshot (which I hope makes it through to the mailing > list) - probably because it mostly avoids the low-saturation (grey-ish) > region of the color space. I guess this probably also has to do with another weird feature of how the colorspace is shaped. You'll often see pictures in books that illustrate it like two cones: http://www.tvtechnology.com/BE_Files/uploads/2013/05/ColorTopCones-305be18.jpg which does capture the general idea that your range of saturations is widest when lightness is in the middle, and narrows down when you move towards black or white. But it's actually a bit more complicated than that -- the actual shape is sorta lumpy, more like the picture here: http://www.gamutvision.com/ In particular, you can have pretty-saturated blues even at very low lightnesses, and pretty-saturated yellows even at high lightnesses. E.g. there literally does not exist a dark red that's as intense as the most intense dark blue. So this makes dark-blue-to-light-yellow the obvious way to go if you want a dark-to-light colormap that is also colorful. I don't think it's a coincidence that parula does exactly this :-) There is an obvious degree of freedom here though -- the color wheel is, like, a wheel, so if you want to go from blue to yellow you can do that either clockwise or counterclockwise, i.e., you can go through green or you can go through red. Parula goes via green (and so does matplotlib's YlGnBu, for that matter). If we want to have a distinctive colormap that people won't confuse with Matlab(R)(TM) then maybe we should make a blue-purple-red-yellow one. And in fact, this is probably theoretically optimal! As another weird quirk of how color works, the 4 focal colors (red/green/blue/yellow) are not actually at right angles to each other on the hue circle -- see the lower diagram on this figure: https://books.google.co.uk/books?id=MZ-TM0f2twAC&lpg=PA323&ots=XB_jHt0wz1&dq=%22the%20cie%20colour%20appearance%20model%22%20hunt%20and%20pointer&pg=PA307#v=onepage&q&f=false >From yellow-to-blue via red is a ~213 degree angle, while yellow-to-blue-via-green is only a ~147 degree angle (in a space where we define our "hue angle" based on perceptual just-noticeable-differences). So a blue-purple-red-yellow colormap should theoretically have higher perceptual resolution than a blue-green-yellow colormap. > But I agree with you that using a curved, rather > than linear, interpolation can probably yield better results. > > It sounds like you have a good deal of experience with various color spaces > and colormaps. Do you have an idea for a good "recipe" how to pick a curve > in a given colorspace that leads to a satisfactory colormap? I haven't tried it yet, but my first idea would be to say that I want a linear ramp in lightness (CIECAM02's "J"), and a linear ramp in hue (CIECAM02's "h"), that starts at blue and ends at yellow, and then run an optimizer to try and find a set of colorfulness values (CIECAM02's "M") that maximize some criteria, i.e.: -- need to stay within the sRGB gamut -- the points should be as close to equidistant as possible (measured in CAM02-UCS space) -- the total arc should be as long as possible (measured in CAM02-UCS space) (this forces it to use the large colorfulness values when available) -- and maybe some sort of wiggliness penalty (integral of squared third derivative or something?) to smooth it out a bit Then it just becomes an optimization problem -- given any proposed set of JMh values we can convert into sRGB to check the gamut, convert in CAM02-UCS to check the distances, etc., and compute an objective function. > My first idea > was to change the interpolating line to a circular arc passing through an > "intermediate" color, but it's not clear to me whether there is any > preferred "direction" for nudging the line into an arc. Also, most other > colormaps, such as the examples "YlGnBu" and "cubehelix" which you > mentioned, use more complicated curves than mere circular arcs (btw, kudos > for your script - it's a great way of visualising these colormaps). I don't > have enough knowledge yet to decide whether either approach is better. I've > started toying with curved interpolations in my code but this is not quite > ready to be pushed to Github yet. Anyway, if you have any suggestions I'd > love to hear them. > > I also found a few more blog posts and papers which I hadn't seen before and > which look extremely useful: > > (i) "Subtleties of color" > > http://earthobservatory.nasa.gov/blogs/elegantfigures/2013/08/05/subtleties-of-color-part-1-of-6/ > > A series of six blog posts with an excellent introduction to color theory > and the issues around choosing colormaps. Well worth a read! It also > suggests that CIE L*c*h* space (which uses the three variables lightness, > chroma (saturation) and hue), may be a better choice than CIE L*a*b*, which > I have been using so far. They're kinda the same thing -- c*h* is just the polar coordinates version of a*b*, so you can switch back and forth depending on which way of thinking about things feels more natural for a given task. Of course if you do linear interpolation in polar coordinates you get some sort of funky curve, so I guess it would make a difference that way :-). (And the Mh that I talk about above are also conceptually just a polar coordinates version of a and b -- the CIECAM02 calculations literally have intermediate values called a and b that play the same role as CIEL*a*b*'s a* and b*.) -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org |