PyX

Hi John,

Replying to myself:
> The new palette will look like
> 
> class palette(attr.changeattr):
>     """color palettes
> 
>     A color palette comprises multiple discrete, ordered collections of colors"""
> 
>     def __init__(self, colorlists):
>         """ initialize the color palette with colorlists
> 
>         colorlists is a list of list of colors, with len(colorlists[i]) = i+1"""
>         self.colorlists = colorlists
> 
>     def select(self, index, total):
>         return self.colorlists[total-1][index]

I somehow misunderstood your use-case (as André immediately pointed out
to me). So in fact the new interface for palettes will be even simpler:

class palette(attr.changelist):
    """color palettes

    A color palette comprises multiple discrete, ordered collections of colors"""
    pass
    

palette.threegreys = palette([grey.black, grey.grey(0.5), grey.white])

        Jörg

--- Joerg Lehmann <joergl@...> wrote:

> One question would be, whether we provide a fallback handling in
> palette.select of the case with total > len(colorlists). 

Seems to me we actually want two fallbacks:

1) The one you state above; I'd advocate possibly doing a gradient
between the first and last color of the most detailed palette entry
(so if a palette had discrete colors for 3-10 colors, use p[10][0]
and p[10][9]).

2) What happens if you don't define enough colors? For instance
most of the colorbrewer schemes only start defining at 3 colors
(there's no 1 or 2 color schemes). In that case if I needed only 2 
colors I'd want to take the first two elements of the least detailed
palette entry.

The above brings up the question of "should the interface be
different because we might not have 1- or 2-color entries in
the palette?". A simple index wouldn't be enough in this case;
instead a hash would be better, because you wouldn't have
an entry for [0] and [1] if you only had 3-10 color entries. I
can write my palettes by manually putting in 1- and 2-color
entries for now though.

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Hi John,

On 20.06.06, John Owens wrote:
> --- Joerg Lehmann <joergl@...> wrote:
> 
> > One question would be, whether we provide a fallback handling in
> > palette.select of the case with total > len(colorlists). 
> 
> Seems to me we actually want two fallbacks:

Note that the first implementation I posted was strongly disliked by
André, i.e., he doesn't want to have a palette with a non-fixed number
of colors -- and in the end he has convinced me. That's why the
implementation I checked into SVN now is just the one of my second mail,
i.e., palette is a simple subclass of attr.changelist (see also my other
previous reply to myself).

> 1) The one you state above; I'd advocate possibly doing a gradient
> between the first and last color of the most detailed palette entry
> (so if a palette had discrete colors for 3-10 colors, use p[10][0]
> and p[10][9]).
> 
> 2) What happens if you don't define enough colors? For instance
> most of the colorbrewer schemes only start defining at 3 colors
> (there's no 1 or 2 color schemes). In that case if I needed only 2 
> colors I'd want to take the first two elements of the least detailed
> palette entry.

In some way your questions show that the concept becomes indeed a bit
shaky.

I understand from the other mail you sent me that the color schemes
are not very different with varying number of total colors, so the
question is whether one really needs to support that instead of
just defining the most detailed palette.

        Jörg

--- Joerg Lehmann <joerg@...> wrote:
> I understand from the other mail you sent me that the color schemes
> are not very different with varying number of total colors, so the
> question is whether one really needs to support that instead of
> just defining the most detailed palette.

Unequivocally they are different enough that we would want to support
a variable number of colors separately within a palette. Compare, say,
the spectral scheme's 3-color and 6-color variants:

        [ rgb(0.988235, 0.552941, 0.349020) ,  rgb(1.000000, 1.000000,
0.749020) ,  rgb(0.600000, 0.835294, 0.580392) ,  ],

        [ rgb(0.835294, 0.243137, 0.309804) ,  rgb(0.988235, 0.552941,
0.349020) ,  rgb(0.996078, 0.878431, 0.545098) ,  rgb(0.901961, 0.960784,
0.596078) ,  rgb(0.600000, 0.835294, 0.580392) ,  rgb(0.196078, 0.533333,
0.741176) ,  ],

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

--- Joerg Lehmann <joerg@...> wrote:

> > 1) The one you state above; I'd advocate possibly doing a gradient
> > between the first and last color of the most detailed palette entry
> > (so if a palette had discrete colors for 3-10 colors, use p[10][0]
> > and p[10][9]).
> > 
> > 2) What happens if you don't define enough colors? For instance
> > most of the colorbrewer schemes only start defining at 3 colors
> > (there's no 1 or 2 color schemes). In that case if I needed only 2 
> > colors I'd want to take the first two elements of the least detailed
> > palette entry.
> 
> In some way your questions show that the concept becomes indeed a bit
> shaky.

Well, to be fair, with any discrete scheme this would come up. The 
first question is "is it useful to have a discrete scheme indexed by 
total number of colors and by color number?" No surprise that I think
the answer is "yes"; I think Dr. Brewer's research shows that color
selection in general, and her schemes in particular, are
generally useful for understanding. Then if you want to support such
a discrete scheme, you have to handle the cases when you haven't
defined colors. The question then is "do you leave that up to the
programmer" or "do you handle it in some reasonable way within the
programming system", and I'd argue for the latter in the way I described
above. 

I defer to you guys on what the interface should look like, but I'm a 
strong advocate of having an interface for discrete schemes.

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

--- Joerg Lehmann <joerg@...> wrote:
> In some way your questions show that the concept becomes indeed a bit
> shaky.

And if you decide that a discretepalette is simply too distasteful to
support :), perhaps we could just figure out how to support a
"functionpalette" in such a way that I could write my own functions,
and store my own dictionary of colors within it, as an end-user. 
I'd prefer it be natively supported, but allowing semi-arbitrary
functions and user-specified data structures for a palette would seem
like a useful addition.

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Hello John,

On 20.06.06, John Owens wrote:
> --- Joerg Lehmann <joerg@...> wrote:
> > I understand from the other mail you sent me that the color schemes
> > are not very different with varying number of total colors, so the
> > question is whether one really needs to support that instead of
> > just defining the most detailed palette.
> 
> Unequivocally they are different enough that we would want to support
> a variable number of colors separately within a palette. Compare, say,
> the spectral scheme's 3-color and 6-color variants:
> 
>         [ rgb(0.988235, 0.552941, 0.349020) ,  rgb(1.000000, 1.000000,
> 0.749020) ,  rgb(0.600000, 0.835294, 0.580392) ,  ],
> 
>         [ rgb(0.835294, 0.243137, 0.309804) ,  rgb(0.988235, 0.552941,
> 0.349020) ,  rgb(0.996078, 0.878431, 0.545098) ,  rgb(0.901961, 0.960784,
> 0.596078) ,  rgb(0.600000, 0.835294, 0.580392) ,  rgb(0.196078, 0.533333,
> 0.741176) ,  ],

How many variants of a certain scheme do typically exist? If there are
not too many, one could simply make the maximum number of colors
explicit:

   palette.spectral_3 = palette([c1, c2, c3])
   palette.spectral_6 = palette([c1, c2, c3, c4, c5, c6])
   ...

This would also solve the questions you raised in your previous mail by
leaving it up to the user what he wants.

Maybe one could also introduce the palette code I posted initially in
addition (under a different name) to the changelist variant. André
already told me that he doesn't like it (because he wants a fixed number
of colors [1]), but thinking about it again, I realized that in fact the
number of colors would also be fixed in the more complicated scheme (of
course ;-))

        Jörg

[1] This would allow us for instance to use an indexed color space in
PostScript and PDF.

On 20.06.06, John Owens wrote:
> --- Joerg Lehmann <joerg@...> wrote:
> 
> > > 1) The one you state above; I'd advocate possibly doing a gradient
> > > between the first and last color of the most detailed palette entry
> > > (so if a palette had discrete colors for 3-10 colors, use p[10][0]
> > > and p[10][9]).
> > > 
> > > 2) What happens if you don't define enough colors? For instance
> > > most of the colorbrewer schemes only start defining at 3 colors
> > > (there's no 1 or 2 color schemes). In that case if I needed only 2 
> > > colors I'd want to take the first two elements of the least detailed
> > > palette entry.
> > 
> > In some way your questions show that the concept becomes indeed a bit
> > shaky.
> 
> Well, to be fair, with any discrete scheme this would come up. The 
> first question is "is it useful to have a discrete scheme indexed by 
> total number of colors and by color number?" No surprise that I think
> the answer is "yes"; 

When posting my initial scheme, this was also obvious to me. On the
other hand, it's not what a color palette usually means: an ordered
collection of a finite number of colors. But we could reconcile
these two pictures of a palette as described in my other mail, but
considering the trivial mapping
  [[c11], [c21, c22], [c31, c32, c33], ...] ->
  [c11, c21, c22, c31, c32, c33, ...]

> I think Dr. Brewer's research shows that color
> selection in general, and her schemes in particular, are
> generally useful for understanding. Then if you want to support such
> a discrete scheme, you have to handle the cases when you haven't
> defined colors. The question then is "do you leave that up to the
> programmer" or "do you handle it in some reasonable way within the
> programming system", and I'd argue for the latter in the way I described
> above. 

This is something which has to be discussed (try import this and look
at line number four, counting empty lines). The needed input would
be:
- how many variants does a given scheme typically have?
- can the fall-back behaviour in the case of a variant not
  exisiting for the given total number of colors be well defined, i.e.
  how implicit would the choice be?

        Jörg

--- Joerg Lehmann <joergl@...> wrote:

> How many variants of a certain scheme do typically exist? If there are
> not too many, one could simply make the maximum number of colors
> explicit:
> 
>    palette.spectral_3 = palette([c1, c2, c3])
>    palette.spectral_6 = palette([c1, c2, c3, c4, c5, c6])
>    ...

There's one variant for each total-number-of-colors-in-the-graph.
The colorbrewer schemes themselves typically range from 3 to about 10.
What you propose above is just fine, but you would have to allow the capability
to select palette.spectral_n using the n_indices argument of the select call.
Which is fine with me, as long as that support is there; it really isn't any 
big deal to define one giant list-of-lists-of-colors or instead assemble
the lists of colors separately with separate names.

I'm appending the "source" for the spectral color scheme. Note for this
one, there's 15 colors total, assembled into schemes for anywhere from
3 to 11 colors inclusive. Probably what you want to look at is the 6th column
which is a letter that indicates an individual color.

JDO

Spectral,3,div,2,1,E,252,141,89,Diverging
,,,,2,H,255,255,191,
,,,,3,K,153,213,148,
Spectral,4,div,2.5,1,C,215,25,28,
,,,,2,F,253,174,97,
,,,,3,J,171,221,164,
,,,,4,M,43,131,186,
Spectral,5,div,3,1,C,215,25,28,
,,,,2,F,253,174,97,
,,,,3,H,255,255,191,
,,,,4,J,171,221,164,
,,,,5,M,43,131,186,
Spectral,6,div,3.5,1,B,213,62,79,
,,,,2,E,252,141,89,
,,,,3,G,254,224,139,
,,,,4,I,230,245,152,
,,,,5,K,153,213,148,
,,,,6,N,50,136,189,
Spectral,7,div,4,1,B,213,62,79,
,,,,2,E,252,141,89,
,,,,3,G,254,224,139,
,,,,4,H,255,255,191,
,,,,5,I,230,245,152,
,,,,6,K,153,213,148,
,,,,7,N,50,136,189,
Spectral,8,div,4.5,1,B,213,62,79,
,,,,2,D,244,109,67,
,,,,3,F,253,174,97,
,,,,4,G,254,224,139,
,,,,5,I,230,245,152,
,,,,6,J,171,221,164,
,,,,7,L,102,194,165,
,,,,8,N,50,136,189,
Spectral,9,div,5,1,B,213,62,79,
,,,,2,D,244,109,67,
,,,,3,F,253,174,97,
,,,,4,G,254,224,139,
,,,,5,H,255,255,191,
,,,,6,I,230,245,152,
,,,,7,J,171,221,164,
,,,,8,L,102,194,165,
,,,,9,N,50,136,189,
Spectral,10,div,5.5,1,A,158,1,66,
,,,,2,B,213,62,79,
,,,,3,D,244,109,67,
,,,,4,F,253,174,97,
,,,,5,G,254,224,139,
,,,,6,I,230,245,152,
,,,,7,J,171,221,164,
,,,,8,L,102,194,165,
,,,,9,N,50,136,189,
,,,,10,O,94,79,162,
Spectral,11,div,6,1,A,158,1,66,
,,,,2,B,213,62,79,
,,,,3,D,244,109,67,
,,,,4,F,253,174,97,
,,,,5,G,254,224,139,
,,,,6,H,255,255,191,
,,,,7,I,230,245,152,
,,,,8,J,171,221,164,
,,,,9,L,102,194,165,
,,,,10,N,50,136,189,
,,,,11,O,94,79,162,


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

--- Joerg Lehmann <joergl@...> wrote:

> When posting my initial scheme, this was also obvious to me. On the
> other hand, it's not what a color palette usually means: an ordered
> collection of a finite number of colors. But we could reconcile
> these two pictures of a palette as described in my other mail, but
> considering the trivial mapping
>   [[c11], [c21, c22], [c31, c32, c33], ...] ->
>   [c11, c21, c22, c31, c32, c33, ...]

I don't really see why the second one is better than the first. The
first has the advantage of being able to identify several different
palettes, one for each number-of-colors, while the second one it's
much harder to do that.

> This is something which has to be discussed (try import this and look
> at line number four, counting empty lines). The needed input would
> be:
> - how many variants does a given scheme typically have?
> - can the fall-back behaviour in the case of a variant not
>   exisiting for the given total number of colors be well defined, i.e.
>   how implicit would the choice be?

I apologize, I'm not sure what you mean by "import this and look at line
number four". To answer your questions:

- A named scheme (in colorbrewer) usually ranges from 3 to 10 
total-number-of-colors (I appended one in my last message).
- Fallback could be either done by the user or by PyX. It'd be nice
if PyX had a default fallback (and I would advocate if too few colors,
take the first n colors of the variant with the smallest number of
colors; if too many colors, do a gradient between the first and last
colors of the variant with the most colors), and that the user could
override the default if he/she chooses.

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Hi John,

On 21.06.06, John Owens wrote:
> --- Joerg Lehmann <joergl@...> wrote:
> 
> > When posting my initial scheme, this was also obvious to me. On the
> > other hand, it's not what a color palette usually means: an ordered
> > collection of a finite number of colors. But we could reconcile
> > these two pictures of a palette as described in my other mail, but
> > considering the trivial mapping
> >   [[c11], [c21, c22], [c31, c32, c33], ...] ->
> >   [c11, c21, c22, c31, c32, c33, ...]
> 
> I don't really see why the second one is better than the first. The
> first has the advantage of being able to identify several different
> palettes, one for each number-of-colors, while the second one it's
> much harder to do that.

This was just a bridge (for André) to the idea that a palette is just an
ordered, finite list of colors.

> > This is something which has to be discussed (try import this and look
> > at line number four, counting empty lines). The needed input would
> > be:
> > - how many variants does a given scheme typically have?
> > - can the fall-back behaviour in the case of a variant not
> >   exisiting for the given total number of colors be well defined, i.e.
> >   how implicit would the choice be?
> 
> I apologize, I'm not sure what you mean by "import this and look at line
> number four".

It was a very implicit way of saying that explicit is better than 
implicit. But it's definitely worth trying "import this" in an interactice
Python shell...

> -To answer your questions:
> - A named scheme (in colorbrewer) usually ranges from 3 to 10 
> total-number-of-colors (I appended one in my last message).

Ok.

> - Fallback could be either done by the user or by PyX. It'd be nice
> if PyX had a default fallback (and I would advocate if too few colors,
> take the first n colors of the variant with the smallest number of
> colors; if too many colors, do a gradient between the first and last
> colors of the variant with the most colors), and that the user could
> override the default if he/she chooses.

But this gradient is not really readily defined/implemented. On the
other hand, if we require the user to specify explicitly from which palette 
with which total number of colors PyX should draw colors, we could
simply cycle around and start with the first color again. 

Anyway, I think we have to wait for André's opinion on this whole issue.
He gave my initial implementation a -1, so even if I would vote with +1,
we would not reach a positive result ;-) I could imagine some
compromise, though, like adding my initial proposal as a third class
which is somewhere in between a gradient and a palette.

        Jörg

--- Joerg Lehmann <joergl@...> wrote:

> This was just a bridge (for André) to the idea that a palette is just an
> ordered, finite list of colors.

Eeesh. I gotta say it's not a flat list of colors, it's a separate list
of colors for each number-of-colors. That doesn't seem to be a 
very good abstraction.

> > - Fallback could be either done by the user or by PyX. It'd be nice
> > if PyX had a default fallback (and I would advocate if too few colors,
> > take the first n colors of the variant with the smallest number of
> > colors; if too many colors, do a gradient between the first and last
> > colors of the variant with the most colors), and that the user could
> > override the default if he/she chooses.
> 
> But this gradient is not really readily defined/implemented. On the
> other hand, if we require the user to specify explicitly from which palette 
> with which total number of colors PyX should draw colors, we could
> simply cycle around and start with the first color again. 
> 
> Anyway, I think we have to wait for André's opinion on this whole issue.
> He gave my initial implementation a -1, so even if I would vote with +1,
> we would not reach a positive result ;-) I could imagine some
> compromise, though, like adding my initial proposal as a third class
> which is somewhere in between a gradient and a palette.

In the meantime, before we maybe settle on a final useful abstraction
that works for everyone, can we try something that leaves the
implementation to the programmer? What would be needed is a 
class that takes a data structure and a function as an argument,
both supplied by the programmer. The function should take "index"
and "n_indices" as an argument and access the data structure. It's
the same interface as the current gradient class, except the data
structure is hardwired. It's specifically because I didn't have access
to such a function/data structure pairing that I brought this whole
issue up in the first place. Perhaps the data structure should be 
limited to a dictionary to start with.

This might be nice for the next release, and then people would write
their own instantiations and we could look at them and see what makes
sense to put inside PyX as a result.

JDO

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Hi,

On 21.06.06, Joerg Lehmann wrote:
> On 20.06.06, John Owens wrote:
> > --- Joerg Lehmann <joerg@...> wrote:
> > 
> > > > 1) The one you state above; I'd advocate possibly doing a gradient
> > > > between the first and last color of the most detailed palette entry
> > > > (so if a palette had discrete colors for 3-10 colors, use p[10][0]
> > > > and p[10][9]).
> > > > 
> > > > 2) What happens if you don't define enough colors? For instance
> > > > most of the colorbrewer schemes only start defining at 3 colors
> > > > (there's no 1 or 2 color schemes). In that case if I needed only 2 
> > > > colors I'd want to take the first two elements of the least detailed
> > > > palette entry.
> > > 
> > > In some way your questions show that the concept becomes indeed a bit
> > > shaky.
> > 
> > Well, to be fair, with any discrete scheme this would come up. The 
> > first question is "is it useful to have a discrete scheme indexed by 
> > total number of colors and by color number?" No surprise that I think
> > the answer is "yes"; 
> 
> When posting my initial scheme, this was also obvious to me. On the
> other hand, it's not what a color palette usually means: an ordered
> collection of a finite number of colors. But we could reconcile
> these two pictures of a palette as described in my other mail, but
> considering the trivial mapping
>   [[c11], [c21, c22], [c31, c32, c33], ...] ->
>   [c11, c21, c22, c31, c32, c33, ...]

I think this is a very bad idea. The color c11 and c21 should not be
used together. They don't belong into the same palette.


André

-- 
by  _ _      _    Dr. André Wobst
   / \ \    / )   wobsta@..., http://www.wobsta.de/
  / _ \ \/\/ /    PyX - High quality PostScript and PDF figures
 (_/ \_)_/\_/     with Python & TeX: visit http://pyx.sourceforge.net/

Hi,

On 21.06.06, John Owens wrote:
> - A named scheme (in colorbrewer) usually ranges from 3 to 10 
> total-number-of-colors (I appended one in my last message).
> - Fallback could be either done by the user or by PyX. It'd be nice
> if PyX had a default fallback (and I would advocate if too few colors,
> take the first n colors of the variant with the smallest number of
> colors; if too many colors, do a gradient between the first and last
> colors of the variant with the most colors), and that the user could
> override the default if he/she chooses.

I don't think PyX should do anything clever. A palette is a very
explicit thing. It contains a number of colors and ... yes, that's it.
Nothing more. NOTHING. I really very much advocate such a very simple
point of view.


André

-- 
by  _ _      _    Dr. André Wobst
   / \ \    / )   wobsta@..., http://www.wobsta.de/
  / _ \ \/\/ /    PyX - High quality PostScript and PDF figures
 (_/ \_)_/\_/     with Python & TeX: visit http://pyx.sourceforge.net/

Hi,

On 22.06.06, John Owens wrote:
> In the meantime, before we maybe settle on a final useful abstraction
> that works for everyone, can we try something that leaves the
> implementation to the programmer? What would be needed is a 
> class that takes a data structure and a function as an argument,
> both supplied by the programmer. The function should take "index"
> and "n_indices" as an argument and access the data structure. It's
> the same interface as the current gradient class, except the data
> structure is hardwired. It's specifically because I didn't have access
> to such a function/data structure pairing that I brought this whole
> issue up in the first place. Perhaps the data structure should be 
> limited to a dictionary to start with.

I'm not convinced at all that this is a good idea. A changeable
attribute is something quite complicated to the regular PyX user
already. Sure, there is this funny feature of a gradient (I've already
switched my mind to the new notation -- which I like very much and
which I'm thankful we've already reached this). But a gradient is
something special. A silly and simple changeable attribute should be
flat. Try

    python -c 'import this'|sed -e '7!d'

Really. We should not complicate that. And I would very much dislike a
palette to return some very stange different colors depending on
certain strange settings.

Overall, if we really want something like you suggest (and sure, I'm
convinced it's a usefull feature), we should make something like a
changeable attribute, which delegates the selection to some other
changeable attribute. Something along the lines:

    class palette(attr.changelist):
        pass

    palette.x3 = palette([c31, c32, c33])
    palette.x4 = palette([c41, c42, c43, c44])
    palette.x5 = palette([c51, c52, c53, c54, c55])


    class discretesomething(attr.changeattr):

        __init__(self, palettes):
            self.palettes = palettes

        select(self, index, total):
            return self.palettes[total].select(index, total)


    discretesomething.x = discretesomething({3: palette.x3,
                                             4: palette.x4,
                                             5: palette.x5})


Yeah. That's certainly possible, but it's a feature build for
convenience *on* *top* of the basic infrastructure. And the
"discretesomething" is a strange object. It's a changeable attribute,
sure. But it's not a palette, since for example there are colors which
can be selected, which should *not* be used together. And it's not a
gradient, since the color is not a function of a single parameter.

So overall I don't see any problem. We seems to have learned that we
wont something like this. The discretesomething should be something
general and should live in attr.py ... we only need a name for that.
And then we could create instances for the colorbrewer.org palettes,
except for the problem, that we're not allowed to do so.


André

-- 
by  _ _      _    Dr. André Wobst
   / \ \    / )   wobsta@..., http://www.wobsta.de/
  / _ \ \/\/ /    PyX - High quality PostScript and PDF figures
 (_/ \_)_/\_/     with Python & TeX: visit http://pyx.sourceforge.net/