Menu
▾
▴
gnuplot-beta
|
From: Ethan A M. <sf...@us...> - 2015-09-14 17:48:10
|
On the discussion board <https://sourceforge.net/p/gnuplot/discussion/5924> Kevin Rattigan asked if there is a gnuplot equivalent to the types of plot provided by the "beeswarm" module for R. He provided this link to examples: http://www.cbs.dtu.dk/~eklund/beeswarm/ It turns out that these plots arise naturally from a simple jitter offset applied to overlapping points in a scatter plot. Since I already had a jitter patch lying around, I modified it to apply a sequential linear jitter rather than a random jitter, and here you go: >From the ChangeLog Syntax: set jitter {overlap <yposition>} {spread <factor>} {wrap <limit>} {swarm|square} When the x coordinates of a data set are restricted to discrete values then many points may lie exactly on top of each other. Jittering introduces an offset to the coordinates of these superimposed points that spreads them into a cluster. This type of plot is called a "bee swarm" plot in R and other packages. Online copy of the new demo http://gnuplot.sourceforge.net/demo_5.1/jitter.html Please try it out and offer suggestions. Would it make sense to apply the jitter to other plot styles in addition to "with points"? Is there a logical 3D counterpart? Ethan |
|
From: Tait <gnu...@t4...> - 2015-09-15 18:57:10
|
I find the implementation in R to be confusing. There are several
"types" of jitter, which appear to confound jitter in two
dimensions. JMP also offers jitter, if you want another
implementation to look at for a constrasting approach.
Jitter means applying a "small" (definition of which is open to
debate) offset to the specified dimension of a data set. I have
implemented it before within the "... using ..." directive to plot,
and that is where I believe it makes the most sense, e.g.
plot 'data.dat' using (jitter($1)):2
As used this way, the jitter offsets don't alter the non-jittered
dimension, which the R examples appear to do.
The extension to higher-dimension plots seems straightforward. One
might jitter in x AND y, or in u and t, or jitter the size and/or
color with "ps variable" or "lc variable".
> ...
> http://www.cbs.dtu.dk/~eklund/beeswarm/
> ...
> Syntax:
> set jitter {overlap <yposition>} {spread <factor>} {wrap <limit>}
> {swarm|square}
> ...
> Would it make sense to apply the jitter to other plot styles
> in addition to "with points"?
> Is there a logical 3D counterpart?
|
|
From: Ethan A M. <sf...@us...> - 2015-09-15 19:56:18
|
On Tuesday, 15 September, 2015 18:16:53 Tait wrote: > > I find the implementation in R to be confusing. There are several > "types" of jitter, which appear to confound jitter in two > dimensions. JMP also offers jitter, if you want another > implementation to look at for a constrasting approach. I have not looked at the JMP code, but the appearance of the example http://www.jmp.com/support/help/images/students.gif looks similar to what my previous attempt at jitter code produced by replacing x = x + scale*rand(0) for all points. I found those random jitter plots to be ugly so I never pursued it. I find the beeswarm plots to be much nicer visually and they also convey more information. In particular they have the very nice property that as the number of points becomes large and the point size becomes small the result converges to a violin plot rather than to a solid rectangle. I.e. the envelope of the distribution remains informative, whereas random jitter conveys no information about the distribution once the points get dense enough to saturate the allowed width. I did not find a rigorous description of what the R package is doing, but the results I get are sufficiently similar in appearance that I guess they are doing something very like what my code does. > Jitter means applying a "small" (definition of which is open to > debate) offset to the specified dimension of a data set. I have > implemented it before within the "... using ..." directive to plot, > and that is where I believe it makes the most sense, e.g. > plot 'data.dat' using (jitter($1)):2 > As used this way, the jitter offsets don't alter the non-jittered > dimension, which the R examples appear to do. That confused me at first also, but it turns out that the visual appearance of the "bee swarm" plots is a result of applying a pure x displacement sequentially to points that have been sorted on y. The jitter is applied only to points that would otherwise overlap. Since successive points have a larger sorted y value and also receive a larger +/- x displacement, you get the upward-sweeping lines of points that are so distinctive. The opposite is true for the "square" option. Although at first glance it looks like this results from applying no displacement on y, in fact the only reason the displaced points line up in horizontal layers is that a y displacement is added to remove the incremental y from successive overlapping points. If the y values are discrete rather than continuous and their separation is greater than the overlap criterion, then both the swarm and square modes produce the same result. > The extension to higher-dimension plots seems straightforward. One > might jitter in x AND y, or in u and t, Sure. I was just wondering if anyone had an example of real-world data that would benefit from this form of display. The only thing I could think of was displaying the energy spread of individual photons striking a 2D pixel array, but I'm not sure that such an energy-sensitive pixel device actually exists. >or jitter the size and/or color with "ps variable" or "lc variable". When would that ever be useful? Spatial jitter removes overlap so that you can see how many points there are. What would perturbing the color or size accomplish? Ethan > > > ... > > http://www.cbs.dtu.dk/~eklund/beeswarm/ > > ... > > Syntax: > > set jitter {overlap <yposition>} {spread <factor>} {wrap <limit>} > > {swarm|square} > > ... > > Would it make sense to apply the jitter to other plot styles > > in addition to "with points"? > > Is there a logical 3D counterpart? |
|
From: Tait <gnu...@t4...> - 2015-09-15 21:32:28
|
> http://www.jmp.com/support/help/images/students.gif > ... > I found those random jitter plots to be ugly so I never pursued it. > I find the beeswarm plots to be much nicer visually and they also > convey more information. ... Random jitter may not be very pretty, but within the bounds of what it's meant to do, it doesn't distort the data. In the limit of enough data points, a random jitter would also approach a 2-D boxplot or "violin" plot. Pretty graphs are nice, but the plotting tool should not encourage them at the expense of accuracy. > > Jitter means applying a "small" (definition of which is open to > > debate) offset to the specified dimension of a data set. I have > > implemented it before within the "... using ..." directive to plot, > > and that is where I believe it makes the most sense, e.g. > > plot 'data.dat' using (jitter($1)):2 > > As used this way, the jitter offsets don't alter the non-jittered > > dimension, which the R examples appear to do. > > That confused me at first also, but it turns out that the visual > appearance of the "bee swarm" plots is a result of applying a pure > x displacement sequentially to points that have been sorted on y. > The jitter is applied only to points that would otherwise overlap. > Since successive points have a larger sorted y value > and also receive a larger +/- x displacement, you get the > upward-sweeping lines of points that are so distinctive. This does not appear to be what's happening in the "method=swarm" plot from the R package, at least. There are points where the greater x-displacement point's y-coordinate clearly overlaps with another point who's x-displacement puts it on or nearer to the central x-axis. There seems to be some chunking function applied to the points before they're sorted, and this distorts the data in a way similar to how histogram binning distorts data. (The solution to that distortion is to use kernel density plots instead, and for the same reason, I'd take a density contour plot* or a heat map any day in preference to a beeswarm plot.) * gnuplot doesn't have very good support for density plots or any sort of plot that requires an aggregate view of the input data. Some of this is done via the "smooth" options, but it's limited. > The opposite is true for the "square" option. Although at first > glance it looks like this results from applying no displacement on y, > in fact the only reason the displaced points line up in horizontal > layers is that a y displacement is added to remove the incremental y > from successive overlapping points. The R "square" and "hex" methods are probably the clearest examples of what not to do, namely grid the data and draw points onto the grid. As you note, this does produce pretty-looking plots and I'm not averse to having such a feature in gnuplot, but it should be named in accordance to what it is -- resampling the data (ala dgrid3d), not "jitter". > > The extension to higher-dimension plots seems straightforward. One > > might jitter in x AND y, or in u and t, > > Sure. I was just wondering if anyone had an example of real-world > data that would benefit from this form of display. ... Anything that you do in 1D, someone might do in 2D. So the first example that comes to mind is any sort of location-based incidence data. Rainfall is an easily-understood example. Each collection station has its own x and y location, and reports the amount of water collected. A map of collection stations with "at least N" rainfall could be represented as points on an xy plane, and if the density of such collection stations were high enough for the desired zoom level, it might be represented using jitter in both x and y. A highly directional receiver antenna array might sweep around a 360-degree circle, saving a record of some relevant signal received, each record having "longitude" of the antenna array's angular position as it rotates, and a "lattitude" corresponding to one of the N lobes in the array. A plot of record count vs. direction would tend to draw multiple points on top of each other, and this jitter in both lattitude and longitude might be desirable to better show number of records. I'm the wrong person to defend use of jitter plots, because in these examples (and any other I can imagine), there's at least one other way to plot the data that is more clear and more accurate And were I the presenter, I'd be using those other plots, not jitter. Even supposing I couldn't find a better plot type, I'd still prefer to plot the points -- without jitter -- using partial transparency, so that as the points/lines stack the color becomes more intense, thereby showing density through color saturation without needing to introduce artificial jitter. And indeed, I frequently use the partial transparency approach when doing exploratory scatter plots. > >or jitter the size and/or color with "ps variable" or "lc variable". > > When would that ever be useful? > Spatial jitter removes overlap so that you can see how many > points there are. What would perturbing the color or size accomplish? I am not sure jitter in color would really be useful, since the peturbation would almost certainly be too small for a person to usefully distinguish. I just threw it in for the sake of completeness. But jitter in point size would definitely be useful. The variable pointsize demo that's already online (using world.cor) is case in point. If the data were real and discrete, rather than "(5.*rand(0))", it would be necessary to jitter the pointsize to distinguish one point from another. |
|
From: Ethan A M. <sf...@us...> - 2015-09-15 22:56:10
|
On Tuesday, 15 September, 2015 21:32:17 Tait wrote: > > http://www.jmp.com/support/help/images/students.gif > > ... > > I found those random jitter plots to be ugly so I never pursued it. > > I find the beeswarm plots to be much nicer visually and they also > > convey more information. ... > > Random jitter may not be very pretty, but within the bounds of what > it's meant to do, it doesn't distort the data. But it does distort! It gives the visual impression of a uniform scatter at every y value, which hides the very feature that it is intended to convey. What you want (or at least what I would want :-) is to leave singleton points unperturbed but separate overlapped points so you can see how many there are. That is what the beeswarm plots do, but not what random jitter does. > In the limit of > enough data points, a random jitter would also approach a 2-D > boxplot or "violin" plot. That is not correct. The distribution of points at each y value will approach the randomization function, not the PDF envelope of the data. And not a standard box plot either. Here is a direct comparison of the output using the swarm jitter and a random (normal distribution) displacement about x. http://skuld.bmsc.washington.edu/~merritt/gnuplot/violin.html The random jitter is better than I might have predicted, but is strictly worse than a violin plot IMHO. A uniform random jitter would convey almost no information. Note that a standard box plot is even worse. > Pretty graphs are nice, but the plotting > tool should not encourage them at the expense of accuracy. Yeah, but in my experience random jitter is not a good way to convey the desired information. I played with it a fair amount a couple of years ago as a possible gnuplot option for version 5 before giving up. At that same time I tried to work out how to implement violin plots, and failed to find a good solution. For both of these reasons I was very pleased when Kevin Rattigan's question made me aware of beeswarm plots. Ethan |
|
From: <pl...@pi...> - 2015-09-16 07:06:18
|
Hi, http://skuld.bmsc.washington.edu/~merritt/gnuplot/violin.html swarm jitter with a large number of small points approximates a violin plot the plot of dataset B has all key characteristics of a mature amanite mushroom in cross-section. Maybe it should be called a mushroom plot instead ;) Peter. |
|
From: Tait <gnu...@t4...> - 2015-09-16 10:29:29
|
> Here is a direct comparison of the output using the swarm jitter > and a random (normal distribution) displacement about x. > > http://skuld.bmsc.washington.edu/~merritt/gnuplot/violin.html At that zoom, the swarm jitter looks reasonable. There are two kinds of "random" jitter, and I was not clear even with myself about which one I meant where. A random offset indiscriminately applied to every point, as you've shown in your example above, is still useful, but I agree not as helpful as the swarm plot. But going back to the zoom levels in http://www.cbs.dtu.dk/~eklund/beeswarm/, what I'd want from a jitter algorithm is that each point's offset from its original coordinate is strictly the minimum offset necessary for that point to not overlap its neighbors. That would eliminate the strings of points getting increasingly large offsets just because the immediately-preceeding point overlaps, when there's an open spot closer to the central axis. I expect the outcome might look closer to the "priority=random" plot near the bottom of the beeswarm page. |
|
From: sfeam <sf...@us...> - 2015-09-16 15:56:33
|
On Wednesday, 16 September 2015 10:29:20 AM Tait wrote: > > Here is a direct comparison of the output using the swarm jitter > > and a random (normal distribution) displacement about x. > > > > http://skuld.bmsc.washington.edu/~merritt/gnuplot/violin.html > > At that zoom, the swarm jitter looks reasonable. There are two kinds > of "random" jitter, and I was not clear even with myself about which > one I meant where. A random offset indiscriminately applied to every > point, as you've shown in your example above, is still useful, but I > agree not as helpful as the swarm plot. I have updated that example to show uniform and Gaussian random jitter, and to show a kernel density plot for the same data. > > But going back to the zoom levels in > http://www.cbs.dtu.dk/~eklund/beeswarm/, what I'd want from a jitter > algorithm is that each point's offset from its original coordinate > is strictly the minimum offset necessary for that point to not > overlap its neighbors. That would eliminate the strings of points > getting increasingly large offsets just because the > immediately-preceeding point overlaps, when there's an open spot > closer to the central axis. It may be that the R code suffers from the symptom you describe, where points are drawn unnecessarily far from the center because of a nearest-neighbor effect. I have not found a rigorous description of how it works. The criterion I used for the gnuplot code is different. It is not "the immediately-preceding point overlaps". Rather it is 1) draw point N on the center line 2) count off how many of the subsequent points would overlap N if not jittered. Call this number j. 3) point N+1 is offset by 1 width, point N+2 is offset by -1 width point N+3 is offset by 2 widths, ... up to point N+j 4) N = N + J + 1 This new point is guaranteed not to overlap if drawn on the center line, so return to (1). A complicating factor is that the exact diameter of the point symbol in terms of plot x coordinates is dependent on the point symbol used, the terminal type, and the current zoom or plot scale. That is the motivation for the "spread" multiplier, which allows you to decrease the spacing if the program overestimates the point width or if the point shape permits closer spacing. E.g. if the delta-y value is greater than zero, an x-offset less than the circle radius can be sufficient to remove overlapping circles, but not squares. Ethan |
|
From: <pl...@pi...> - 2015-09-17 22:04:13
|
On 16/09/15 17:53, sfeam wrote: > It may be that the R code suffers from the symptom you describe, > where points are drawn unnecessarily far from the center because > of a nearest-neighbor effect. I have not found a rigorous > description of how it works. Which is fairly typical of a lot of things R does. :( Probably why it's more of a hit with business and econometrics that science. Peter. |
