|
From: Philipp K. J. <ja...@ie...> - 2015-01-27 03:28:25
|
The new plot style "with parallelaxes" is cute. Unfortunately, it also dumps core - whenever there are more than 7 columns to plot. 1) This limitation is not mentioned in the doc. (As far as I can see.) 2) Dumping core unconditionally is not a good user experience. It would be better to detect and reject the illegal input and warn the user. 3) The limitation to 7 cols (which, I suspect, depends on limitations in the way "using" is parsed) puts the entire plot style into question. The whole point of parallelaxes plots (to the degree they HAVE a point) is to deal with high-dim data, easily exceeding 7. I like this feature, in principle, but I am afraid the limitation to 7 columns really gets in the way. There is one other thing: for practical purposes, the ability to highlight a single record (or group of records) in a parallel axis plot is really, really important (obviously the best way is with the mouse). I tried to plot only a selected record via "every", but with a strange results: when selecting a single record (ev 170::170::) I get NO graph - just an empty canvas. (No warning either.) Best, Ph. |
|
From: sfeam <sf...@us...> - 2015-01-27 04:40:11
|
On Monday, 26 January 2015 07:28:19 PM Philipp K. Janert wrote: > > The new plot style "with parallelaxes" > is cute. Unfortunately, it also dumps > core - whenever there are more than 7 > columns to plot. The maximum is set at compile time, but yes the default is 7. > 1) This limitation is not mentioned in > the doc. (As far as I can see.) I thought it was, but now I can't find it either. That is an oversight. > 2) Dumping core unconditionally is not > a good user experience. It would be > better to detect and reject the illegal > input and warn the user. Indeed. > 3) The limitation to 7 cols (which, I > suspect, depends on limitations in the > way "using" is parsed) puts the entire > plot style into question. The whole > point of parallelaxes plots (to the > degree they HAVE a point) is to deal > with high-dim data, easily exceeding 7. If you have such a use-case, go ahead and file a feature request. It should not be too hard to make the axis structures dynamic rather than static. The original request for this feature provided examples from the literature but none of them had even as many as 7 axes in a single plot. So that seemed adequate as a default. Failing to check for more columns of input than expected is purely a bug. Ethan > I like this feature, in principle, but > I am afraid the limitation to 7 columns > really gets in the way. > > There is one other thing: for practical > purposes, the ability to highlight a > single record (or group of records) in > a parallel axis plot is really, really > important (obviously the best way is > with the mouse). I tried to plot only > a selected record via "every", but with > a strange results: when selecting a single > record (ev 170::170::) I get NO graph - > just an empty canvas. (No warning either.) > > Best, > > Ph. > > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming. The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > gnuplot-beta mailing list > gnu...@li... > Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta |
|
From: Philipp K. J. <ja...@ie...> - 2015-01-27 04:55:19
|
[snip] > > If you have such a use-case, go ahead and file a > feature request. It should not be too hard to make > the axis structures dynamic rather than static. > The original request for this feature provided examples > from the literature but none of them had even as many > as 7 axes in a single plot. So that seemed > adequate as a default. Failing to check for more > columns of input than expected is purely a bug. Yes, it should definitely be dynamic - and the question is whether there is a better way of doing it than having to enter all cols explicitly. Regarding "references": my book (1st ed) contains an example with 9 columns... Unwin's book "Graphics of Large Datasets" shows one w/ 20 cols (p153). It's a moot point, anyway, because an arbitrary limitation of cols is counter to the spirit of this method. (For practical reasons one may want to limit it, but at a high number - 150 or so.) [snip] > > > > There is one other thing: for practical > > purposes, the ability to highlight a > > single record (or group of records) in > > a parallel axis plot is really, really > > important (obviously the best way is > > with the mouse). I tried to plot only > > a selected record via "every", but with > > a strange results: when selecting a single > > record (ev 170::170::) I get NO graph - > > just an empty canvas. (No warning either.) Did you see this, too? Both the need to highlight records, and the odd behavior with "every"? |
|
From: Daniel J S. <dan...@ie...> - 2015-01-27 09:06:30
|
On 01/26/2015 10:38 PM, sfeam wrote: > On Monday, 26 January 2015 07:28:19 PM Philipp K. Janert wrote: >> >> The new plot style "with parallelaxes" >> is cute. Unfortunately, it also dumps >> core - whenever there are more than 7 >> columns to plot. > > The maximum is set at compile time, but > yes the default is 7. > >> 1) This limitation is not mentioned in >> the doc. (As far as I can see.) > > I thought it was, but now I can't find it either. > That is an oversight. > >> 2) Dumping core unconditionally is not >> a good user experience. It would be >> better to detect and reject the illegal >> input and warn the user. > > Indeed. > >> 3) The limitation to 7 cols (which, I >> suspect, depends on limitations in the >> way "using" is parsed) puts the entire >> plot style into question. The whole >> point of parallelaxes plots (to the >> degree they HAVE a point) is to deal >> with high-dim data, easily exceeding 7. > > If you have such a use-case, go ahead and file a > feature request. It should not be too hard to make > the axis structures dynamic rather than static. > The original request for this feature provided examples > from the literature but none of them had even as many > as 7 axes in a single plot. So that seemed > adequate as a default. Failing to check for more > columns of input than expected is purely a bug. Here's the hunk of code from datafile.c that you are referring to: /* check we have room for at least 7 columns */ if (df_max_cols < 7) expand_df_column(7); df_no_cols = sscanf(line, df_format, &df_column[0].datum, &df_column[1].datum, &df_column[2].datum, &df_column[3].datum, &df_column[4].datum, &df_column[5].datum, &df_column[6].datum); and I'm trying to recall how the internal storage of data points works. If I'm remembering correctly, there is some type of limitation on the number of elements of a data point, e.g., x, y, x tolerance, y tolerance, z (or color), etc. owing to the use of a structure to store the data. Was the size of the structure up to seven elements, therefore it was generally thought that only up to seven columns of a data file line would be used? Is this parallel axes plot using the same limitation? I'm for allowing more than seven columns (note a recursive use of sscanf seems required), but at the same time internal memory storage should be adjustable too--a much bigger project. That is, for large data sets like images, having seven elements for each point could use a lot more memory than necessary. Dan |
|
From: sfeam <sf...@us...> - 2015-01-27 16:40:14
|
On Tuesday, 27 January 2015 03:06:16 AM Daniel J Sebald wrote: > On 01/26/2015 10:38 PM, sfeam wrote: > > On Monday, 26 January 2015 07:28:19 PM Philipp K. Janert wrote: > >> > >> The new plot style "with parallelaxes" > >> is cute. Unfortunately, it also dumps > >> core - whenever there are more than 7 > >> columns to plot. > > > > The maximum is set at compile time, but > > yes the default is 7. > > > >> 1) This limitation is not mentioned in > >> the doc. (As far as I can see.) > > > > I thought it was, but now I can't find it either. > > That is an oversight. > > > >> 2) Dumping core unconditionally is not > >> a good user experience. It would be > >> better to detect and reject the illegal > >> input and warn the user. > > > > Indeed. > > > >> 3) The limitation to 7 cols (which, I > >> suspect, depends on limitations in the > >> way "using" is parsed) puts the entire > >> plot style into question. The whole > >> point of parallelaxes plots (to the > >> degree they HAVE a point) is to deal > >> with high-dim data, easily exceeding 7. > > > > If you have such a use-case, go ahead and file a > > feature request. It should not be too hard to make > > the axis structures dynamic rather than static. > > The original request for this feature provided examples > > from the literature but none of them had even as many > > as 7 axes in a single plot. So that seemed > > adequate as a default. Failing to check for more > > columns of input than expected is purely a bug. > > Here's the hunk of code from datafile.c that you are referring to: > > /* check we have room for at least 7 columns */ > if (df_max_cols < 7) > expand_df_column(7); > > df_no_cols = sscanf(line, df_format, > &df_column[0].datum, > &df_column[1].datum, > &df_column[2].datum, > &df_column[3].datum, > &df_column[4].datum, > &df_column[5].datum, > &df_column[6].datum); Not correct. That code path applies _only_ to the command plot <foo> using 1:2:... "format-for-using-statement" where an explicit format is provided. > and I'm trying to recall how the internal storage of data points works. > If I'm remembering correctly, there is some type of limitation on the > number of elements of a data point, e.g., x, y, x tolerance, y > tolerance, z (or color), etc. owing to the use of a structure to store > the data. Was the size of the structure up to seven elements, therefore > it was generally thought that only up to seven columns of a data file > line would be used? Is this parallel axes plot using the same limitation? Not relevant. The data for parallel axis plots is allocated dynamically, one array per axis. Ethan |
|
From: Daniel J S. <dan...@ie...> - 2015-01-27 17:28:21
|
On 01/27/2015 10:25 AM, Philipp K. Janert wrote: > On Tue, 27 Jan 2015 03:06:16 -0600 > Daniel J Sebald<dan...@ie...> wrote: > >> On 01/26/2015 10:38 PM, sfeam wrote: >>> On Monday, 26 January 2015 07:28:19 PM Philipp K. Janert wrote: >>>> >>>> The new plot style "with parallelaxes" >>>> is cute. Unfortunately, it also dumps >>>> core - whenever there are more than 7 >>>> columns to plot. >>> >>> The maximum is set at compile time, but >>> yes the default is 7. >>> > > Daniel - > > Thanks for looking into this! > > One comment, though: before going down this > path (of extending the permissible number of > cols in "using") very far, let's consider > whether this is the right way to do it. > > Already, entering 7 explicit col numbers is a > pain. Entering 35 (or whatever) different ones > is simply not practical. Well, as with binary data, it could be that the data file and the command itself are constructed by automation, i.e., a computer program or script. Plus, it isn't that difficult to build up a long command in an input file using an editor. > Should the whole approach be rethought? Is > "plot" even the right command? I'd argue (not > having looked at the implementation) that we > should consider "splot", because splot naturally > deals with data on a rectangular grid (matrix data). I'm not sure a rectangular grid is any type of requisite. From the documentation, it sounds like the parallelaxes is a means to visually show correlation between disparate measurements. That is, if there are strong color bands, I assume that is more correlation, be it positive or negative. > And that's exactly the type of data used for > parallel axes plots. Put another way: splot > already handles dynamic rows AND columns. It actually may be its own type of plot, neither "plot" nor "splot". From what Ethan describes, I'm guessing that the data file is re-read once for each axis to get the column of data that is to appear on the axis. In some sense it's a one-dimensional plot, isn't it? That is, the data only appears on one axis only and it is simply that there are many axes. So, some analogous terms might be: plot -> plot2d splot -> plot3d paxis -> plot1d We can think of plot2d being a subset of plot3d, with a viewing angle that's orthographic. We can think of plot1d as being a subset of plot2d, again with a viewing angle that eliminates one of the dimensions. But parallel axes is some kind of new creature, basically connecting points across multiple plots. In theory it could be done with 2d and 3d plots, but would be a mess to look at of course. So, parallelaxis is more along the lines of "multiplot", isn't it? Should it's syntax fall more along that lines? E.g., set parallelaxis layout 7,1 Dan |
|
From: Philipp K. J. <ja...@ie...> - 2015-01-27 16:25:24
|
On Tue, 27 Jan 2015 03:06:16 -0600 Daniel J Sebald <dan...@ie...> wrote: > On 01/26/2015 10:38 PM, sfeam wrote: > > On Monday, 26 January 2015 07:28:19 PM Philipp K. Janert wrote: > >> > >> The new plot style "with parallelaxes" > >> is cute. Unfortunately, it also dumps > >> core - whenever there are more than 7 > >> columns to plot. > > > > The maximum is set at compile time, but > > yes the default is 7. > > Daniel - Thanks for looking into this! One comment, though: before going down this path (of extending the permissible number of cols in "using") very far, let's consider whether this is the right way to do it. Already, entering 7 explicit col numbers is a pain. Entering 35 (or whatever) different ones is simply not practical. Should the whole approach be rethought? Is "plot" even the right command? I'd argue (not having looked at the implementation) that we should consider "splot", because splot naturally deals with data on a rectangular grid (matrix data). And that's exactly the type of data used for parallel axes plots. Put another way: splot already handles dynamic rows AND columns. I'll toss in one more thought: whatever solution is devised, it must contain decent support for highlighting of individual records and groups of records. Parallel axes are simply not useful without that - and an implementation that does not support this action is little more than a demo. Best, Ph. > > Here's the hunk of code from datafile.c that you are referring to: > > /* check we have room for at least 7 columns */ > if (df_max_cols < 7) > expand_df_column(7); > > df_no_cols = sscanf(line, df_format, > &df_column[0].datum, > &df_column[1].datum, > &df_column[2].datum, > &df_column[3].datum, > &df_column[4].datum, > &df_column[5].datum, > &df_column[6].datum); > > and I'm trying to recall how the internal storage of data points > works. If I'm remembering correctly, there is some type of limitation > on the number of elements of a data point, e.g., x, y, x tolerance, y > tolerance, z (or color), etc. owing to the use of a structure to > store the data. Was the size of the structure up to seven elements, > therefore it was generally thought that only up to seven columns of a > data file line would be used? Is this parallel axes plot using the > same limitation? > > I'm for allowing more than seven columns (note a recursive use of > sscanf seems required), but at the same time internal memory storage > should be adjustable too--a much bigger project. That is, for large > data sets like images, having seven elements for each point could use > a lot more memory than necessary. > > Dan > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming. The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, > is your hub for all things parallel software development, from weekly > thought leadership blogs to news, videos, case studies, tutorials and > more. Take a look and join the conversation now. > http://goparallel.sourceforge.net/ > _______________________________________________ gnuplot-beta mailing > list gnu...@li... > Membership management via: > https://lists.sourceforge.net/lists/listinfo/gnuplot-beta |
|
From: Philipp K. J. <ja...@ie...> - 2015-01-27 17:44:53
|
[snip]
> > One comment, though: before going down this
> > path (of extending the permissible number of
> > cols in "using") very far, let's consider
> > whether this is the right way to do it.
> >
> > Already, entering 7 explicit col numbers is a
> > pain. Entering 35 (or whatever) different ones
> > is simply not practical.
>
> Well, as with binary data, it could be that the data file and the
> command itself are constructed by automation, i.e., a computer
> program or script. Plus, it isn't that difficult to build up a long
> command in an input file using an editor.
>
Maybe so, but I really consider it a mistake
to rely on it, ie: designing the UI under
the assumption that the user automates.
>
> > Should the whole approach be rethought? Is
> > "plot" even the right command? I'd argue (not
> > having looked at the implementation) that we
> > should consider "splot", because splot naturally
> > deals with data on a rectangular grid (matrix data).
>
> I'm not sure a rectangular grid is any type of requisite. From the
> documentation, it sounds like the parallelaxes is a means to visually
> show correlation between disparate measurements. That is, if there
> are strong color bands, I assume that is more correlation, be it
> positive or negative.
>
Yes, it is a prereq. The idea is that you have a
number of records, with each record having multiple
measurements. It is a safe (but not guaranteed)
assumption that each record has the same number
of measurements. (If this is not true, parallel
axis plots don't really apply - edge cases
notwithstanding.)
>
> > And that's exactly the type of data used for
> > parallel axes plots. Put another way: splot
> > already handles dynamic rows AND columns.
>
> It actually may be its own type of plot, neither "plot" nor "splot".
Ok, point well taken. From looking at the way it
currently works, I assumed that there was a desire to
"piggy-pack" parallel axis on existing infrastructure.
If so, then splot might be a better starting point.
But if that's not the guiding principle, then parallel
axis should not be treated as an isolated type, but
merely as a special case of multidimensional plots in
general. At the very least, they should be able to handle
polar coordinates together with "parallel axes" ("star plots").
> From what Ethan describes, I'm guessing that the data file is
> re-read once for each axis to get the column of data that is to
> appear on the axis. In some sense it's a one-dimensional plot, isn't
> it? That is, the data only appears on one axis only and it is simply
> that there are many axes. So, some analogous terms might be:
>
> plot -> plot2d
> splot -> plot3d
> paxis -> plot1d
>
> We can think of plot2d being a subset of plot3d, with a viewing angle
> that's orthographic. We can think of plot1d as being a subset of
> plot2d, again with a viewing angle that eliminates one of the
> dimensions. But parallel axes is some kind of new creature,
> basically connecting points across multiple plots. In theory it
> could be done with 2d and 3d plots, but would be a mess to look at of
> course. So, parallelaxis is more along the lines of "multiplot",
> isn't it? Should it's syntax fall more along that lines? E.g.,
>
> set parallelaxis layout 7,1
I think this is going in a similar direction to what I
said earlier - treat it as its own beast, but more
generally. The idea of hitching onto the "multiplot"
features strikes me as interesting. That might deliver
the "star plot" application for (almost) free, by the way.
>
> Dan
|
|
From: Daniel J S. <dan...@ie...> - 2015-01-27 18:14:46
|
On 01/27/2015 11:44 AM, Philipp K. Janert wrote: > > [snip] > >>> One comment, though: before going down this >>> path (of extending the permissible number of >>> cols in "using") very far, let's consider >>> whether this is the right way to do it. >>> >>> Already, entering 7 explicit col numbers is a >>> pain. Entering 35 (or whatever) different ones >>> is simply not practical. >> >> Well, as with binary data, it could be that the data file and the >> command itself are constructed by automation, i.e., a computer >> program or script. Plus, it isn't that difficult to build up a long >> command in an input file using an editor. >> > > Maybe so, but I really consider it a mistake > to rely on it, ie: designing the UI under > the assumption that the user automates. There are plenty of programs that use gnuplot in an automated fashion. Large amounts of data isn't something new. But the parallel axes plots can only be so many before spatial resolution is lost. That is, if there were one hundred parallel axes, it would be difficult to resolve any useful information between axes. Then again, if somehow the user wanted 100 axes strewn across some really wide bitmap that say is put into an HTML viewer so that a scroll bar could be used to scan across all axes, then maybe. In any case, it seems that a dynamic number of axes is the way to go. >> set parallelaxis layout 7,1 > > I think this is going in a similar direction to what I > said earlier - treat it as its own beast, but more > generally. The idea of hitching onto the "multiplot" > features strikes me as interesting. That might deliver > the "star plot" application for (almost) free, by the way. I kind of like that approach instead of putting parallel axes "inside" of "plot". As Ethan's previous post described, the axes thought of as an object (structure, pointer) makes a lot of sense. I'd rather have the syntax sort of reflect what is happening internally--somehow that seems more flexible, say if the user wants to put data points on the axes (writing "with parallelaxes with points" seems dodgy). Dan |
|
From: Ethan A M. <sf...@us...> - 2015-01-27 17:47:22
|
On Monday, 26 January, 2015 19:28:19 Philipp K. Janert wrote:
> 3) The limitation to 7 cols (which, I
> suspect, depends on limitations in the
> way "using" is parsed) puts the entire
> plot style into question.
Somewhat tangential, but here goes.
There are 3 related but distinct limitations:
1) The number of columns read from a data file.
This is essentially unlimited. For example:
plot 'data' using 0:(sum [i=1:999] column(i))
will happily read 999 columns of input data
2) The number of properties associated with a
single "point" in a plot. This is currently 8:
x y xlow xhigh ylow yhigh z color
The first 7 are fields in (struct coordinate).
The 8th is kept in a separate location, but only
if the plot involves variable (i.e. per-point) color.
3)The number of fields in a "using" specifier.
This is currently defined by
#define MAXDATACOLS (MAX_NUM_VAR+2)
and is primarily relevant to the "fit" command
However, none of these are the reason for a cap on the number
of parallel axes. That comes instead from a poor design
decision now lost in the mists of program history.
There is one "struct axis" for each axis holding the
range limits, scaling, tic information, axis labels, etc.
That's fine.
But the axis parameter passed to all the subroutines
and macros that manipulate this data is not a pointer to
an instance of an axis structure, but instead an index into
the fixed array
struct axis axis_array[AXIS_ARRAY_SIZE]
This is bad, because you can't just allocate a new
axis structure and pass it to any of the existing
subroutines or macros.
So the first step towards allowing dynamically allocated
axes has to be refactoring all the code to use an axis
pointer rather than an array index. Not terribly
difficult, but it touches a huge amount of code.
I think this would be a nice cleanup, but until now
there has not been sufficient motivation to tackle it.
Ethan |
|
From: Hans-Bernhard B. <HBB...@t-...> - 2015-01-28 21:55:46
|
Am 27.01.2015 um 18:46 schrieb Ethan A Merritt: > However, none of these are the reason for a cap on the number > of parallel axes. That comes instead from a poor design > decision now lost in the mists of program history. I disagree about that decision having been a poor one. It was correct at the time it was originally made, because it matched the capabilities and of the program at the time. There was really no way anyone could have anticipated the amount of stuff that was later grafted onto the original design. A truly bad design would never have withstood 20+ years of add-ons. > This is bad, because you can't just allocate a new > axis structure and pass it to any of the existing > subroutines or macros. That wouldn't work anyway, and for rather more important reasons than the implementation detail of whether axis methods' primary argument is an index or a pointer. The axes have to be in an array because the indices into that array have more meaning than just as the indicator of one array entry to work with. The sequence of AXIS_INDEX enumeration values has been the same since just about forever, and it has extra properties. The entire first vs. second axes mechanism is built on these properties. IOW: as long as there remains * any use of the macros FIRST_AXES and SECOND_AXES * any loop over a variable of type AXIS_INDEX * any inequality comparison among AXIS_INDEX values you won't get rid of axis_array[]. The best one could do before that would be to make axis_array[] itself dynamically sized. That, however, would mean that _all_ uses of pointer-to-AXIS would have to be forbidden, because the array itself could move when reallocated. I.e. it would push things into the opposite direction of your intention. |
|
From: sfeam <sf...@us...> - 2015-01-29 01:55:26
|
On Wednesday, 28 January 2015 10:55:40 PM Hans-Bernhard Bröker wrote:
> Am 27.01.2015 um 18:46 schrieb Ethan A Merritt:
>
> > However, none of these are the reason for a cap on the number
> > of parallel axes. That comes instead from a poor design
> > decision now lost in the mists of program history.
>
> I disagree about that decision having been a poor one. It was correct at
> the time it was originally made, because it matched the capabilities and
> of the program at the time. There was really no way anyone could have
> anticipated the amount of stuff that was later grafted onto the original
> design. A truly bad design would never have withstood 20+ years of add-ons.
Closer to 30 years now :-)
> > This is bad, because you can't just allocate a new
> > axis structure and pass it to any of the existing
> > subroutines or macros.
>
> That wouldn't work anyway, and for rather more important reasons than
> the implementation detail of whether axis methods' primary argument is
> an index or a pointer.
>
> The axes have to be in an array because the indices into that array have
> more meaning than just as the indicator of one array entry to work with.
> The sequence of AXIS_INDEX enumeration values has been the same since
> just about forever, and it has extra properties. The entire first vs.
> second axes mechanism is built on these properties.
>
> IOW: as long as there remains
>
> * any use of the macros FIRST_AXES and SECOND_AXES
> * any loop over a variable of type AXIS_INDEX
> * any inequality comparison among AXIS_INDEX values
>
> you won't get rid of axis_array[].
I take your point, but that by itself isn't an argument against
designing the various subroutines to accept pointers rather than
indices. It is no harder to call sub(&array_axis[INDEX]) than
it is to call sub(INDEX). All the FIRST and SECOND axes could
continue to live in an array just as they do now. But unlike now
it would also be possible to dynamically allocate a temporary axis
structure, or a contiguous array of them if that makes sense.
As to manipulations using FIRST_AXES and SECOND_AXES,
there are not very many of these. I suspect it would suffice to
add a field or flag to the axis structure.
Instead of having code like axis.c:1381
TBOOLEAN axis_is_second = ((axis / SECOND_AXES) == 1);
you would have
TBOOLEAN axis_is_second = axis->is_second_axis;
> The best one could do before that would be to make axis_array[] itself
> dynamically sized. That, however, would mean that _all_ uses of
> pointer-to-AXIS would have to be forbidden, because the array itself
> could move when reallocated. I.e. it would push things into the
> opposite direction of your intention.
I agree that option is not very workable.
Ethan
|