Menu
▾
▴
gnuplot-beta
|
From: Karl R. <ra...@un...> - 2014-10-02 13:10:26
|
With the new version of the timecolumn() function now including a format string, i think it is time to deprecate the use of the "set xdata time" / "set timefmt" combination, the former influencing inconveniently both input and output. This would technically (as I see) only need a new option "time" to "set xtics" so the interpretation of the output format string can be controlled without "set xdata time". Worst part of this would probably be rewriting a lot of documentation, for which i´d volunteer. Do this for gp50? The technical change would be backward-compatible, but changes of recommended syntax should probably not happen at minor releases. Best, Karl -- Karl-Friedrich Ratzsch (Dipl. Chem.) Freiburger Materialforschungszentrum / Universität Freiburg Stefan-Meier-Straße 21, 79104 Freiburg im Breisgau Tel. 0761/203-4748 Fax:-4701 ra...@un... |
|
From: Ethan A M. <sf...@us...> - 2014-10-02 17:55:54
|
On Thursday, 02 October, 2014 14:57:15 Karl Ratzsch wrote: > > With the new version of the timecolumn() function now including a format > string, i think it is time to deprecate the use of the "set xdata time" > / "set timefmt" combination, > the former influencing inconveniently both input and output. I agree that it would be worth re-thinking these commands. > This would technically (as I see) only need a new option "time" to "set > xtics" so the interpretation of the output format string can be > controlled without "set xdata time". It's more complicated than that. For one thing, the desire for time formats is not limited to axis labeling, and even when labeling an axis it does not follow that "time" is the same thing as "date". See for example https://sourceforge.net/p/gnuplot/feature-requests/392/ which points out that it's really annoying if a quantity that represents elapsed time is treated as a date rather than simply some number of days+hours+minutes+seconds. In that particular use case the quantity is a vector length, not an axis coordinate per se, so "set xtics time" would not really fill the need. Yes the x coordinate is a date, but the vector length is elapsed time - two different formats are required. > Worst part of this would probably be rewriting a lot of documentation, > for which i´d volunteer. Do this for gp50? It turns out the current documentation for "set timefmt" is simply wrong (left over from version 4.4 I guess). I did a quick re-write yesterday for inclusion in v5, but yes it would be nice to have a more comprehensive revision, better yet if it comes with examples or demos. > The technical change would be backward-compatible, but changes of > recommended syntax should probably not happen at minor releases. Input ===== We already broke some backwards compatibility by adding a 2nd parameter to timecolumn(). The new version obviates the need for "set xdata time" to handle input, although arguably it is uglier to put the information in the "using" part of every plot command rather than setting it only once via "set xdata time; set timefmt x 'foo'". Output ====== I think this is actually the messier part. timecolumn() can be used to differentiate individual columns on input, or for that matter different formats in different input files on the same plot. But we have no corresponding mechanism for writing out times that are not dates. Not for axis tic labels and certainly not for arbitrary columns written by "set table" or "plot .... with table". Your suggestion for a "set xtics time 'timefmt'" would partially address this. But I think we would want to distinguish set xtics {time|date|numeric|geographic|other?} And we would still need some additional way to control tabuler output. Other issues ================= That leaves the problem that "set xdata time; set timefmt foo" is used to interpret axis ranges as well as input data. This is IMHO completely confusing already, since it doesn't match what is used to actually label the axis ("set format x"). I don't have a good suggestion on how to fix this. Related to that, I think that mousing readout and zoom also depend on "set xdata". V5 supports geographic coordinates (degrees minutes seconds) for axis labels, but not for data input. I.e. "set xdata geographic" affects only the output side. So it would go naturally into a revised "set xtics" command and could be removed as an option to "set xdata" since it was never in v4. Ethan |
|
From: Karl R. <ra...@un...> - 2014-10-03 13:04:20
|
On 02.10.2014 19:55, Ethan A Merritt wrote: > https://sourceforge.net/p/gnuplot/feature-requests/392/ > which points out that it's really annoying if a quantity that represents > elapsed time is treated as a date rather than simply some number of > days+hours+minutes+seconds. In that particular use case the > quantity is a vector length, not an axis coordinate per se, > so "set xtics time" would not really fill the need. Yes the x coordinate > is a date, but the vector length is elapsed time - two different formats > are required. Right, "date" is a more appropriate name for the option to "set xtics"/"set format". Later a new option "time" can be introduced, which checks for the biggest unit in the format string and consequently does not wrap e.g. at 24hs. Other format string varieties ("numeral","geographic", ...) can be added as necessary. For tabular output, "set format <>" could be extended to allow setting the format for individual column numbers. > We already broke some backwards compatibility by adding a 2nd > parameter to > timecolumn(). The new version obviates the need for "set xdata time" to > handle input, although arguably it is uglier to put the information > in the > "using" part of every plot command rather than setting it only once via > "set xdata time; set timefmt x 'foo'". Would it be difficult to make the second parameter to timecolumn() optional, and use whatever is set via "set timefmt" if no format string is given? Alternatively, new command "set data <coln> [time/numeral/...] <formatstring>" ? Karl |
|
From: Ethan A M. <sf...@us...> - 2014-10-04 00:40:07
|
On Friday, 03 October, 2014 15:04:07 Karl Ratzsch wrote: > > We already broke some backwards compatibility by adding a 2nd > > parameter to > > timecolumn(). The new version obviates the need for "set xdata time" to > > handle input, although arguably it is uglier to put the information > > in the > > "using" part of every plot command rather than setting it only once via > > "set xdata time; set timefmt x 'foo'". > > Would it be difficult to make the second parameter to timecolumn() > optional, and use whatever is set via "set timefmt" if no format > string is given? Yes, it is possible to trap the case of a single parameter and fall back to a default timefmt. That loses the ability to have separate default formats for the x and y axes, but see below. The whole idea of having the time format be per-axis rather than per-datafile makes little sense to me. So I'm now thinking we could remove the timefmt field from (struct axis) and have one global default. This is in fact what the 4.6 documentation describes, even though it hasn't been true for a very long time. So although it would internally be a change, it would keep the previous documentated behavior. It would only affect scripts that used the undocumented command: set timefmt y "format". Ethan |
|
From: <pl...@pi...> - 2014-10-04 07:49:29
|
On 10/04/14 02:17, Ethan A Merritt wrote: > Yes, it is possible to trap the case of a single parameter and > > fall back to a default timefmt. That loses the ability to have > > separate default formats for the x and y axes, but see below. > I was going to suggest this myself. It would be preferable to maintain b/w compatibility if possible. While there is the possibility due to major version change this should only be exercised when there is a clear need to do so. IMO. Peter. |
|
From: Karl R. <ra...@un...> - 2014-10-04 18:37:23
|
On 04.10.2014 02:17, Ethan A Merritt wrote: > On Friday, 03 October, 2014 15:04:07 Karl Ratzsch wrote: >> Would it be difficult to make the second parameter to timecolumn() >> optional, and use whatever is set via "set timefmt" if no format >> string is given? > > Yes, it is possible to trap the case of a single parameter and > fall back to a default timefmt. That loses the ability to have > separate default formats for the x and y axes, but see below. > > The whole idea of having the time format be per-axis rather than > per-datafile makes little sense to me. So I'm now thinking we could > remove the timefmt field from (struct axis) and have one global default. > This is in fact what the 4.6 documentation describes, even though > it hasn't been true for a very long time. So although it would > internally be a change, it would keep the previous documentated > behavior. It would only affect scripts that used the undocumented > command: set timefmt y "format". Even that can be kept, imo. timecolumn() could check for a format string as (in that order) second parameter, in timefmt <colnum> (new), timefmt <x/y), general timefmt, and use the first it finds set. Karl -- Karl-Friedrich Ratzsch (Dipl. Chem.) Freiburger Materialforschungszentrum, Universität Freiburg Stefan-Meier-Straße 21, 79104 Freiburg i. Br. Tel.: 0761/203-4748 Fax: -4701 |
|
From: sfeam <sf...@us...> - 2014-10-04 19:02:35
|
On Saturday, 04 October 2014 08:37:17 PM Karl Ratzsch wrote: > On 04.10.2014 02:17, Ethan A Merritt wrote: > > On Friday, 03 October, 2014 15:04:07 Karl Ratzsch wrote: > >> Would it be difficult to make the second parameter to timecolumn() > >> optional, and use whatever is set via "set timefmt" if no format > >> string is given? > > > > Yes, it is possible to trap the case of a single parameter and > > fall back to a default timefmt. That loses the ability to have > > separate default formats for the x and y axes, but see below. > > > > The whole idea of having the time format be per-axis rather than > > per-datafile makes little sense to me. So I'm now thinking we could > > remove the timefmt field from (struct axis) and have one global default. > > This is in fact what the 4.6 documentation describes, even though > > it hasn't been true for a very long time. So although it would > > internally be a change, it would keep the previous documentated > > behavior. It would only affect scripts that used the undocumented > > command: set timefmt y "format". > > Even that can be kept, imo. timecolumn() could check for a format > string as (in that order) second parameter, in timefmt <colnum> > (new), timefmt <x/y), general timefmt, and use the first it finds set. That isn't possible, because at the level of interpretation where f_timecolumn() receives control it has no idea which axis may or may not be involved. So it can't look up a per-axis quantity. This wasn't working reliably in earlier versions either, which was part of the reason for adding the second parameter. Since the per-axis timeformat was not reliable and was never documented anyhow (perhaps for that reason?), I don't feel badly about failing to provide a 100% backwards-compatible replacement. I have a patch to provide backwards compatibility for the common case: plot <foo> using (timecolumn(N)):M It does not catch more complicated cases in which the timecolumn() call is part of a longer expression, as in plot <foo> using (Offset_2_weeks + timecolumn(N)):M I don't see a way to catch that without adding additional bookkeeping code to the expression parsing. That might be worth looking into at some point, but right now it would be a major distraction. Ethan |
|
From: sfeam <sf...@us...> - 2014-10-05 03:27:32
|
I have made a set of changes in 5.1 CVS related to timefmt.
These changes are not yet in 5.0, but unless they produce unforseen
problems I expect they will be added before the final release of 5.0.
The changes can be summarized briefly.
- The per-axis flag for how to interpret input/output format statements
has been split into two separate flags
axis->datatype continues to select between numeric or time data on input
axis->tictype (new) specifies how the axis tics are to be handled (output)
- The new flag is set/unset by a corresponding new command option
set xtics {numeric|time|geographic}
For now the command "set xdata time" implicitly also does "set xtics time"
so all old scripts should continue to act as they did previously with
regard to time formats
- The per-axis string axis->timefmt has been replaced by a single global
timefmt string. This is what the documentation always described,
although it was not true in the code.
Existing scripts will only see a change if they used the undocumented
command: set timefmt y 'something-different-from-x'.
This case is handled more flexibly in version 5 by
plot ... using (timecolumn(N,format1)):(timecolumn(M,format2))
- XYZ positions are printed using time/date formatting if it is active.
This makes the output consistent with input requirements.
I.e. the sequence of commands
set xdata time
set label "FOO" at "01/02/14,11:23", graph 0.5
save 'temp'
load 'temp'
show label
Did not act as you would expect in version 4. Now it does.
- The timecolumn(N,format) command tries to detect being called
with only 1 parameter, as was the case in version 4. In this
case the version 5 code now emulates version 4 behavior, so this
is an improvement, albeit imperfect, in backwards-compatibility.
In short these changes are intended to both extend the flexibility
of time format handling in version 5 and increase the backwards
compatibility with scripts written for version 4. Please report
any use cases where these goals not succeeded.
Ethan
|
|
From: Karl R. <ra...@un...> - 2014-10-05 13:24:52
|
i just checked out the latest 5.1 cvs, and it seems to work just nicely. Am 04.10.2014 21:00, schrieb sfeam: > plot <foo> using (Offset_2_weeks + timecolumn(N)):M > > I don't see a way to catch that without adding additional bookkeeping Also no problem here, if the "timefmt" is set, so there seems to be full compatibility to the (documented part of) v4 behaviour. I noticed two minor glitches: - Changing back to "set xtics numeric" keeps a previously set time formatstring, which then throws an error "Bad format character". I think the different output format strings should be kept in separate variables, with default values for each. (Dang, that would break backward compatibility to some extent, depending on the order of commands. Perhaps store them in additional variables, and copy them to the actually used tics format variable whenever an "(un)set xdata time" or "set xtics time/numeric/.." command is issued. "set xtics format <fstring>" would only change the latter, then.) - The synonymous "set format x/y/x2/.. <formatstring>" command also needs the time/numeric/.. option. Karl -- Karl-Friedrich Ratzsch (Dipl. Chem.) Freiburger Materialforschungszentrum / Universität Freiburg Stefan-Meier-Straße 21, 79104 Freiburg im Breisgau Tel. 0761/203-4748 Fax:-4701 ra...@un... |
|
From: Karl R. <ra...@un...> - 2014-10-05 15:19:40
|
Am 05.10.2014 15:27, schrieb Karl Ratzsch: > i just checked out the latest 5.1 cvs, and it seems to work just nicely. > One more : Coordinates given for e.g. "set arrow" still need "xdata time": set timefmt "%H:%M:%S" set arrow 1 from "12:31:01.4",10 to "12:31:01.7",35 This is alright for backward compatibility, however the new syntax cannot decide if a string is just a numeral stored as a string variable or a timestring that needs to be interpreted according to the format string set by "set timefmt". To resolve this, i propose to give the function strptime() a default format e.g. -1 which refers to the momentary "set timefmt" setting. set arrow 1 from strptime(-1,"12:31:01.4"),10 \ to strptime(-1,"12:31:01.7"),35 For actual use, the doc can then suggest defining a function TSx(timestr) = strptime(-1,timestr) to save typing. Karl P.S. Should there be a threat about a possible future removal of "set xdata time" in the docs? Perhaps not. The underlying input/output problem should be resolved, and for people who plot a lot of time scales it is just quite convenient. P.P.S. I noticed this while working on gnuplot.doc but will stop now until this is decided. |
|
From: sfeam <sf...@us...> - 2014-10-05 15:53:59
|
On Sunday, 05 October 2014 03:27:34 PM Karl Ratzsch wrote: > i just checked out the latest 5.1 cvs, and it seems to work just nicely. > > Am 04.10.2014 21:00, schrieb sfeam: > > plot <foo> using (Offset_2_weeks + timecolumn(N)):M > > > > I don't see a way to catch that without adding additional bookkeeping After making the large set of changes I went back and modified the parsing to handle "timecolumn" as a special case with a variable number of parameters. So this caveat no longer applies. > > - Changing back to "set xtics numeric" keeps a previously set time > formatstring, which then throws an error "Bad format character". The analogous problem was present before this change also: version 4: set xdata time set format x "%b-%Y" plot [0:1e8] # All fine unset xdata replot # Bad format character The only difference is that now you can trigger the same thing with "set xtics". > I think > the different output format strings should be kept in separate > variables, with default values for each. (Dang, that would break > backward compatibility to some extent, depending on the order of > commands. Perhaps store them in additional variables, and copy them to > the actually used tics format variable whenever an "(un)set xdata time" > or "set xtics time/numeric/.." command is issued. "set xtics format > <fstring>" would only change the latter, then.) That's what the version 4 code was doing (although it was not documented). But it didn't avoid the problem above. Maybe we can come up with a clever improvement. > - The synonymous "set format x/y/x2/.. <formatstring>" command also > needs the time/numeric/.. option. I was thinking that putting the keyword there was an alternative to putting in "set xtics". But maybe you are right that it should be allowed in either place. If you don't give any keyword at all in such a command, would it reset to "numeric" or would it leave the current setting unchanged? I can see advantages either way. |
|
From: sfeam <sf...@us...> - 2014-10-05 17:04:09
|
On Sunday, 05 October 2014 05:22:22 PM Karl Ratzsch wrote: > Am 05.10.2014 15:27, schrieb Karl Ratzsch: > > i just checked out the latest 5.1 cvs, and it seems to work just nicely. > > > > One more : > > Coordinates given for e.g. "set arrow" still need "xdata time": > > set timefmt "%H:%M:%S" > set arrow 1 from "12:31:01.4",10 to "12:31:01.7",35 Yes, because all positional coordinates are treated as "input" rather than "output". So axis ranges, objects, labels, etc all use xdata+timefmt rather than xtics+format. That's the way it has always been. I don't see a strong argument for changing it. No matter what you choose it's going to be at least a bit confusing. And really it does make sense. That's the format the time coordinates would require if the vector endpoints were being plotted from a data file. Shouldn't the coordinate format be the same regardless of whether the endpoints come from a file or from the command line? > This is alright for backward compatibility, however the new syntax > cannot decide if a string is just a numeral stored as a string variable > or a timestring that needs to be interpreted according to the format > string set by "set timefmt". ?? I'm not following you there. I thought that the behavior has always been, and continues to be, that a bare number was treated as numerical seconds while a string was fed through strptime. A numeral stored as a string also works, so long as timefmt contains "%s". I might have broken this for some particular case, but I did test it for labels and axis ranges. Do you have an example of breakage? > To resolve this, i propose to give the function strptime() a default > format e.g. -1 which refers to the momentary "set timefmt" setting. > > set arrow 1 from strptime(-1,"12:31:01.4"),10 \ > to strptime(-1,"12:31:01.7"),35 > > For actual use, the doc can then suggest defining a function > > TSx(timestr) = strptime(-1,timestr) > > to save typing. Might as well just do the save for strptime as I did for timecolumn, change it internally to accept a variable number of parameters. 1 parameter means "use default timefmt". 2 parameters means "use the time format in parameter 1". But I don't think this is needed. > P.S. Should there be a threat about a possible future removal of "set > xdata time" in the docs? Perhaps not. The underlying input/output > problem should be resolved, and for people who plot a lot of time scales > it is just quite convenient. "set xdata time" is still needed in order to control input data parsing. The recent changes only affect how the tic labels and mouse coord readout are handled on output. The input processing remains the same as before unless I inadvertently broke something. Ethan |
|
From: Karl R. <ra...@un...> - 2014-10-06 16:31:06
|
Am 06.10.2014 um 17:01 schrieb sfeam:
> On Monday, 06 October 2014 12:53:07 PM Karl Ratzsch wrote:
>
>> Wasn´t the idea to deprecate "set xdata time" because it always
>> changes the ticslabel format type? I don´t see how it is still
>> necessary now, except for backward compatibility, of course.
>
>
> I'm not following you here. If we deprecated "set xdata time",
> wouldn't we just have to introduce another command that did the
> same thing? What would be the point?
There is no need for a replacement, i think. timecolumn() and strptime()
can handle all timedata input. Plus "_data time" is limited to certain
columns, depending on the plot style, and it is limited to one format.
So in many cases, people cannot use "_data time" anyway. For the sake of
a uniform UI, i´d say don´t use it at all.
> The thing that we might want to change is that "set xdata time"
> might stop _also_ affecting 'set xtics time'. That would make the
> input/output separation more complete, but would not be backwards
> compatible.
Ah. We´ve been talking under different presumptions. ;-)
Here´s mine:
It´s rather impossible to really separate in/output with "set xdata
time", because it´s nomenclature says "plot abscissa" when it really
means "first input column". As this can´t be fixed, i say keep it for
backward compatibility, but rebuild the functionality otherwise.
The functionality we already have now, with timecolumn(), strptime() and
"set _tics time", so i thought the change ought to be made now, with 5.0
at the door.
One small thing is still missing, imho: "set arrow" et al. should check
if coordinates are set to time mode with "set _tics" (as it checks for
"xdata time" now), and use the format set via "set timefmt" in that
case. (not the one used to format the tic labels.)
Additional features are possible, of course:
One could for example introduce something like "set data <colnum>
{time|date|..} {format}" so people don´t have to use the lenghty
timecolumn() function every time.
What´d you think?
Best, Karl
--
Karl-Friedrich Ratzsch (Dipl. Chem.)
Freiburger Materialforschungszentrum / Universität Freiburg
Stefan-Meier-Straße 21, 79104 Freiburg im Breisgau
Tel. 0761/203-4748 Fax:-4701
ra...@un...
|
|
From: Ethan A M. <sf...@us...> - 2014-10-06 17:32:11
|
On Monday, 06 October, 2014 18:33:42 Karl Ratzsch wrote:
> Am 06.10.2014 um 17:01 schrieb sfeam:
> > On Monday, 06 October 2014 12:53:07 PM Karl Ratzsch wrote:
> >
> >> Wasn´t the idea to deprecate "set xdata time" because it always
> >> changes the ticslabel format type? I don´t see how it is still
> >> necessary now, except for backward compatibility, of course.
> >
> >
> > I'm not following you here. If we deprecated "set xdata time",
> > wouldn't we just have to introduce another command that did the
> > same thing? What would be the point?
>
> There is no need for a replacement, i think. timecolumn() and strptime()
> can handle all timedata input. Plus "_data time" is limited to certain
> columns, depending on the plot style, and it is limited to one format.
>
> So in many cases, people cannot use "_data time" anyway.
I think everyone agrees on that specific point. That was why
timecolumn() got a 2nd parameter, even before this recent round of revision.
> For the sake of a uniform UI, i´d say don´t use it at all.
But there we run head-on into the policy for backward compatibility
if possible. Yes you can now write scripts that do not use "set xdata".
But that by itself doesn't justify breaking old scripts that do use it.
And getting back to your other example, it's still convenient to say
set xdata time
set timefmt "%d-%b"
set arrow 1 from "01-Jan", y0 to "05-Sep",y0+ydel
rather than
timefmt = "%d-%b"
set arrow 1 from strptime(timefmt,"01-Jan"),y0 \
to strptime(timefmt,"05-Sep"),y0+ydel
> > The thing that we might want to change is that "set xdata time"
> > might stop _also_ affecting 'set xtics time'. That would make the
> > input/output separation more complete, but would not be backwards
> > compatible.
>
> Ah. We´ve been talking under different presumptions. ;-)
> Here´s mine:
>
> It´s rather impossible to really separate in/output with "set xdata
> time", because it´s nomenclature says "plot abscissa" when it really
> means "first input column".
The more important point is that it means "use timefmt specifiers
rather than normal C numeric format specifiers". Without this flag,
the gnuplot parsing routines don't know which library routine to
feed the format statement to. That functionality is not tied to any
particular input column or file, since it is also relevant to position info
and in particular to axis ranges. I think of "set xdata" as being
parallel in syntax to "set xrange".
> As this can´t be fixed, i say keep it for backward compatibility, but rebuild
> the functionality otherwise.
Exactly.
But "keep it for backward compatibility" is opposite to "deprecate it",
right?
> One small thing is still missing, imho: "set arrow" et al. should check
> if coordinates are set to time mode with "set _tics" (as it checks for
> "xdata time" now), and use the format set via "set timefmt" in that
> case. (not the one used to format the tic labels.)
I'm confused. I thought we agreed in a separate Email thread that
positional coordinates, including arrows, should continue to be considered
'input' (using xdata/timefmt) rather than 'output' (using xtics/format)?
Under the new scheme it would never be correct to mix these up,
using axis->tictype (internal flag for "set xtics") to control use of timefmt,
which is controlled instead by axis->datatype (internal flag for "set xdata").
> Additional features are possible, of course:
> One could for example introduce something like "set data <colnum>
> {time|date|..} {format}" so people don´t have to use the lenghty
> timecolumn() function every time.
Sure, but let's leave that for later. It would be a purely new feature,
so it doesn't affect current discussions of compatibility or
syntax change.
Ethan
|
