gnuplot-beta

30-year time offset reading nonuniform matrix %s input

[Tait <gnu...@t4...>]

Consider these two examples. First:
  reset
  set datafile separator ","
  set xdata time
  set timefmt "%s"
  set format x "%Y-%m-%d %H:%M:%S"
  set view 47,54
  splot '-' using 1:2:3 with impulses
  1364987000,50,19
  1364987000,100,1
  1364987000,150,0

  1364990600,50,15
  1364990600,100,4
  1364990600,150,1

  1364942000,50,9
  1364942000,100,5
  1364942000,150,7

  1365001400,50,11
  1365001400,100,7
  1365001400,150,2
  e

... and second:
  reset
  set datafile separator ","
  set xdata time
  set timefmt "%s"
  set format x "%Y-%m-%d %H:%M:%S"
  set view 47,54
  splot '-' nonuniform matrix using 2:1:3 with impulses
  0,50.00,100.00,150.00
  1364987000,19,1,0
  1364990600,15,4,1
  1364942000,9,5,7
  1365001400,11,7,2
  e
  e

(I don't understand why the second example requires two 'e's either,
but I digress...)

In the first case, the year is correctly set to 2013. In the second,
it is displayed as 2043. This seems to be a mistake in the nonuniform
matrix time input?

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Wednesday, April 03, 2013 11:38:27 am Tait wrote:
> 
> Consider these two examples. First:
>   reset
>   set datafile separator ","
>   set xdata time
>   set timefmt "%s"
>   set format x "%Y-%m-%d %H:%M:%S"
>   set view 47,54
>   splot '-' using 1:2:3 with impulses
>   1364987000,50,19
>   1364987000,100,1
>   1364987000,150,0
> 
>   1364990600,50,15
>   1364990600,100,4
>   1364990600,150,1
> 
>   1364942000,50,9
>   1364942000,100,5
>   1364942000,150,7
> 
>   1365001400,50,11
>   1365001400,100,7
>   1365001400,150,2
>   e
> 
> ... and second:
>   reset
>   set datafile separator ","
>   set xdata time
>   set timefmt "%s"
>   set format x "%Y-%m-%d %H:%M:%S"
>   set view 47,54
>   splot '-' nonuniform matrix using 2:1:3 with impulses
>   0,50.00,100.00,150.00
>   1364987000,19,1,0
>   1364990600,15,4,1
>   1364942000,9,5,7
>   1365001400,11,7,2
>   e
>   e
> 
> (I don't understand why the second example requires two 'e's either,
> but I digress...)
> 
> In the first case, the year is correctly set to 2013. In the second,
> it is displayed as 2043. This seems to be a mistake in the nonuniform
> matrix time input?

Does anyone know why, way back in the mists of time, gnuplot decided to 
define its own epoch boundary as 1-jan-2000 rather than use the conventional
unix epoch boundary 1-jan-1970? 
  
Did it have something to do with fitting an integer number of seconds 
in a 32-bit, or even 16-bit, integer? 

How many things would break if we did away with this 30 year disagreement?

	Ethan


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Juhász P. <pet...@gm...>]

On Wed, 2013-04-03 at 12:36 -0700, Ethan Merritt wrote:
> On Wednesday, April 03, 2013 11:38:27 am Tait wrote:
> > 
> > Consider these two examples. First:
> >   reset
> >   set datafile separator ","
> >   set xdata time
> >   set timefmt "%s"
> >   set format x "%Y-%m-%d %H:%M:%S"
> >   set view 47,54
> >   splot '-' using 1:2:3 with impulses
> >   1364987000,50,19
> >   1364987000,100,1
> >   1364987000,150,0
> > 
> >   1364990600,50,15
> >   1364990600,100,4
> >   1364990600,150,1
> > 
> >   1364942000,50,9
> >   1364942000,100,5
> >   1364942000,150,7
> > 
> >   1365001400,50,11
> >   1365001400,100,7
> >   1365001400,150,2
> >   e
> > 
> > ... and second:
> >   reset
> >   set datafile separator ","
> >   set xdata time
> >   set timefmt "%s"
> >   set format x "%Y-%m-%d %H:%M:%S"
> >   set view 47,54
> >   splot '-' nonuniform matrix using 2:1:3 with impulses
> >   0,50.00,100.00,150.00
> >   1364987000,19,1,0
> >   1364990600,15,4,1
> >   1364942000,9,5,7
> >   1365001400,11,7,2
> >   e
> >   e
> > 
> > (I don't understand why the second example requires two 'e's either,
> > but I digress...)
> > 
> > In the first case, the year is correctly set to 2013. In the second,
> > it is displayed as 2043. This seems to be a mistake in the nonuniform
> > matrix time input?
> 
> Does anyone know why, way back in the mists of time, gnuplot decided to 
> define its own epoch boundary as 1-jan-2000 rather than use the conventional
> unix epoch boundary 1-jan-1970? 
>   
> Did it have something to do with fitting an integer number of seconds 
> in a 32-bit, or even 16-bit, integer? 
> 
> How many things would break if we did away with this 30 year disagreement?
> 
> 	Ethan
> 
> 

Currently this disagreement manifests itself even in the user interface:
the meaning of "%s" is different, depending on whether it was used as an
input or output spec.

Doing away with it might break a few scripts that depend on the epoch
being 2000.0, but on the plus side it would make everything clearer and
more consistent - both the internal implementation and the user
documentation, not to mention that it would also be consistent with the
rest of the (Unix) world.

I might add that 4.6.0 would have been a good place to make this change.

Peter

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Wednesday, April 03, 2013 02:12:55 pm Juhász Péter wrote:
> On Wed, 2013-04-03 at 12:36 -0700, Ethan Merritt wrote:

> > 
> > Does anyone know why, way back in the mists of time, gnuplot decided to 
> > define its own epoch boundary as 1-jan-2000 rather than use the conventional
> > unix epoch boundary 1-jan-1970? 
> >   
> > Did it have something to do with fitting an integer number of seconds 
> > in a 32-bit, or even 16-bit, integer? 
> > 
> > How many things would break if we did away with this 30 year disagreement?
> > 
> > 	Ethan
> > 
> > 
> 
> Currently this disagreement manifests itself even in the user interface:
> the meaning of "%s" is different, depending on whether it was used as an
> input or output spec.
> 
> Doing away with it might break a few scripts that depend on the epoch
> being 2000.0, but on the plus side it would make everything clearer and
> more consistent - both the internal implementation and the user
> documentation, not to mention that it would also be consistent with the
> rest of the (Unix) world.
> 
> I might add that 4.6.0 would have been a good place to make this change.
>
> Peter

Ah, but the major version change 4->5 would be an even better time.
I will add it to my "Things to break in 5.0" list [*].

My current thinking is that a 4.7 snapshot could be put out as a 
release candidate for 5.0 late this year.

	Ethan

[*] "break" in this case means "make no guarantee of backward compatibility"

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Wednesday, April 03, 2013 11:38:27 am Tait wrote:
> 
> Consider these two examples. First:
>   reset
>   set datafile separator ","
>   set xdata time
>   set timefmt "%s"
>   set format x "%Y-%m-%d %H:%M:%S"
>   set view 47,54
>   splot '-' using 1:2:3 with impulses
>   1364987000,50,19
>   1364987000,100,1
>   1364987000,150,0
> 
>   1364990600,50,15
>   1364990600,100,4
>   1364990600,150,1
> 
>   1364942000,50,9
>   1364942000,100,5
>   1364942000,150,7
> 
>   1365001400,50,11
>   1365001400,100,7
>   1365001400,150,2
>   e
> 
> ... and second:
>   reset
>   set datafile separator ","
>   set xdata time
>   set timefmt "%s"
>   set format x "%Y-%m-%d %H:%M:%S"
>   set view 47,54
>   splot '-' nonuniform matrix using 2:1:3 with impulses
>   0,50.00,100.00,150.00
>   1364987000,19,1,0
>   1364990600,15,4,1
>   1364942000,9,5,7
>   1365001400,11,7,2
>   e
>   e
> 
> (I don't understand why the second example requires two 'e's either,
> but I digress...)
> 
> In the first case, the year is correctly set to 2013. In the second,
> it is displayed as 2043. This seems to be a mistake in the nonuniform
> matrix time input?

This probably fixes it, but the possible consequences elsewhere are not
certain.  At a minimum it makes all references to the year 2000 in the
manual incorrect, since it resets the epoch boundary to 1970.

	Ethan


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Petr M. <mi...@ph...>]

>> In the first case, the year is correctly set to 2013. In the second,
>> it is displayed as 2043. This seems to be a mistake in the nonuniform
>> matrix time input?
>
> This probably fixes it, but the possible consequences elsewhere are not
> certain.  At a minimum it makes all references to the year 2000 in the
> manual incorrect, since it resets the epoch boundary to 1970.

What about adding
 	set datafile epoch 2000
so that the year can be set by the user?

---
PM

Re: 30-year time offset reading nonuniform matrix %s input

[Juhász P. <pet...@gm...>]

On Thu, 2013-04-04 at 17:08 +0200, Petr Mikulik wrote:
> >> In the first case, the year is correctly set to 2013. In the second,
> >> it is displayed as 2043. This seems to be a mistake in the nonuniform
> >> matrix time input?
> >
> > This probably fixes it, but the possible consequences elsewhere are not
> > certain.  At a minimum it makes all references to the year 2000 in the
> > manual incorrect, since it resets the epoch boundary to 1970.
> 
> What about adding
>  	set datafile epoch 2000
> so that the year can be set by the user?
> 
> ---
> PM

While I can imagine some specific use cases where this would not only
make sense, but would be actually useful (e.g. for an astronomical time
series plot one might want to set the epoch to MJD 0), in general I
think the added complexity and confusion makes such a feature not worth
it.

Peter

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Thursday, April 04, 2013 08:08:08 am Petr Mikulik wrote:
> Tait wrote:
> >> In the first case, the year is correctly set to 2013. In the second,
> >> it is displayed as 2043. This seems to be a mistake in the nonuniform
> >> matrix time input?
> 
> Ethan wrote:
> > This probably fixes it, but the possible consequences elsewhere are not
> > certain.  At a minimum it makes all references to the year 2000 in the
> > manual incorrect, since it resets the epoch boundary to 1970.

Petr wrote:
> What about adding
>  	set datafile epoch 2000
> so that the year can be set by the user?

The epoch affects time calculation throughout the program, so tying 
it to "set datafile" would be misleading.  For example
gnuplot> print time(0)
prints number of seconds since the epoch boundary without any 
data file being accessed.

Or did you mean that we should add a per-datafile option that
would temporarily override the program's default epoch only 
for the purpose of reading that one file?  That would be
possible, but I am not sure that it solves any real-world problem.
If the data file contains time represented as a number of raw seconds,
which is I think the only case this would affect, then you can
already read it in as a plain number and convert to time/date using 
whatever epoch adjustment you like.  Right?  Or maybe not? I don't
normally use the time format routines so I am kind of hazy on what
they can or can't do.

I'd still like to know what the origianl rationale was for 
choosing 2000 rather than 1970.  I know plenty of problems that
this choice caused, but what problems did it solve?
Are they still relevant?

	Ethan

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Daniel J S. <dan...@ie...>]

On 04/04/2013 01:38 PM, Ethan Merritt wrote:

> I'd still like to know what the origianl rationale was for
> choosing 2000 rather than 1970.  I know plenty of problems that
> this choice caused, but what problems did it solve?
> Are they still relevant?

The threat of planetary collapse due to the millennium time change?

Dan

Re: 30-year time offset reading nonuniform matrix %s input

[Dave D. <drd...@gm...>]

Ethan Merritt <merritt@u.washington.edu> writes:

>
> I'd still like to know what the origianl rationale was for 
> choosing 2000 rather than 1970.  I know plenty of problems that
> this choice caused, but what problems did it solve?
> Are they still relevant?
>

Sorry, I think it was probably me.  I didn't use unix back then,
so 2000 seemed as arbitrary as 1970.

Originally it wasn't intended that it be visible to the user anyway,
but it was discovered later that you could use numbers rather than strings
in things like setting ranges for time data.

I think I also had in mind that, because time was stored a real number,
it wasn't good to carry a large offset around since that limited the
resolution available for small numbers. But I think that's probably not
relevant in practise.

Dave D

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Friday, April 05, 2013 08:46:42 am Dave Denholm wrote:
> Ethan Merritt <merritt@u.washington.edu> writes:
> 
> > I'd still like to know what the origianl rationale was for 
> > choosing 2000 rather than 1970.  I know plenty of problems that
> > this choice caused, but what problems did it solve?
> > Are they still relevant?
> >
> 
> Sorry, I think it was probably me.  I didn't use unix back then,
> so 2000 seemed as arbitrary as 1970.
> 
> Originally it wasn't intended that it be visible to the user anyway,
> but it was discovered later that you could use numbers rather than strings
> in things like setting ranges for time data.
> 
> I think I also had in mind that, because time was stored a real number,
> it wasn't good to carry a large offset around since that limited the
> resolution available for small numbers. But I think that's probably not
> relevant in practise.
>
> Dave D

Thank you for insight into the history.
I'll go ahead and change it in CVS for the development version, 
with comments attached in the relevant places.  
Reversion, if necessary, would be trivial.

We know this fixes several problems, including the one that kicked
off the current thread of discussion.  The only down side I know of
is that the javascript scripts used for mousing svg and HTML5 plots
must now match the version of gnuplot used to generate the plot.
This was probably true already for other reasons, but now a mismatch
would be evident when tracking time coordinates as well.


	Ethan


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742