Re: [Gnuplot-info] columns with time format as "y"

[Ethan M. <eam...@gm...>]

On Tuesday, 21 April 2020 13:33:03 PDT John L. Males wrote:
> Hello,
> 
> I have had the data separator set to comma for few years as the
> data is a CSV file.
> 
> The issue is the leading comma.  It appears that the date
> format assumes the time information is in the first two fields
> of the data.  

That has never been true.  Aside from that, the introduction of
explicit time conversion functions strptime(), strftime(), and 
timecolumn() decoupled the input (datafile columns) and output
(plot axes).

> Ergo the leading comma of the data has caused
> error messages to effect of invalid time from gnuplot which is
> why the data is pre-processed to remove the leading comma of
> the data before processed by gnuplot.

Could you provide an example of this?
I don't understand how you could get an error message about a
leading comma when you have told gnuplot that it is a comma-separated
csv file.  Each field being parsed starts by definition _after_ the field
seperator, so it should not be possible for it to see a leading comma.
At worst you should be looking at a column too far to the right because
an extra comma caused it to skip ahead more than intended.


> The issue is not if I can plot the data of a CSV file.  

This works here on your snippet of csv data.  Season to taste.

    set datafile sep comma
    total_time(dummy) = strptime("%Y-%m-%d",strcol(2)) + strptime("%H:%M:%S", strcol(3))
    plot 'data' using 4:(total_time(0))

If you want time format labels along x, add:

    set xtics time format "%H:%M:%S"

and similarly for y

	Ethan

> John L. Males
> Toronto, Ontario
> Canada
> 21 April 2020 16:33 -0400 EDT
> 
> 

> 
> Message replied to:
> 
> Date: Mon, 20 Apr 2020 22:17:35 +0200
> From: "t.sefzick" <t.s...@gm...>
> To: "John L. Males" <jl...@gm...>
> Cc: eam...@gm..., gnu...@li...,
> t.s...@gm... Subject: Re: [Gnuplot-info] columns with time
> format as "y"
> 
> 
> > The separator of your data is the comma, so
> >
> > set datafile separator comma
> >
> > splits your data into
> >
> > <empty> date time value1 value2
> >
> > Because of the empty first column you need to start
> > with column 2:
> >
> > set timefmt "%Y-%m-%d,%H:%M:%S"
> > plot "datafile.dat" using 2:4
> >
> > On Mon, Apr 20, 2020 at 08:00:02PM +0000, John L. Males wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > Hello,
> > >
> > > As follow on to the OP question I have data format of the
> > > form:
> > >
> > > ,2019-04-16,20:30:00,30522,5
> > > ,2019-04-16,20:31:00,30523,6
> > > ,2019-04-16,20:32:00,30524,7
> > > ,2019-04-16,20:33:00,30525,8
> > > ,2019-04-16,20:34:00,30526,9
> > > ,2019-04-16,20:35:00,30527,10
> > > ,2019-04-16,20:36:00,30528,11
> > > ,2019-04-16,20:37:00,30529,12
> > > ,2019-04-16,20:38:00,30530,13
> > > ,2019-04-16,20:39:00,30531,14
> > >
> > > Yes, the data lines start with "," for reasons I will skip
> > > other than to say this is the data by design.
> > >
> > > I had tried a:
> > >
> > > set timefmt ",%Y-%m-%d,%H:%M:%S"
> > >
> > > a few years ago and again a few months ago with no success.
> > > Just an error message to effect of invalid time format in
> > > data.  I have had to add extra external step to filter the
> > > data.  The issue is the data is very large so it adds a
> > > couple minutes to filter the data before I can input the
> > > date to gnuplot.  I really would like to eliminate the data
> > > filter step that eliminates the leading comma of the data
> > > as this data is processed every 15 minutes for now as I
> > > cannot filter the data every 5 minutes due to the added
> > > filtering step that then uses:
> > >
> > > set timefmt "%Y-%m-%d,%H:%M:%S"
> > >
> > >
> > > John L. Males
> > > Toronto, Ontario
> > > Canada
> > > 20 April 2020 16:00 -0400 EDT
> > >

> > > Message replied to:
> > >
> > > Date: Mon, 20 Apr 2020 12:00:40 -0700
> > > From: Ethan Merritt <eam...@gm...>
> > > To: gnu...@li...
> > > Subject: Re: [Gnuplot-info] columns with time format as "y"
> > >
> > >
> > > > On Monday, 20 April 2020 11:30:03 PDT co...@tr...
> > > > wrote:
> > > > >
> > > > > Hello All,
> > > > >
> > > > > I need help with the following problem:
> > > > >
> > > > > Let's say my input data look like (columns delimited by
> > > > > \t):
> > > > >
> > > > >    2019-04-16 20:30:00	30522	5
> > > > >    2019-04-16 20:31:00	30523	6
> > > > >    2019-04-16 20:32:00	30524	7
> > > > >    2019-04-16 20:33:00	30525	8
> > > > >    2019-04-16 20:34:00	30526	9
> > > > >    2019-04-16 20:35:00	30527	10
> > > > >    2019-04-16 20:36:00	30528	11
> > > > >    2019-04-16 20:37:00	30529	12
> > > > >    2019-04-16 20:38:00	30530	13
> > > > >    2019-04-16 20:39:00	30531	14
> > > > >
> > > > > Column 1 & 2 (space separated) considered as date/time.
> > > > >
> > > > > Plotting Col. 3 or 4 against date/time works fine:
> > > > >
> > > > >    set xdata time
> > > > >    set timefmt "%Y-%m-%d\t%H:%M:%S"
> > > > >    set format x "%d.%m. %H:%M"
> > > > >    plot 'my_file.txt' using 1:3 title "col 3" axis x1y1
> > > > > with linespoints pt7 lc 4, '' using 1:4 title "col 4"
> > > > > axis x1y1 with linespoints pt7 lc 7
> > > > >
> > > > > Special feature: Although columns 1 and 2 *together*
> > > > > give the timestamp, only "1:..." is given for "using"
> > > > > statement. This method is often found in examples.
> > > > >
> > > > > But now I would like - don't ask me why - to plot the
> > > > > timestamp against column 3, i.e. column 3 as "x" and the
> > > > > timestamp (columns 1 & 2) as "y".
> > > > >
> > > > > I was trying:
> > > > >
> > > > >    set ydata time "%Y-%m-%d\t%H:%M:%S"
> > > > >    set timefmt "%Y-%m-%d\t%H:%M:%S"
> > > > >    set format y "%d.%m. %H:%M"
> > > > >    plot 'my_file.txt' using 3:1 title "col 3" axis x1y1
> > > > > with linespoints pt7 lc 4
> > > > >
> > > > > Result: only date is used for plotting "y", time
> > > > > ignored!
> > > >
> > > > Replace the tab with a space in timefmt:
> > > >
> > > >     set timefmt "%Y-%m-%d %H:%M:%S"
> > > >
> > > > The space character in the format will match any
> > > > whitespace sequence, including <tab>.
> > > >
> > > > 	Ethan