Re: [Gnuplot-info] columns with time format as "y"

[t.sefzick <t.s...@gm...>]

The date format takes the columns of the input data which you
specify with the "using" option.

For data likes these

,2019-04-16,20:30:00,30522,5
,2019-04-16,20:31:00,30523,6

and the comma as separator, after the data have been read in
by gnuplot the 1st field is empty, the date is in the 2nd one:

<empty> 2019-04-16 20:30:00 30522 5
<empty> 2019-04-16 20:31:00 30523 6

So the plot command will use column 2 as x and e.g.
column 4 as y:

set timefmt "%Y-%m-%d,%H:%M:%S"
plot "datafile.dat" using 2:4

On Tue, Apr 21, 2020 at 08:33:03PM +0000, John L. Males wrote:
>
> Hello,
>
> I have had the data separator set to comma for few years as the
> data is a CSV file.
>
> The issue is the leading comma.  It appears that the date
> format assumes the time information is in the first two fields
> of the data.  Ergo the leading comma of the data has caused
> error messages to effect of invalid time from gnuplot which is
> why the data is pre-processed to remove the leading comma of
> the data before processed by gnuplot.
>
> The issue is not if I can plot the data of a CSV file.  I have
> been plotting CSV files for a number of years.  I have scripts
> that run every 15 minutes creating over 20 plots from 5
> different CSV files for number of years.  I like to eliminate
> the extra external step that processes the CSV data to remove
> the leading comma in the data.  The data that is very large
> really needs to be refreshed every 5 minutes, but cannot as the
> data has to be passed over twice to plot the data with
> gnuplot.  If I did not have to have the external step to
> remove the leading comma in the data then this very large data
> could be plotted every 5 minutes.  The smaller CSV files also
> would plot much faster be eliminating the extra step to remove
> the leading comma before gnuplot can process the CSV data.
>
>
> John L. Males
> Toronto, Ontario
> Canada
> 21 April 2020 16:33 -0400 EDT
>
>
> Message replied to:
>
> Date: Mon, 20 Apr 2020 22:17:35 +0200
> From: "t.sefzick" <t.s...@gm...>
> To: "John L. Males" <jl...@gm...>
> Cc: eam...@gm..., gnu...@li...,
> t.s...@gm... Subject: Re: [Gnuplot-info] columns with time
> format as "y"
>
>
> > The separator of your data is the comma, so
> >
> > set datafile separator comma
> >
> > splits your data into
> >
> > <empty> date time value1 value2
> >
> > Because of the empty first column you need to start
> > with column 2:
> >
> > set timefmt "%Y-%m-%d,%H:%M:%S"
> > plot "datafile.dat" using 2:4
> >
> > On Mon, Apr 20, 2020 at 08:00:02PM +0000, John L. Males wrote:
> > >
> > > Hello,
> > >
> > > As follow on to the OP question I have data format of the
> > > form:
> > >
> > > ,2019-04-16,20:30:00,30522,5
> > > ,2019-04-16,20:31:00,30523,6
> > > ,2019-04-16,20:32:00,30524,7
> > > ,2019-04-16,20:33:00,30525,8
> > > ,2019-04-16,20:34:00,30526,9
> > > ,2019-04-16,20:35:00,30527,10
> > > ,2019-04-16,20:36:00,30528,11
> > > ,2019-04-16,20:37:00,30529,12
> > > ,2019-04-16,20:38:00,30530,13
> > > ,2019-04-16,20:39:00,30531,14
> > >
> > > Yes, the data lines start with "," for reasons I will skip
> > > other than to say this is the data by design.
> > >
> > > I had tried a:
> > >
> > > set timefmt ",%Y-%m-%d,%H:%M:%S"
> > >
> > > a few years ago and again a few months ago with no success.
> > > Just an error message to effect of invalid time format in
> > > data.  I have had to add extra external step to filter the
> > > data.  The issue is the data is very large so it adds a
> > > couple minutes to filter the data before I can input the
> > > date to gnuplot.  I really would like to eliminate the data
> > > filter step that eliminates the leading comma of the data
> > > as this data is processed every 15 minutes for now as I
> > > cannot filter the data every 5 minutes due to the added
> > > filtering step that then uses:
> > >
> > > set timefmt "%Y-%m-%d,%H:%M:%S"
> > >
> > >
> > > John L. Males
> > > Toronto, Ontario
> > > Canada
> > > 20 April 2020 16:00 -0400 EDT
> > >
> > >
> > > Message replied to:
> > >
> > > Date: Mon, 20 Apr 2020 12:00:40 -0700
> > > From: Ethan Merritt <eam...@gm...>
> > > To: gnu...@li...
> > > Subject: Re: [Gnuplot-info] columns with time format as "y"
> > >
> > >
> > > > On Monday, 20 April 2020 11:30:03 PDT co...@tr...
> > > > wrote:
> > > > >
> > > > > Hello All,
> > > > >
> > > > > I need help with the following problem:
> > > > >
> > > > > Let's say my input data look like (columns delimited by
> > > > > \t):
> > > > >
> > > > >    2019-04-16 20:30:00	30522	5
> > > > >    2019-04-16 20:31:00	30523	6
> > > > >    2019-04-16 20:32:00	30524	7
> > > > >    2019-04-16 20:33:00	30525	8
> > > > >    2019-04-16 20:34:00	30526	9
> > > > >    2019-04-16 20:35:00	30527	10
> > > > >    2019-04-16 20:36:00	30528	11
> > > > >    2019-04-16 20:37:00	30529	12
> > > > >    2019-04-16 20:38:00	30530	13
> > > > >    2019-04-16 20:39:00	30531	14
> > > > >
> > > > > Column 1 & 2 (space separated) considered as date/time.
> > > > >
> > > > > Plotting Col. 3 or 4 against date/time works fine:
> > > > >
> > > > >    set xdata time
> > > > >    set timefmt "%Y-%m-%d\t%H:%M:%S"
> > > > >    set format x "%d.%m. %H:%M"
> > > > >    plot 'my_file.txt' using 1:3 title "col 3" axis x1y1
> > > > > with linespoints pt7 lc 4, '' using 1:4 title "col 4"
> > > > > axis x1y1 with linespoints pt7 lc 7
> > > > >
> > > > > Special feature: Although columns 1 and 2 *together*
> > > > > give the timestamp, only "1:..." is given for "using"
> > > > > statement. This method is often found in examples.
> > > > >
> > > > > But now I would like - don't ask me why - to plot the
> > > > > timestamp against column 3, i.e. column 3 as "x" and the
> > > > > timestamp (columns 1 & 2) as "y".
> > > > >
> > > > > I was trying:
> > > > >
> > > > >    set ydata time "%Y-%m-%d\t%H:%M:%S"
> > > > >    set timefmt "%Y-%m-%d\t%H:%M:%S"
> > > > >    set format y "%d.%m. %H:%M"
> > > > >    plot 'my_file.txt' using 3:1 title "col 3" axis x1y1
> > > > > with linespoints pt7 lc 4
> > > > >
> > > > > Result: only date is used for plotting "y", time
> > > > > ignored!
> > > >
> > > > Replace the tab with a space in timefmt:
> > > >
> > > >     set timefmt "%Y-%m-%d %H:%M:%S"
> > > >
> > > > The space character in the format will match any
> > > > whitespace sequence, including <tab>.
> > > >
> > > > 	Ethan