#185 how to ignore blank lines in datafile

[JP]

I have a 40MB datafile that contains xyz pixel values, one pixel per line, and each physical pixel row of values in the file separated by two blank lines, e.g., last pixels of row 99 to first pixels of row 100 looks like this:

...
99 1867 0.4516
99 1868 0.4285
99 1869 0.4259

100 0 0.0011
100 1 0.0056
100 2 0.0029
...

I am able to create 2d and 3d spatial plots of the pixel data despite the double blank lines. But for plotting histogram I realized that two consecutive blank lines indicates a new data set, which I don't want. I cannot recreate this datafile from scratch and I don't have a tool to remove these blanks.
How can I tell histogram plot to ignore blank lines and treat this file as one data set?
Thanks.
JP

[Ethan Merritt]

Any particular reason you can't do

plot "< grep -v '^$' datafile.dat"

[JP]

Hi Ethan,
I don't follow the language used, is this a Linux-type command typed into the gnuplot console?

I tried putting the string in place of my datafile name in my .plt script file and it gave an error:

gnuplot> plot [0:1][0:*] [0:1][0:numpix] "< grep -v '^$' datafile.csv" using (bin(column(3))):(1.0) smooth frequency title 'frequency' with boxes, '' using 3:(1.0) smooth cumulative title 'cumulative' axes x1y2
^
cannot create pipe for data
"datafile.plt", line 106:

Can you clarify where to use this grep syntax?

btw, I know my plot command works, thanks to your help, because a proper histogram is generated using whatever rows I remove the double blank lines between.

[Ethan Merritt]

"grep" is a standard extremely useful system utility

http://en.wikipedia.org/wiki/Grep

You should learn it.

I don't know why you got an error message, although I note as I did before that I strongly recommend not placing ranges at the front of a plot command.

[JP]

I knew somebody who used grep a lot but not being a Unix guy myself I never learned it. I found the equiv Windows version to run from a command prompt:

findstr /v "^$" datafile.csv > datafileout.csv

So now my plot program runs fine. Consider grep learned, and consider this closed.
Thanks.

[JP]

You may close the other two open cases under user my username jp2112, they
were solved.

Thanks.

JP

From: Ethan Merritt [mailto:sfeam@users.sf.net]
Sent: Monday, April 20, 2015 3:46 PM
To: [gnuplot:support-requests]
Subject: [gnuplot:support-requests] #185 how to ignore blank lines in
datafile

• status: open --> closed
• Group: -->

---

[support-requests:#185]
http://sourceforge.net/p/gnuplot/support-requests/185 how to ignore blank
lines in datafile

Status: closed
Group:
Labels: datafile
Created: Mon Apr 20, 2015 07:38 PM UTC by JP
Last Updated: Mon Apr 20, 2015 10:43 PM UTC
Owner: nobody

I have a 40MB datafile that contains xyz pixel values, one pixel per line,
and each physical pixel row of values in the file separated by two blank
lines, e.g., last pixels of row 99 to first pixels of row 100 looks like
this:

...
99 1867 0.4516
99 1868 0.4285
99 1869 0.4259

100 0 0.0011
100 1 0.0056
100 2 0.0029
...

I am able to create 2d and 3d spatial plots of the pixel data despite the
double blank lines. But for plotting histogram I realized that two
consecutive blank lines indicates a new data set, which I don't want. I
cannot recreate this datafile from scratch and I don't have a tool to remove
these blanks.
How can I tell histogram plot to ignore blank lines and treat this file as
one data set?
Thanks.
JP

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/gnuplot/support-requests/185/
https://sourceforge.net/p/gnuplot/support-requests/185

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
https://sourceforge.net/auth/subscriptions