Menu
▾
▴
gnuplot-beta
|
From: Petr M. <mi...@ph...> - 2006-03-02 22:10:56
Attachments:
fileexist.diff
|
What do you think about adding function
fileexist("a.dat")
that would return 1 or 0 if file exists or does not exist?
I think there is a good chance this name does not clash with most user
variables.
I have already needed that several times ... solution via `test` is not
portable ...
The code is as easy as
return (access(filename, F_OK) != 0))
see the attachment.
---
PM |
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-03 01:32:44
|
On Thursday 02 March 2006 02:10 pm, Petr Mikulik wrote:
> What do you think about adding function
> fileexist("a.dat")
> that would return 1 or 0 if file exists or does not exist?
>
> I have already needed that several times ... solution via `test` is
> not portable ...
Is access() really that much more portable than test?
> The code is as easy as
> return (access(filename, F_OK) != 0))
> see the attachment.
man 2 access
[...]
RESTRICTIONS
access may not work correctly on NFS file systems with UID
mapping enabled, because UID mapping is done on the server
and hidden from the client, which checks permissions.
If your test doesn't work across NFS-mounted volumes, that's
not very portable!
Can you give an example of why such a thing is needed?
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-03 10:25:18
|
>> What do you think about adding function
>> fileexist("a.dat")
>> that would return 1 or 0 if file exists or does not exist?
>>
>> I have already needed that several times ... solution via `test` is
>> not portable ...
>
> Is access() really that much more portable than test?
>
>> The code is as easy as
>> return (access(filename, F_OK) != 0))
>> see the attachment.
> That's not terribly portable, then. Access() is POSIX, but that doesn't
> help us much on Windows, OS/2 and others.
Then OS/2 is OK, also GNU compilers under windows (mingw, cygwin). Cannot
say about others ... #ifdef HAVE_UNISTD_H should suffice.
> If we want to be portable, the only real option is to ry to fopen() the
> file and see if it works.
> man 2 access
> [...]
> RESTRICTIONS
> access may not work correctly on NFS file systems with UID
> mapping enabled, because UID mapping is done on the server
> and hidden from the client, which checks permissions.
>
> If your test doesn't work across NFS-mounted volumes, that's
> not very portable!
So you recommend to use fopen() even for unices, and thus get rid of
access()? No problem, I will change the patch.
> Can you give an example of why such a thing is needed?
Example 1:
plot 'measure.dat'
if fileexist('fit.dat') replot 'fit.dat'
pause 5
reread
Example 2:
Execution of commands passed via pipes between gnuplot and octave (for
example) takes some time and it's hard to stop/wait without breaking the
script (error message). The above solution is fine for some cases.
Actually, what some users ask for real-time plotting, is something like
pause untilfileexist "next.dat"
pause untilfilechanged "next.dat"
---
PM
|
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-03 17:42:28
|
On Friday 03 March 2006 02:25 am, Petr Mikulik wrote:
> > Can you give an example of why such a thing is needed?
>
> Example 1:
> plot 'measure.dat'
> if fileexist('fit.dat') replot 'fit.dat'
> pause 5
> reread
So far as I know, with the current cvs version you might
just as well say
plot 'measure.dat','fit.dat'
pause 5
reread
A missing data file should no longer prevents the plot from being
constructed.
> Example 2:
> Execution of commands passed via pipes between gnuplot and octave
> (for example) takes some time and it's hard to stop/wait without
> breaking the script (error message). The above solution is fine for
> some cases.
I do not follow you there at all.
Neither your `test` nor your access() or fopen() will work on a pipe.
> Actually, what some users ask for real-time plotting, is something
> like
> pause untilfilechanged "next.dat"
I doubt there is a universally portable solution to this one.
It is difficult even if you restrict the solution to linux/unix.
The best I can think of without writing an external helper that
monitors the file is something like:
ready = words(system("find . -name 'next.dat' -newer 'lock'"))
if (ready) plot "next.dat"
if (ready) system("touch lock")
pause 1
reread
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-03 18:05:35
|
>> Example 1:
>> plot 'measure.dat'
>> if fileexist('fit.dat') replot 'fit.dat'
>> pause 5
>> reread
>
> So far as I know, with the current cvs version you might
> just as well say
> plot 'measure.dat','fit.dat'
> pause 5
> reread
>
> A missing data file should no longer prevents the plot from being
> constructed.
No:
gnuplot> plot 'a.dat', 'bla'
^
can't read data file "bla"
util.c: No such file or directory
>> Example 2:
>
> I do not follow you there at all.
It was not a good example, sorry.
So, I'll commit the "fileexist" patch which uses fopen().
>> Actually, what some users ask for real-time plotting, is something
>> like
>> pause untilfileexist "next.dat"
>> pause untilfilechanged "next.dat"
>
> I doubt there is a universally portable solution to this one.
> It is difficult even if you restrict the solution to linux/unix.
Why? What about "select" and "stat" functions? If select() is not available,
a poor-man solution is to check the status with one-second resolution.
(Well, I never used select() but I think it provides the waiting.)
---
PM
|
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-03 18:29:49
|
On Friday 03 March 2006 10:05 am, Petr Mikulik wrote:
> > A missing data file should no longer prevents the plot from being
> > constructed.
>
> No:
>
> gnuplot> plot 'a.dat', 'bla'
> ^
> can't read data file "bla"
> util.c: No such file or directory
OK. I'll try to fix that.
I guess it's only the "file exists but contains no data"
condition that is now acceptable.
> So, I'll commit the "fileexist" patch which uses fopen().
Can we discuss this some more first?
Or at least let me try out the patch first?
I'm not 100% convinced it will always work, even on linux.
> >> Actually, what some users ask for real-time plotting, is something
> >> like
> >> pause untilfileexist "next.dat"
> >> pause untilfilechanged "next.dat"
> >
> > I doubt there is a universally portable solution to this one.
> > It is difficult even if you restrict the solution to linux/unix.
>
> Why? What about "select" and "stat" functions?
I don't think these do what you want. "select" will tell you if
there is more data available to read on an open file. But you are
asking for a test that the file exists in the first place, which is an
entirely different question.
"stat" is essentially the same as "test", if you mean the command
line version. If you mean the library call "stat", I really have no
idea whether it is available outside of the usual linux/unix/POSIX
world. And even if the routine itself is available, I don't know if
the interpretation of the subfields describing modification time
is universal. You would have to do a lot of bookkeeping to
implement a gnuplot routine that essential did
if (file_has_changed_since_the_last_time_I_asked("filename"))
Could you track multiple file? What if the file no longer exists?
It seems to me easier to use system("find . -name <foo> -newer <bar>")
and let the system itself do all the bookkeeping. The "find" command
is in POSIX, so you get exactly the same level of guarantee that you
would for building in a call to "stat", with the advantage that if your
system doesn't follow POSIX you could fix it in your script file rather
than having to edit and rebuild from source.
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-03 18:44:59
Attachments:
fileexist_02.diff
|
> Can we discuss this some more first?
> Or at least let me try out the patch first?
> I'm not 100% convinced it will always work, even on linux.
It is enclosed, please try it. I think it must work everywhere where fopen()
is implemented. (And gnuplot without fopen() cannot live.)
>>>> Actually, what some users ask for real-time plotting, is something
>>>> like
>>>> pause untilfileexist "next.dat"
>>>> pause untilfilechanged "next.dat"
>
> It seems to me easier to use system("find . -name <foo> -newer <bar>")
This is not portable either.
Actually, the most trivial solution seems to be:
prev=`ls -l bla.dat` or prev=`dir bla.dat`
if (!variable_exist(new)) new=prev # 1st time here ??????????????
if (pi > 3.0) pi=0; new=prev # 1st time here ??????????????
if (new == prev) pause 5; reread
plot 'bla.dat'
new = prev
reread
??? how to detect that a variable has not been initialized?
---
PM |
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-03 21:10:13
|
On Friday 03 March 2006 10:44 am, Petr Mikulik wrote:
> > It seems to me easier to use system("find . -name <foo> -newer
> > <bar>")
>
> This is not portable either.
Not universally. But it works without any changes to gnuplot.
Why add a new routine to gnuplot if it doesn't give any advantage
over what you can already do?
> Actually, the most trivial solution seems to be:
I think you meant:
if (!defined(prev)) prev = "" # first time only
new=system("ls -l bla.dat")
if (new ne prev) plot 'bla.dat'
prev = new
pause 5; reread
But this doesn't work very well.
`ls -l` will not tell whether new data has been written since your
previous plot. For one thing, the time is only given to the nearest
minute. I thought you wanted a real-time test, that would update on
the order of seconds? There is a gnu-specific extension:
`ls --full-time`, but this won't help you on general unix boxes and
it certainly won't help on VMS or other less unix-like systems.
Anyhow, is there anysystem that provides "ls" that does not also
provide "find"?
> ??? how to detect that a variable has not been initialized?
if (defined(VAR))
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-04 00:04:30
|
On Friday 03 March 2006 10:29 am, Ethan Merritt wrote:
> On Friday 03 March 2006 10:05 am, Petr Mikulik wrote:
> >
> > gnuplot> plot 'a.dat', 'bla'
> > ^
> > can't read data file "bla"
> > util.c: No such file or directory
Here's a simple patch to fix this.
It is directly parallel to the recently added test that skips
files containing no data points.
Please try it and comment.
I already know that it emits an extraneous error message
for a couple of special cases (e.g. after 'set xdata time'),
and that it doesn't handle missing files passed to "fit".
I think the 2D case is probably solid. I have not tested all
possible 3D plot modes, so it is possible that some additional
checks need to be added there.
Ethan
--- gnuplot/src/datafile.c 2006-01-27 08:36:48.000000000 -0800
+++ gnuplot-cvs/src/datafile.c 2006-03-03 15:19:13.000000000 -0800
@@ -1275,7 +1275,9 @@
if ((data_fp = loadpath_fopen(df_filename, df_binary ? "rb" : "r")) ==
#endif
(FILE *) NULL) {
- os_error(name_token, "can't read data file \"%s\"", df_filename);
+ /* os_error(name_token, "can't read data file \"%s\"", df_filename); */
+ int_warn(name_token, "Skipping unreadable file \"%s\"", df_filename);
+ return -1;
}
}
/*}}} */
--- gnuplot/src/plot2d.c 2005-11-28 11:02:56.000000000 -0800
+++ gnuplot-cvs/src/plot2d.c 2006-03-03 15:33:24.000000000 -0800
@@ -1787,6 +1787,11 @@
++line_num;
}
if (this_plot->plot_type == DATA) {
+ if (specs < 0) {
+ /* Error check to handle missing or unreadable file */
+ this_plot->plot_type = NODATA;
+ goto SKIPPED_EMPTY_FILE;
+ }
/* actually get the data now */
if (get_data(this_plot) == 0) {
/* EAM 2005 - warn, but keep going */
--- gnuplot/src/plot3d.c 2006-01-12 11:55:14.000000000 -0800
+++ gnuplot-cvs/src/plot3d.c 2006-03-03 15:32:54.000000000 -0800
@@ -1560,6 +1560,9 @@
struct surface_points *first_dataset = this_plot;
/* pointer to the plot of the first dataset (surface) in the file */
int this_token = this_plot->token;
+ /* Error check to handle unreadable or missing data file */
+ if (specs < 0)
+ df_eof = 1;
while (!df_eof) {
this_plot = *tp_3d_ptr;
assert(this_plot != NULL);
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-04 20:40:38
|
>>> gnuplot> plot 'a.dat', 'bla'
>>> ^
>>> can't read data file "bla"
>>> util.c: No such file or directory
>
> Here's a simple patch to fix this.
> Please try it and comment.
I like it, please commit it to cvs then ... with just some fix for:
> I think the 2D case is probably solid. I have not tested all
> possible 3D plot modes, so it is possible that some additional
> checks need to be added there.
I have found only this non-working ('bla' does not exist):
plot x, 'bla'
splot x*y, 'bla'
this gives no plot and:
gnuplot> plot x, 'bla'
^
warning: Skipping unreadable file "bla"
^
x range is invalid
I would prefer just the function x is plotted.
---
PM
|
|
From: Daniel J S. <dan...@ie...> - 2006-03-05 19:56:59
|
Ethan Merritt wrote:
Well, the patch Ethan created is in the right area of datafile.c. In order to have the
Can't open data file "xxx"
take precedence over
Unsupported file type
I'd say it means rearranging the options string interpretation code and the code that opens the file in "df_open". That is simply the way it has traditionally been, I guess. Check options first then attempt opening file.
An easy change, but there needs to be care with one thing. If the file is opened first, then checked for options, any error in the options will leave an open file hanging around.
An alternative might be to move just this chunk of code:
#ifdef HAVE_SYS_STAT_H
struct stat statbuf;
if ((stat(df_filename, &statbuf) > -1) &&
!S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
os_error(name_token, "\"%s\" is not a regular file or pipe",
df_filename);
}
#endif /* HAVE_SYS_STAT_H */
which I think sort of checks the existence of the file without really opening it. And replace it with
#ifdef HAVE_SYS_STAT_H
struct stat statbuf;
if ((stat(df_filename, &statbuf) > -1) &&
!S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
os_error(name_token, "\"%s\" is not a regular file or pipe",
df_filename);
}
#else
if (<open fails>)
error("Can't open data file "xxx"");
else
close_file();
#endif /* HAVE_SYS_STAT_H */
Would that kind of thing work? That would mean if there is no "stat()" then the file would be opened and closed for checking existence (also check for emptiness?), then after options are checked opened again for reading.
Or would you prefer opening and keeping track of the file pointer and doing a "close" before each instance of "error()"?
Dan
> --- gnuplot/src/datafile.c 2006-01-27 08:36:48.000000000 -0800
> +++ gnuplot-cvs/src/datafile.c 2006-03-03 15:19:13.000000000 -0800
> @@ -1275,7 +1275,9 @@
> if ((data_fp = loadpath_fopen(df_filename, df_binary ? "rb" : "r")) ==
> #endif
> (FILE *) NULL) {
> - os_error(name_token, "can't read data file \"%s\"", df_filename);
> + /* os_error(name_token, "can't read data file \"%s\"", df_filename); */
> + int_warn(name_token, "Skipping unreadable file \"%s\"", df_filename);
> + return -1;
> }
> }
> /*}}} */
> --- gnuplot/src/plot2d.c 2005-11-28 11:02:56.000000000 -0800
> +++ gnuplot-cvs/src/plot2d.c 2006-03-03 15:33:24.000000000 -0800
> @@ -1787,6 +1787,11 @@
> ++line_num;
> }
> if (this_plot->plot_type == DATA) {
> + if (specs < 0) {
> + /* Error check to handle missing or unreadable file */
> + this_plot->plot_type = NODATA;
> + goto SKIPPED_EMPTY_FILE;
> + }
> /* actually get the data now */
> if (get_data(this_plot) == 0) {
> /* EAM 2005 - warn, but keep going */
> --- gnuplot/src/plot3d.c 2006-01-12 11:55:14.000000000 -0800
> +++ gnuplot-cvs/src/plot3d.c 2006-03-03 15:32:54.000000000 -0800
> @@ -1560,6 +1560,9 @@
> struct surface_points *first_dataset = this_plot;
> /* pointer to the plot of the first dataset (surface) in the file */
> int this_token = this_plot->token;
> + /* Error check to handle unreadable or missing data file */
> + if (specs < 0)
> + df_eof = 1;
> while (!df_eof) {
> this_plot = *tp_3d_ptr;
> assert(this_plot != NULL);
>
>
--
Dan Sebald
phone: 608 256 7718
email: daniel DOT sebald AT ieee DOT org
URL: http://webpages DOT charter DOT net/dsebald/
|
|
From: Daniel J S. <dan...@ie...> - 2006-03-03 18:47:15
|
Petr Mikulik wrote:
>>> Example 1:
>>> plot 'measure.dat'
>>> if fileexist('fit.dat') replot 'fit.dat'
Not so sure about the syntax "fileexist".
>>> Actually, what some users ask for real-time plotting, is something
>>> like
>>> pause untilfileexist "next.dat"
>>> pause untilfilechanged "next.dat"
When a file is open by another program and is being written to, it appears in the directory. So wouldn't one have to wait until the file is closed to be certain that reading it will give you all the data meant for plotting?
Dan
|
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-03 22:35:01
|
On Friday 03 March 2006 10:44 am, Petr Mikulik wrote:
> > Can we discuss this some more first?
> > Or at least let me try out the patch first?
> > I'm not 100% convinced it will always work, even on linux.
>
> It is enclosed, please try it. I think it must work everywhere where
> fopen() is implemented. (And gnuplot without fopen() cannot live.)
I think this function could use some more polishing to remove
rough edges.
Here's one scenario which could cause much confusion.
The new function does not use the same path to look for
files that is used by the actual plot command:
gnuplot> !printenv GNUPLOT_LIB
../demo
gnuplot> print (fileexist("../demo/silver.dat")) ? "OK" : "File not
found"
OK
gnuplot> print (fileexist("silver.dat")) ? "OK" : "File not found"
File not found
gnuplot> plot "silver.dat"
[plots normally even though fileexist didn't find it]
Here's another. The new function does not recognize pipes
gnuplot> DATAFILE = "< head -40 ../demo/silver.dat"
gnuplot> plot DATAFILE
gnuplot> print (fileexist(DATAFILE)) ? "OK" : "File not found"
File not found
I agree that no one is likely to type such a sequence of commands
from the command line. But I take it that the purpose of the new
command would be to place in an automated script, so that the
script can handle non-existence of a data file cleanly. But then
it is much harder for the caller to understand why these example
do not work as expected. For example, the user might think it
was a clever idea to use GNUPLOT_LIB to point to any one of
several data directories:
setenv GNUPLOT_LIB /jan2006/data
gnuplot script.gp
setenv GNUPLOT_LIB /feb2006/data
gnuplot script.gp
Where script.gp also tries to be clever, but in an incompatible way:
# Loop over data files and plot each one we find
if (!defined(plotno)) plotno = 1
file = sprintf("datafile.%d",plotno)
if (!fileexist(file)) exit
plot file
plotno = plotno+1
reread
The pipe failure could also arise from a plausible attempt to
automate a sequence of plots that involve pre-processing the
data.
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-04 22:13:29
|
> Here's another. The new function does not recognize pipes
The routine fileexist() is for files only; it makes no sense for pipes.
For pipes, do
if (fileexist('a.dat')) plot '<tr . , a.dat'
> gnuplot> !printenv GNUPLOT_LIB
It looks for the file in the current directory by default.
> was a clever idea to use GNUPLOT_LIB to point to any one of
> several data directories:
Then, there is a need for function (taking the name from Octave)
file_in_loadpath(name)
We can add it.
=> return the absolute path/name for the given file anywhere along the
loadpath.
Then, you could use either of
if (fileexist(f))
if (fileexist(file_in_loadpath(f)))
> Anyhow, is there anysystem that provides "ls" that does not also
> provide "find"?
On OS/2 and Windows
- there is no "ls" by default
- there is "find", but with completely different syntax from GNU find
> if (defined(VAR))
Thanks, that I was looking for (in Octave & Matlab, it's called exist()).
---
PM
|
|
From: Ethan A M. <merritt@u.washington.edu> - 2006-03-05 00:52:56
|
On Saturday 04 March 2006 12:45 pm, Petr Mikulik wrote: > Ah, these in addition: Unfortunately, I don't understand the codepath for reading binary files well enough to back out of these failures cleanly. At the time the message is printed, we are several routines deep in the binary file code. I think you would have to add error return codes to each layer of routines, and error-handling code at all the call sites. Maybe Daniel would be willing to take a look at it after I sort out the normal [ascii] input file case. It was his code originally, I think. > gnuplot> plot 'xxx' binary filetype=edf with image > Can't open data file "xxx" > util.c: No such file or directory > gnuplot> plot 'xxx' binary filetype=avs with image > Can't open data file "xxx" > util.c: No such file or directory > > gnuplot> plot 'demo.edfxxx' binary filetype=auto with image > ^ > Unsupported file type > > -- Ethan A Merritt Biomolecular Structure Center University of Washington, Seattle 98195-7742 |
|
From: Ethan A M. <merritt@u.washington.edu> - 2006-03-05 00:23:08
|
On Saturday 04 March 2006 02:13 pm, Petr Mikulik wrote:
> > Here's another. The new function does not recognize pipes
>
> The routine fileexist() is for files only; it makes no sense for pipes.
> For pipes, do
> if (fileexist('a.dat')) plot '<tr . , a.dat'
Sure. But that is only possible if you already know whether it is a
pipe or not. What about the case where you want to put the
fileexist() test in a loadable routine that is passed the filename
string as a parameter? I tried to show a plausible use for such
a thing in the previous mail. At the time you are writing the
loadable routine, you don't know whether the caller will pass you
a file or a pipe at some future time.
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742
|
|
From: Petr M. <mi...@ph...> - 2006-03-05 20:00:46
|
>> The routine fileexist() is for files only; it makes no sense for pipes.
>> For pipes, do
>> if (fileexist('a.dat')) plot '<tr . , a.dat'
>
> Sure. But that is only possible if you already know whether it is a
> pipe or not. What about the case where you want to put the
> fileexist() test in a loadable routine that is passed the filename
> string as a parameter?
Then it needs yet another routine pipeexist().
---
PM
|
|
From: Ethan A M. <merritt@u.washington.edu> - 2006-03-05 22:07:35
|
On Sunday 05 March 2006 12:00 pm, you wrote:
> >> The routine fileexist() is for files only; it makes no sense for pipes.
> >> For pipes, do
> >> if (fileexist('a.dat')) plot '<tr . , a.dat'
> >
> > Sure. But that is only possible if you already know whether it is a
> > pipe or not. What about the case where you want to put the
> > fileexist() test in a loadable routine that is passed the filename
> > string as a parameter?
>
> Then it needs yet another routine pipeexist().
I think you are making these tests way too specific.
Wouldn't it be better to duplicate the code that would actually
be used to open the file while plotting?
Trimmed from lines 1240-1280 of datafile.c:
=========================================================================
/* filename cannot be static array! */
gp_expand_tilde(&df_filename);
/*{{{ open file */
if (*df_filename == '<') {
if ((data_fp = popen(df_filename + 1, "r")) == (FILE *) NULL)
os_error(name_token, "cannot create pipe for data");
} else if (*df_filename == '-' && strlen(df_filename) == 1) {
plotted_data_from_stdin = TRUE;
} else {
struct stat statbuf;
if ((stat(df_filename, &statbuf) > -1) &&
!S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
os_error(name_token, "\"%s\" is not a regular file or pipe",
df_filename);
}
if (!(data_fp = loadpath_fopen(df_filename, df_binary ? "rb" : "r"))
os_error(name_token, "can't read data file \"%s\"", df_filename);
}
=========================================================================
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742
|
|
From: Petr M. <mi...@ph...> - 2006-03-07 22:44:28
Attachments:
fileexist_03.diff
|
> Then it needs yet another routine pipeexist().
So, now there are (patch enclosed) both
fileexist(s)
which detects regular files, and
pipeexist(s) which returns 1 for "-" and 1/0 whether it is a pipe
wherefrom you can read at least 1 byte; that's because:
> Trimmed from lines 1240-1280 of datafile.c:
> =========================================================================
> /* filename cannot be static array! */
> gp_expand_tilde(&df_filename);
This line makes sense only for regular files and thus it should be moved to
just above the "stat" function.
> /*{{{ open file */
> if (*df_filename == '<') {
> if ((data_fp = popen(df_filename + 1, "r")) == (FILE *) NULL)
> os_error(name_token, "cannot create pipe for data");
This will never work. The truth is that popen() "never" returns NULL
(contrary to specification) -- at least for OS/2, Windows, Linux. Or, have
seen the contrary on some system? It is necessary to read at least 1 B from
the opened stream, but if popen() fails, then you always see an offending
message by your system.
See this on Linux:
gnuplot> plot 'bla'
^
can't read data file "bla"
util.c: No such file or directory
gnuplot> plot '<bla'
sh: bla: command not found
^
warning: Skipping data file with no valid points
^
x range is invalid
See this of wgnuplot under Wine (pipes disabled):
gnuplot> load 'bla'
^
Cannot open load file 'bla'
gnuplot> load '<bla'
^
Cannot open load file '<bla'
See this of wgnuplot_pipes under Wine:
gnuplot> plot 'bla'
^
can't read data file "bla"
gnuplot> plot '<bla'
^
no data point found in specified file
The last message seems to be the most proper one, while "sh: bla: command
not found" is something unwanted.
> } else if (*df_filename == '-' && strlen(df_filename) == 1) {
> plotted_data_from_stdin = TRUE;
> } else {
> struct stat statbuf;
> if ((stat(df_filename, &statbuf) > -1) &&
> !S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
> os_error(name_token, "\"%s\" is not a regular file or pipe",
> df_filename);
> }
> if (!(data_fp = loadpath_fopen(df_filename, df_binary ? "rb" : "r"))
> os_error(name_token, "can't read data file \"%s\"", df_filename);
> }
---
PM |
|
From: Daniel J S. <dan...@ie...> - 2006-03-05 09:02:54
|
Ethan A Merritt wrote: > On Saturday 04 March 2006 12:45 pm, Petr Mikulik wrote: > >>Ah, these in addition: > > > Unfortunately, I don't understand the codepath for reading > binary files well enough to back out of these failures cleanly. > At the time the message is printed, we are several routines > deep in the binary file code. I think you would have to > add error return codes to each layer of routines, and > error-handling code at all the call sites. > > Maybe Daniel would be willing to take a look at it > after I sort out the normal [ascii] input file case. > It was his code originally, I think. > > >>gnuplot> plot 'xxx' binary filetype=edf with image >> Can't open data file "xxx" >> util.c: No such file or directory >>gnuplot> plot 'xxx' binary filetype=avs with image >> Can't open data file "xxx" >> util.c: No such file or directory >> >>gnuplot> plot 'demo.edfxxx' binary filetype=auto with image >> ^ >> Unsupported file type I waited for a delayed gnuplot-beta-list email to appear on this, but nothing has arrived. I don't fully follow the issue here. Is there a problem recognizing some file in the present working directory? The above first two complaints are indicating that. The third complaint is that the file type is to be recognized automatically. Gnuplot makes this decision based only upon the file extension. So, 'edf' is in the table of "extension to function pointer" mappings, but 'edfxxx' isn't. If we want to have something where gnuplot will also search into the file itself and look for keys about type, then we'd probably have to add another function pointer in the table containing a function that does the searching in addition to the one that handles parameter settings for the file. Dan |
|
From: Petr M. <mi...@ph...> - 2006-03-05 18:32:07
|
>>> gnuplot> plot 'demo.edfxxx' binary filetype=auto with image >>> ^ >>> Unsupported file type > > recognizing some file in the present working directory? The error message should be "file does not exist" instead of the above one. --- PM |
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-08 21:00:18
|
On Tuesday 07 March 2006 02:44 pm, Petr Mikulik wrote:
> > Then it needs yet another routine pipeexist().
>
> So, now there are (patch enclosed) both
> fileexist(s)
> which detects regular files, and
> pipeexist(s) which returns 1 for "-" and 1/0 whether it is a pipe
> wherefrom you can read at least 1 byte;
Huh? But that is quite broken. If you read from that data source,
even 1 byte, then you have poisoned it for actual plotting.
That's no good. This is a *pipe*, not a file. You cannot
necessarily go back to the beginning and read that byte a
second time.
> that's because:
> > Trimmed from lines 1240-1280 of datafile.c:
> > ===================================================================
> >====== /* filename cannot be static array! */
> > gp_expand_tilde(&df_filename);
>
> This line makes sense only for regular files and thus it should be
> moved to just above the "stat" function.
>
> > /*{{{ open file */
> > if (*df_filename == '<') {
> > if ((data_fp = popen(df_filename + 1, "r")) == (FILE *)
> > NULL) os_error(name_token, "cannot create pipe for data");
>
> This will never work.
???
But this is what the code does *now*.
Are you saying that the current pipe code doesn't work for you?
> The truth is that popen() "never" returns NULL
> (contrary to specification) -- at least for OS/2, Windows, Linux.
On linux it returns NULL if the pipe open fails.
OK, that's almost never, unless you have exceeded your quota for
open files. That is different from the case where the pipe is opened
but nothing can be read.
>
> gnuplot> plot '<bla'
> sh: bla: command not found
This is the correct error message.
There is no command "bla", so you cannot pipe its output.
> gnuplot> plot '<bla'
> ^
> no data point found in specified file
>
> The last message seems to be the most proper one, while "sh: bla:
> command not found" is something unwanted.
On the contrary. The pipe failed because there is no such command,
and that is the most enlightening error message.
"no data point..." would be correct if the pipe was successfully opened,
but the data stream contained no valid points. That can happen also,
but it is a separate error.
Anyhow, my main point is that an xxx_exists() routine should be
exactly parallel to the routine used for actual data input. Otherwise
you cannot trust the result as an indication of whether you will or
will not be able to plot it. Thus it should handle both files and
pipes, because the data input routine handles both files and pipes.
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From: Petr M. <mi...@ph...> - 2006-03-08 21:26:50
|
>> pipeexist(s) which returns 1 for "-" and 1/0 whether it is a pipe
>> wherefrom you can read at least 1 byte;
>
> Huh? But that is quite broken. If you read from that data source,
> even 1 byte, then you have poisoned it for actual plotting.
I don't think so. plot '<awk {blalbal} bla.dat' gives always the same.
Can you give an counter-example?
> That's no good. This is a *pipe*, not a file. You cannot
> necessarily go back to the beginning and read that byte a
> second time.
I do pclose().
>>> /*{{{ open file */
>>> if (*df_filename == '<') {
>>> if ((data_fp = popen(df_filename + 1, "r")) == (FILE *)
>>> NULL) os_error(name_token, "cannot create pipe for data");
>>
>> This will never work.
>
> ???
> But this is what the code does *now*.
> Are you saying that the current pipe code doesn't work for you?
>
>> The truth is that popen() "never" returns NULL
>> (contrary to specification) -- at least for OS/2, Windows, Linux.
>
> On linux it returns NULL if the pipe open fails.
No, it does NOT! Try it!
popen("bla bla", "r") does not return NULL
>> gnuplot> plot '<bla'
>> sh: bla: command not found
>
> This is the correct error message.
> There is no command "bla", so you cannot pipe its output.
I would prefer no system error message at all.
> On the contrary. The pipe failed because there is no such command,
> and that is the most enlightening error message.
But did not return NULL. So I must read at least one byte.
> Anyhow, my main point is that an xxx_exists() routine should be
> exactly parallel to the routine used for actual data input. Otherwise
> you cannot trust the result as an indication of whether you will or
> will not be able to plot it. Thus it should handle both files and
> pipes, because the data input routine handles both files and pipes.
Then I propose streamexist() that would do all of that; it would be
equivalent to
if (fileexist(file_in_path(x)) || pipeexist(x))
---
PM
|
|
From: Ethan M. <merritt@u.washington.edu> - 2006-03-08 22:44:23
|
On Wednesday 08 March 2006 01:26 pm, Petr Mikulik wrote:
> >> pipeexist(s) which returns 1 for "-" and 1/0 whether it is a
> >> pipe wherefrom you can read at least 1 byte;
> >
> > Huh? But that is quite broken. If you read from that data source,
> > even 1 byte, then you have poisoned it for actual plotting.
>
> I don't think so. plot '<awk {blalbal} bla.dat' gives always the
> same. Can you give an counter-example?
Any real-time data source?
A keyboard input stream?
A pipe to another interactive program?
> >> The truth is that popen() "never" returns NULL
> >> (contrary to specification) -- at least for OS/2, Windows, Linux.
> >
> > On linux it returns NULL if the pipe open fails.
>
> No, it does NOT! Try it!
> popen("bla bla", "r") does not return NULL
It does not return NULL in that case because it successfully opened
a pipe to the forked command "/bin/sh bla bla". The forked shell
itself returned an error ("command not found").
But there is no pipe failure here.
> I would prefer no system error message at all.
Then you must configure your shell environment to not
print an error. This is being printed by /bin/sh, not by gnuplot.
The fact that it is printed at all is an indication that the popen()
did in fact succeed.
> But popen() did not return NULL. So I must read at least one byte.
You are looking in the wrong place for the error return.
If you want to do error checking, you need something like this:
--- gnuplot/src/datafile.c 2006-01-27 08:36:48.000000000 -0800
+++ gnuplot-cvs/src/datafile.c 2006-03-08 14:25:53.000000000 -0800
@@ -1343,7 +1343,9 @@
if (!mixed_data_fp) {
#if defined(PIPES)
if (df_pipe_open) {
- (void) pclose(data_fp);
+ int ierr = pclose(data_fp);
+ if (ierr)
+ fprintf(stderr,"pipe error %s\n",strerror(ierr));
df_pipe_open = FALSE;
} else
#endif /* PIPES */
With that patch in place, your test example gives:
gnuplot> plot "< bla bla"
sh: bla: command not found
pipe error Unknown error 32512
I'm not sure how to make it print a more interpretable error message
string, but I assume that 32512 in fact means "command not found".
> > Anyhow, my main point is that an xxx_exists() routine should be
> > exactly parallel to the routine used for actual data input.
> > Otherwise you cannot trust the result as an indication of whether
> > you will or will not be able to plot it. Thus it should handle
> > both files and pipes, because the data input routine handles both
> > files and pipes.
>
> Then I propose streamexist() that would do all of that; it would
> be equivalent to
> if (fileexist(file_in_path(x)) || pipeexist(x))
OK, but the pipe part of the test must not actually read any data.
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle WA
|
|
From:
<br...@ph...> - 2006-03-09 17:04:06
|
Ethan Merritt wrote: > Huh? But that is quite broken. If you read from that data source, > even 1 byte, then you have poisoned it for actual plotting. Not necessarily. If you do it via <stdio.h> functions, you can still use fungetc() to put the djinn back into the bottle. But the real problem is, of course a different one: blocking I/O. If you try to read from an existing stream, but there's nothing in it, gnuplot may just sit there and wait forever > Anyhow, my main point is that an xxx_exists() routine should be > exactly parallel to the routine used for actual data input. I had given up on following this discussion for a while, because I still maintain a rather more serious doubt. The length at which you guys have discussed the vagaries of implementing this features brings me back to that issue: I simply don't believe that either of these functions does something that gnuplot has to do, or even *can* do meaningfully. gnuplot is a plotting program. It's not a command-line shell, and it's not a full-featured script language processor. Nor, IMHO, should it try and become either of those. What's the point of asking a question like "if fileexists", when there's nothing useful that could be done with the answer, that we can't already do later on, when a command actually tries to read it? Why ask if the bridge is there before you come to it? What's the assumption based on that this file should exist, but maybe it doesn't? AFAICS, it's based on the fact that somewhere a program is running that is supposed to create that file. So what makes it gnuplot's job to check for this event? It's not as if gnuplot had thousands of other terribly important things to do that it should take care of while some other program sets up that file. So why was gnuplot started or that script loaded, before the datafile was ready? We're not the shell. We're just another program. So if some other program is to create a data file and then gnuplot should read that file, the right place to ensure that sequence is in the other program, or in the shell. gnuplot is the wrong place to do this. Worse yet, existence of a file is rather certainly not even the information we actually need about that file in the first place. We need to know whether whichever process is writing that file considers itself done with it. There's *no* way at all we could deduce that from looking at the file. It's not even worth trying. So to sum it up: I really don't see why we would need these functions. And given the apparent complexities of implementing it, I strongly suggest we just drop this plan. It's getting us nowhere we should be. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X