#609 Add support for '<&n' (read from file descriptor)

[W. Trevor King]

Currently gnuplot supports the '< SHELL COMMAND' syntax. This patch extends gnuplot with a similar syntax to read data from already opened file descriptors (using the stdin case of Bash's `[n]<&digit-` syntax for moving file descriptors). I'm not entirely clear on how the surrounding code is supposed to work, so it's possible that I've missed something. The following simple tests do work:

$ seq 10 | gnuplot -p -e "plot '<&0'"
$ cat data-1 | ./src/gnuplot -p -e "plot '<&0', '<&3', '<&4'" 3<data-2 4<data-3

[W. Trevor King]

Add support for '<&n' file descriptor filenames

[Ethan Merritt]

I find serious problems when testing this patch:

plot '<&' or plot '<&0'

correctly plots one set of input points (terminated by ^D) but then hangs the program, which must be killed externally.

plot '<&1'

apparently plots from stdin (why?) but clobbers the echo of input characters to the terminal so that from that point on one is typing blind.

plot '<&2'

again plots from stdin (why?). Does not clobber echo to terminal but does clobber stderr.

So at least on my test machines (linux 32-bit gcc; linux 64-bit clang; libreadline6) this is not working correctly.

[W. Trevor King]

Here are some additional patches fixing most of these issues. There is at least one outstanding problems using fcntl checking, where:

$ gnuplot -e "set terminal dumb; plot '<&1" 1< <(seq 5)

hangs. This is because gnuplot reads the data from file descriptor 1 (the piped in seq output) and then attempts to plot to stdout. I'm not sure how to handle this case. Ideally, gnuplot would realize that FD 1 was read-only and would die when it tried to write to it. I'll look into this next.

A simple solution would be to raise an error if the requested FD matched STD*_FILENO, but that smells funny to me. I may change my mind depending on how difficult fixing the above issue turns out to be.

I've attached the additional patches. Feel free to squash when applying.

Hmm, looks like this is one patch at a time. Additional new patches to come ;).

[W. Trevor King]

Extra motivation

[W. Trevor King]

Check for read-only file descriptors with '<&n'

[Ethan Merritt]

Some meta questions:

1. Why should it be legal to accept input from stdout or stderr? I think that should be rejected as an error.
2. I'm not so sure about accepting input on stdin by this mechanism either. Is this necessary, given that there are already ways to read data from stdin? I'm worried as a purely practical matter that having two or more parallel channels all reading from stdin is going to cause problems.
3. Even apart from the bad effects from closing stdin/stdout/stderr, wouldn't it be better to leave data_fd open after plotting? That way you can continue to plot from the same input channel more than once. If you close it - that's the end of it.
4. What happens if gnuplot is running on a system that doesn't use bash? VMS? SunOS? Windows?

[W. Trevor King]

1. Why should it be legal to accept input from stdout or stderr? I think that should be rejected as an error.

It shouldn't be legal. That's what the read-only check is for. Your alternative way of avoiding this is requiring FDs > 2, which will work except when:

1. The user uses those file descriptors for something else (in which case reading data from them should be legal):

   $ gnuplot -p -e "plot '<&1'" 1< <(my-program)

2. The user dups those file descriptors to higher values (in which case reading data from the higher values should be illegal):

   $ exec 4>&1
   $ gnuplot -p -e "plot '<&4'"p 4< <(my-program)

Maybe these pathalogical cases are too unlikely to be worth dealing with?

1. Even apart from the bad effects from closing stdin/stdout/stderr, wouldn't it be better to leave data_fd open after plotting? That way you can continue to plot from the same input channel more than once. If you close it - that's the end of it.

That's a good idea, and works around most of the stdin/stdout/stderr issues. We'd need a way to flag file datafiles for not-closing though.

1. What happens if gnuplot is running on a system that doesn't use bash? VMS? SunOS? Windows?

The input redirection syntax is in POSIX. If you're not running POSIX, I dunno how you setup the file descriptors, but I don't know how you'd setup named pipes there either ;).

[Ethan Merritt]

I worry that your case 1 (redirecting program output through gnuplot's stdin) will cause synchronization problems. Won't this result in stdin being open twice in gnuplot? Some reads will come via one stream (fp), some by a different stream. In particular I am dubious about multiplexed mouse input, which is synchronized using FD_SET/select().

It seems to me that one of the nice things about your patch is that it would provide a way to feed data to gnuplot that bypasses the need to mix commands and data in a single stream (stdin). So my inclination is to forbid opening stdin via this mechanism. Is there a standard way for the code to test whether a requested fd is really stdin?

[Ethan Merritt]

I'm thinking that what we want is

data_fd = <extract from command string>;
if (data_fd == fileno(stdin) 
||  data_fd == fileno(stdout)
||  data_fd == fileno(stderr))
    int_error(c_token, "Cannot take file input from stdin/stdout/stderr");
else
    data_fp = fd_open(data_fd, "r");

[Ethan Merritt]

Something like this...

[W. Trevor King]

This looks good to me, especially the rewinding. Minor nits:

1. I'd use “Cannot use stdin, stdout, or stderr as data files” instead of “Cannot map input file to stdin/stdout/stderr”.

2. I'm not sure about error message capitalization. The existing source has examples of both lowercase, and sentence case. This probably doesn't really matter ;).

3. I'd stick with os_error for fdopen, so the user can see the errno message (but maybe int_error also prints errno?).

4. You add an unrelated “Special filenames” comment.

5. You add an an unrelated newline.

6. There is trailing whitespace after “harmless if it connects to a pipe.”

7. There isn't a space after stderr, in the fprintf call.

8. Paths in the new doc example assume you're in the gnuplot src directory.

Cleaned up version of the patch attached.

[Ethan Merritt]

Add to CVS for 4.6 and 4.6. As per Email discussion, we may need to add a fflush() command in df_reset_after_error(). The problem is that it is not clear how to find which fd to flush at this point.