Menu
▾
▴
gnuplot-beta
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-06 15:52:34
|
On Tue, 6 Jan 2004, Arnd Baecker wrote: > Redoing the copy and paste a second time it works as expected. (setting > `unset mouse` or -nofeedback did not change anything). That's the original X11 buffering/pasting problem alright, but possibly in a new disguise. It shouldn't happen in "unset mouse" mode, though. You may have to disable both mouse and feedback now. AFAICS, the problem happens whenever keyboard input arrives while gnuplot is in the process of drawing a plot. Both font size feedback and mouse command feedback try to feed their input into the same channel as the keyboard, leading to a kind of race-condition. The process is entirely timing-dependent, and thus quite hard to debug. -- Hans-Bernhard Broeker (br...@ph...) Even if all the snow were burnt, ashes would remain. |
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-08 12:32:50
|
On Wed, 7 Jan 2004, Ethan Merritt wrote: > So it isn't really a race. It's just that if we are going to > switch to unbuffered mode we should first make sure the > buffer is empty, or else store it elsewhere for later execution. Doing so will *make* it a race, then. A race between gnuplot going from 'read all that's in the buffer' and 'set to unbuffered' on one side of the track, and the external program re-filling the buffer on the other. And that's before we consider that there's no way, AFAIK, to read only the buffer, for stdin. You only get to notice the buffer is empty when it tries to read one past the end --- at which point it blocks, and gnuplot becomes irresponsive. I suspect we'll have to avoid stdin buffering and possibly blocking I/O altogether. I.e. don't switch off buffering halfway into the program, but immediately after startup, before the first character is read. -- Hans-Bernhard Broeker (br...@ph...) Even if all the snow were burnt, ashes would remain. |
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-08 12:49:44
|
On Wed, 7 Jan 2004, Ethan Merritt wrote:
> The X11 paste buffer is now working, and piping through awk
> now works too. I left the call to setvbuf() protected by
> #ifdef X11 but I doubt that it needs to be. Are any non-X11
> platforms also suffering lost characters in the input stream?
If so, they've kept themselves remarkably quiet, or haven't even
noticed the problem yet.
There are only a few noteworthy candidates besides X11, to begin with:
*) OS/2
*) MS Windows
These both use a non-<stdio.h> input scheme, I think, so there's
no buffering to be worried about.
*) MacOS classic --- not our business, we never made that version
*) MacOS X / OpenStep --- I've no idea.
*) Full-screen direct VGA drivers for the PC (linuxvga, DOS versions, GGI)
These rely on a keypress event to switch back and forth between graph
and command line. But none of them has mouse or other feedback
coming from the driver, so they should be safe.
--
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.
|
|
From: Petr M. <mi...@ph...> - 2004-01-08 13:27:30
|
> If so, they've kept themselves remarkably quiet, or haven't even > noticed the problem yet. > > There are only a few noteworthy candidates besides X11, to begin with: > > *) OS/2 There are no problems there. There is a bidirectional named pipe gnuplot/pm.trm <=> gnupmdrv.exe, and mousing commands gnupmdrv => gnuplot go via shared memory and an event semaphore. Commands from user/gnuplot and mouse/hotkey/gnupmdrv don't mix because the waiting for mouse commands is encapsulated in a thread, see command.c. --- Petr Mikulik |
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-09 13:58:06
|
On Thu, 8 Jan 2004, Arnd Baecker wrote: > After this the mouse coordinates in the plot window > are not updated anymore. That's not exactly a surprise. When you made a plot to some other terminal driver, you essentially decoupled gnuplot_x11 from gnuplot, making several kinds of mouse interaction unavailable. IOW, 'set term post ; replot' essentially acts as a (temporary) equivalent of 'set mouse off'. -- Hans-Bernhard Broeker (br...@ph...) Even if all the snow were burnt, ashes would remain. |
|
From: Arnd B. <arn...@we...> - 2004-01-09 13:58:42
|
On Fri, 9 Jan 2004, Hans-Bernhard Broeker wrote: > On Thu, 8 Jan 2004, Arnd Baecker wrote: > > > After this the mouse coordinates in the plot window > > are not updated anymore. > > That's not exactly a surprise. When you made a plot to some other > terminal driver, you essentially decoupled gnuplot_x11 from gnuplot, > making several kinds of mouse interaction unavailable. IOW, > 'set term post ; replot' essentially acts as a (temporary) equivalent > of 'set mouse off'. Ok, I wasn't aware of this - so the solution ist pretty easy, just add a replot at the end, i.e. set xrange [0:10] print "blurb" plot sin(x) set out "tst.ps" set term post rep set out set term x11 replot and the mouse works fine again. Many thanks, Arnd |
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-10 14:40:55
|
On Sat, 10 Jan 2004, Daniel J Sebald wrote:
[....]
> You'll have to forgive me. I've tried following the code to see exactly
> the path that the commands come in via stdin, where the pipe is read for
> the mouse, etc. and I'm having a difficult time keeping track of it all.
The core function is generally term->waitforinput(), which in the case of
the x11 terminal is provided by X11_waitforinput() in term/x11.trm.
There are different call paths leading there. Let me provide an overview.
Cscope helps:
C symbol: waitforinput
File Function Line
0 term_api.h <global> 172 int (*waitforinput) __PROTO((void ));
1 command.c pause_command 961 if (term && term->waitforinput) {
2 command.c pause_command 964 term->waitforinput();
3 command.c fgets_ipc 2212 if (term && term->waitforinput) {
4 command.c fgets_ipc 2219 c = term->waitforinput();
5 readline.c getc_wrapper 84 if (term && term->waitforinput && interactive) {
6 readline.c getc_wrapper 88 c = term->waitforinput();
7 readline.c ansi_getc 775 if (term && term->waitforinput && interactive)
8 readline.c ansi_getc 776 c = term->waitforinput();
The call paths ultimately start at read_line in command.c. I'll assume
MOUSE is active:
no READLINE (neither gnuplot's own, nor -lreadline):
GET_STRING ==> fgets
some READLINE, !interactive:
fgets_ipc ==> term->waitforinput
GNU libreadline, interactive:
read_line ==> rlgets ==> readline_ipc ==> rl_callback_read_char
(getc_wrapper passed to libreadline as input hook)
gnuplot readline.c, interactive:
read_line ==> rlgets ==> readline_ipc ==> readline
==> special_getc ==> ansi_getc
So that's how we get to the core function. Now let's look at
X11_waitforinput itself: it uses select() to dispatch between the two
input streams. The root of all our trouble is that select() operates not
on <stdio.h> objects like stdin, but rather on unbuffered Unix file
handles. I.e. select doesn't know a thing about <stdio.h> buffering.
> I find it hard to fathom that this is not possible.
It's quite certainly possible. The tricky part is to make it possible
in the context of a massively portable program like gnuplot, without
breaking the whole program for use on other platforms.
> I mean, doesn't this seem like a fundamental task of operating systems?
> That is, to distribute I/O around the system? Let's consider an fread()
> or a fgets() or fgetc().
For starters, fgetc() and friends aren't even functions of the operating
system to begin with. They're functions of the C runtime library --- but
ISO/ANSI C doesn't standardize asynchronous I/O.
> Can't I just do an fread() from the stdin (the place where the commands
> are typed in) and whatever is there I store in a "command buffer" and as
> I'm putting the characters in a command buffer I watch for the "new
> line" or "end of gnuplot command" character.
No, you can't. The reason is that file I/O on Unix (and all other
platforms I've seen) is by default a "blocking" operation:
> If there is nothing in the buffer or there are less characters than
> asked for, doesn't the fread() simple return and EOF or -1 or number of
> bytes read?
If there's buffered input, but less than requested, it'll return what
there is. But if there's currently _nothing_ in the buffer, it won't
return at all until there is. You only get an EOF condition if the stream
is actually ended (typical Unix setup: if you typed Ctrl-D at the
keyboard), not if there's just no buffered input available at the moment.
> >F) The usual way around this dilemma is to use poll or select to notify
> > us when either or both of two input streams is presenting new data.
> > We currently use select in X11_waitforinput().
> >
>
> Yes, this is one way. But if "fread()" is inherently forced to return
> something even if the buffer is empty, isn't that a form of polling the
> input as well?
The problem is we *can't* force fread() to do that. Non-blocking I/O
is not foreseen in the context of <stdio.h> functions.
> As I said before, it seems to me that current Gnuplot
> really isn't wanting polling for "characters available", rather it is
> attempting to poll for "gnuplot command in buffer available" and "mouse
> command in buffer available".
I'm quite sure that reading characters versus lines is unrelated to the
problem at hand.
> >G) Unfortunately, poll/select notification depends on whether the
> > input stream is buffered or non-buffered. I do not know if this is
> > supposed to be the case, but that's what we see in practice.
Indeed select() doesn't know a thing about buffering, so it will not
report a stream as having data available if all of it is buffered.
--
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.
|
|
From: Hans-Bernhard B. <br...@ph...> - 2004-01-11 00:10:39
|
On Sat, 10 Jan 2004, Petr Mikulik wrote: > As I said earlier, I don't know about the techniques you describe in the > previous letters .. but cannot be something modern used to get rid of these > problems? Modern solutions have the crucial problem of being modern; meaning that they're not particularly unified or reliably available all across the spread of platforms we're dealing with. > Or would this mean big non-portability to different X11 systems? I strongly suspect so. -- Hans-Bernhard Broeker (br...@ph...) Even if all the snow were burnt, ashes would remain. |
|
From: Arnd B. <arn...@we...> - 2004-01-15 09:24:51
|
Hi, for me both pasting from X11 and piping (from python) works great now - Many many thanks ! The example for accessing gnuplot variables and waiting for mouse-clicks from python can be obtained as http://www.physik.tu-dresden.de/~baecker/python/GnuplotBiDir.py see http://www.physik.tu-dresden.de/~baecker/python/gnuplot.html for a few notes. Arnd |
|
From: <arn...@we...> - 2004-01-07 09:07:16
|
On Tue, 6 Jan 2004, Hans-Bernhard Broeker wrote:
> On Tue, 6 Jan 2004, Arnd Baecker wrote:
>
> > Redoing the copy and paste a second time it works as expected. (setting
> > `unset mouse` or -nofeedback did not change anything).
>
> That's the original X11 buffering/pasting problem alright, but possibly
> in a new disguise. It shouldn't happen in "unset mouse" mode, though.
> You may have to disable both mouse and feedback now.
I tried this as well, however, the problem remains.
> AFAICS, the problem happens whenever keyboard input arrives while gnuplot
> is in the process of drawing a plot. Both font size feedback and mouse
> command feedback try to feed their input into the same channel as the
> keyboard, leading to a kind of race-condition. The process is entirely
> timing-dependent, and thus quite hard to debug.
Yes this is really nasty (sorry for bringing it up again ;-) -
anyway, I am just wondering if there is any chance at all that
this can be made to work in a reliable way?
Are there any reasonable alternatives?
Alright, then I tried to re-compile with
./configure --disable-mouse
Here the compile ended with
if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../term -I../term
-DBINDIR=\"/home/python/PYTHON//bin\"
-DX11_DRIVER_DIR=\"/home/python/PYTHON//libexec/gnuplot/3.8k\"
-DCONTACT=\"gnu...@li...\"
-DHELPFILE=\"/home/python/PYTHON//share/gnuplot/3.8k/gnuplot.gih\"
-I/usr/X11R6/include -I/usr/include/libpng12 -g -O2 -MT term.o -MD -MP
-MF ".deps/term.Tpo" \
-c -o term.o `test -f 'term.c' || echo './'`term.c; \
then mv -f ".deps/term.Tpo" ".deps/term.Po"; \
else rm -f ".deps/term.Tpo"; exit 1; \
fi
In file included from term.h:267,
from term.c:1043:
../term/x11.trm: In function `X11_args':
../term/x11.trm:304: error: `X11_MOUSE_FEEDBACK' undeclared (first use in
this function)
../term/x11.trm:304: error: (Each undeclared identifier is reported only
once
../term/x11.trm:304: error: for each function it appears in.)
make[3]: *** [term.o] Error 1
make[3]: Leaving directory
`/home/python/INSTALL_PYTHON/CompileDir/gnuplot/src'
I.e., the source of this error are the lines
if (strcmp(*argv, "-nofeedback") == 0)
X11_MOUSE_FEEDBACK = FALSE;
Commenting out these two lines compilation happily continued to the end.
With this there is no problem with pasting anymore
(however, all the nice mousing stuff is gone ;-)...)
Two more remarks:
1.) I mentioned in my previous mail (in the P.P.S.) that I have
some problems when piping commands to gnuplot.
Whereas an `unset mouse` did not cure them, the
--disable-mouse did.
So all this feedback and mousing stuff seems problematic
when sending commands to gnuplot via a pipe...
2.) I'd really love to have mousing working, even when gnuplot
is steered via a pipe.
Example:
- do a plot (in my case from python)
- send a "pause mouse" command
- read MOUSE_X and MOUSE_Y back to the calling program
- start some program/computation with these coords
as input
- go to step one with the new data
See http://www.physik.tu-dresden.de/~baecker/python/plot.html
for a brief example on how to get variables from gnuplot
back to python.
Regards,
Arnd
|
|
From: Ethan M. <merritt@u.washington.edu> - 2004-01-07 17:56:30
|
On Wednesday 07 January 2004 01:07, arn...@we... wrote:
>
> Yes this is really nasty (sorry for bringing it up again ;-) -
> anyway, I am just wondering if there is any chance at all that
> this can be made to work in a reliable way?
I do not think it is possible so long as the terminal and
X11 input streams are intermingled.
> Are there any reasonable alternatives?
Yes, but it would be a major, major change to the existing layout.
So it won't happen any time soon.
> Alright, then I tried to re-compile with
> ./configure --disable-mouse
> Here the compile ended with
>
> In file included from term.h:267,
> from term.c:1043:
> ../term/x11.trm:304: error: `X11_MOUSE_FEEDBACK' undeclared (first use
> in this function)
I'll fix that immediately.
> So all this feedback and mousing stuff seems problematic
> when sending commands to gnuplot via a pipe...
This is a known problem, but none of us fully understand it.
It seems to depend on the environment from which the pipe
is opened. For me, it works from C and from octave but fails from awk.
It seems to work from perl also, but my perl scripts are not
normally being run interactively so I'm not sure I would have
seen the failure in practice.
In the case of awk scripts, the problem can be ameliorated by
commenting out the call to setvbuf() in x11.trm as in the=20
following patch:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
--- gnuplot/term/x11.trm 2003-09-30 13:14:43.000000000 -0700
+++ gnuplot-cvs/term/x11.trm 2003-10-20 17:26:10.000000000 -0700
@@ -599,12 +607,14 @@
ipc_back_fd =3D fdes_back[0];
close(fdes_back[1]); /* the parent doesn't need this *=
/
=20
+#ifdef BROKEN /* EAM Oct 2003 - This has broken awk+gnuplot */
/* apparently multi-character inputs like escape sequences
* but also mouse-pasted text got buffered and therefore
* didn't trigger the select() function in X11_waitforinput()=
=2E
* Switching to unbuffered input solved this (although I don'=
t
* really like this solution 23 Jan 2002 (joze) */
setvbuf(stdin, (char *) 0, _IONBF, 0);
+#endif=20
} else {
/* we do not open a bidirectional communication
* for non-tty's by default. If this is desired,
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
You may find that the same is true for your python environment.
I say "ameliorated" rather than "fixed" because if you apply this
patch then other peculiarities result - the main one being that
"pause -1" behaves unreliably.=20
> 2.) I'd really love to have mousing working, even when gnuplot
> is steered via a pipe.
> Example:
> - do a plot (in my case from python)
> - send a "pause mouse" command
> - read MOUSE_X and MOUSE_Y back to the calling program
For that you need the full-blown feedback loop, not just --enable-mouse
I did get this working nicely from C code, so the basic idea is
sound. The problem comes from some difference in the scripting
environment. I wish I knew exactly what.
> See http://www.physik.tu-dresden.de/~baecker/python/plot.html
> for a brief example on how to get variables from gnuplot
> back to python.
You may find that removing the call to setvbuf() makes your
python example work. Please let us know.
--=20
Ethan A Merritt merritt@u.washington.edu
Biomolecular Structure Center Box 357742
University of Washington, Seattle, WA 98195
|
|
From: Daniel J S. <dan...@ie...> - 2004-01-07 18:12:57
|
Ethan Merritt wrote: >On Wednesday 07 January 2004 01:07, arn...@we... wrote: > > >>Yes this is really nasty (sorry for bringing it up again ;-) - >>anyway, I am just wondering if there is any chance at all that >>this can be made to work in a reliable way? >> >> > >I do not think it is possible so long as the terminal and >X11 input streams are intermingled. > <snip> I should have double-checked my email before sending my last note. -- Dan |
|
From: Daniel J S. <dan...@ie...> - 2004-01-07 18:06:11
|
arn...@we... wrote: >On Tue, 6 Jan 2004, Hans-Bernhard Broeker wrote: > > > >>On Tue, 6 Jan 2004, Arnd Baecker wrote: >> >> >> >>>Redoing the copy and paste a second time it works as expected. (setting >>>`unset mouse` or -nofeedback did not change anything). >>> >>> >>That's the original X11 buffering/pasting problem alright, but possibly >>in a new disguise. It shouldn't happen in "unset mouse" mode, though. >>You may have to disable both mouse and feedback now. >> >> > >I tried this as well, however, the problem remains. > > > >>AFAICS, the problem happens whenever keyboard input arrives while gnuplot >>is in the process of drawing a plot. Both font size feedback and mouse >>command feedback try to feed their input into the same channel as the >>keyboard, leading to a kind of race-condition. The process is entirely >>timing-dependent, and thus quite hard to debug. >> >> > >Yes this is really nasty (sorry for bringing it up again ;-) - >anyway, I am just wondering if there is any chance at all that >this can be made to work in a reliable way? >Are there any reasonable alternatives? > >Alright, then I tried to re-compile with >./configure --disable-mouse >Here the compile ended with > >if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../term -I../term >-DBINDIR=\"/home/python/PYTHON//bin\" >-DX11_DRIVER_DIR=\"/home/python/PYTHON//libexec/gnuplot/3.8k\" >-DCONTACT=\"gnu...@li...\" >-DHELPFILE=\"/home/python/PYTHON//share/gnuplot/3.8k/gnuplot.gih\" >-I/usr/X11R6/include -I/usr/include/libpng12 -g -O2 -MT term.o -MD -MP >-MF ".deps/term.Tpo" \ > -c -o term.o `test -f 'term.c' || echo './'`term.c; \ >then mv -f ".deps/term.Tpo" ".deps/term.Po"; \ >else rm -f ".deps/term.Tpo"; exit 1; \ >fi >In file included from term.h:267, > from term.c:1043: >../term/x11.trm: In function `X11_args': >../term/x11.trm:304: error: `X11_MOUSE_FEEDBACK' undeclared (first use in >this function) >../term/x11.trm:304: error: (Each undeclared identifier is reported only >once >../term/x11.trm:304: error: for each function it appears in.) >make[3]: *** [term.o] Error 1 >make[3]: Leaving directory >`/home/python/INSTALL_PYTHON/CompileDir/gnuplot/src' > >I.e., the source of this error are the lines >if (strcmp(*argv, "-nofeedback") == 0) > X11_MOUSE_FEEDBACK = FALSE; >Commenting out these two lines compilation happily continued to the end. > >With this there is no problem with pasting anymore >(however, all the nice mousing stuff is gone ;-)...) > >Two more remarks: > 1.) I mentioned in my previous mail (in the P.P.S.) that I have > some problems when piping commands to gnuplot. > Whereas an `unset mouse` did not cure them, the > --disable-mouse did. > So all this feedback and mousing stuff seems problematic > when sending commands to gnuplot via a pipe... > 2.) I'd really love to have mousing working, even when gnuplot > is steered via a pipe. > Example: > - do a plot (in my case from python) > - send a "pause mouse" command > - read MOUSE_X and MOUSE_Y back to the calling program > - start some program/computation with these coords > as input > - go to step one with the new data > See http://www.physik.tu-dresden.de/~baecker/python/plot.html > for a brief example on how to get variables from gnuplot > back to python. > > Arnd, I will recap my thoughts on this from the discussion a while back as I remember this issue. I think to do this right a new scheme and structure for the code would be necessary. I have a suspicion that the approach of feeding mouse data into the keyboard stream has a fatal flaw that will show up in one way or another. That is, combining two input streams would not be a problem if there would be some device or code included along with the data to distinguish where the input originated from. The devices don't currently work that way and that cannot be changed. So, in my opinion, the input streams need to be kept separate all the way to the Gnuplot core and then there they can be handled separately. I think more versatility is a good idea, as the calling program idea you describe is something I would see useful. My preference would be to have a command that instructs how the mouse should be directed. (I think someone, probably Hans, gave a logical argument for this being difficult to do.) That is, the system could be a "handler" gets connected to the mouse stream via a pipe or something. The default handler in Gnuplot would be the one that behaves as Gnuplot currently does. Primitives about the button pressed and position (i.e., info that Ethan has displayed in his demonstration recently) could be passed to the handler. The handler could be some routine in the calling program. Regardless if you agree with the scheme I propose (I'm fine with a variable approach, I guess), there is still a fundamental flaw with the current structure of Gnuplot for keeping the keyboard and mouse separate. If I recall correctly, the core of Gnuplot sort of branches off to an input routine where it waits for input from the keyboard. That's not a good programming scheme in my opinion. Rather, there should be a tight loop in the core of Gnuplot that checks if keyboard input is available. If so, then process it by checking if a line is completed and ready for interpretation. Next, it should check if primitives are available from the mouse (unless the mouse handler has been directed somewhere else). If so, then process it by checking if it represents a valid redraw command, etc. Program flow should be a tight loop as such where it is never really waiting on anything. In any case, those are my thoughts. I don't want to make any work for Ethan, but I think he is the one who could come up with a better scheme and implement it fairly easy. Perhaps he'd rather put effort into revamping the input than dealing with what seems to have become frustration with chasing down this problem.... post 4.0, I would say. Dan -- Dan Sebald email: daniel DOT sebald AT ieee DOT org URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/ |
Thanks for helping keep SourceForge clean.
X