#2292 pause -1 "swallows" plots if I do 'set terminal x11'

[Anonymous]

Dear all, maybe this was the wrong channel: https://sourceforge.net/p/gnuplot/mailman/gnuplot-bugs/?viewmonth=202006&viewday=22 However, the problem is still present in Gnuplot version 5.4.0. If I do "set terminal x11", pause -1 "swallows" plots. It seems to work without "set terminal x11", thus using the default which is x11... Best regards, Simon

[Ethan Merritt]

What mechanism are you using to execute this script?

I find no errors with
gnuplot> load 'pause.bug'
or
gnuplot pause.bug

but yes I can reproduce the behavior if I use the form
gnuplot < pause.bug

Since the "pause -1" command doesn't really make any sense in that context, I'm not sure how serious this bug is. The recommended method of introducing an interactive pause into a piped script is
pause mouse close

Does that work for your application?

Also, are you saying that you see this only with the x11 terminal? To the extent that I can trigger this using the 3rd mechanism above it seems to be indepent of the terminal selection. E.g. even if I set GNUTERM to "post", a non-interactive terminal. the result is the same.

[Simon Lutz]

Dear Ethan,

thanks a lot for your comments. I run
gnuplot pause.gnuplot
And this is my (extended) pause.gnuplot script:

show version;
print GPVAL_COMPILE_OPTIONS;
show terminal;
plot sin(x);pause -1 "Hit return to continue";
plot cos(x);pause -1 "Hit return to continue";
plot tan(x);pause -1 "Hit return to continue";
plot sin(x);pause -1 "Hit return to continue";
set terminal x11;
show terminal;
plot sin(x);pause -1 "Hit return to continue";
plot cos(x);pause -1 "Hit return to continue"; # this plot is swallowed
plot tan(x);pause -1 "Hit return to continue";
plot sin(x);pause -1 "Hit return to continue";
set terminal x11;
show terminal;
plot sin(x);pause mouse close "Close the Gnuplot-X11 window to continue\n";
plot cos(x);pause mouse close "Close the Gnuplot-X11 window to continue\n";
plot tan(x);pause mouse close "Close the Gnuplot-X11 window to continue\n";
plot sin(x);pause mouse close "Close the Gnuplot-X11 window to continue\n";

Your suggestion with pause mouse close does work. However, if I have dozens of plots to inspect I would prefer the pause -1 mechanism with the return key.
The behavior is a bit strange but very much reproducable on my machine.

Kind regards,

Simon

[Ethan Merritt]

Thank you for the clarification. I cannot reproduce that here so at the moment I do not even know where to start looking. Can you confirm the original report you linked to that the behavior is different in two states : (1) immediately after program entry when x11 is chosen as the default terminal no problem is seen (2) after "set term x11", which should not change anything since the terminal is already x11 anyhow, the problem occurs? That seems truly strange and might possibly be a hint to something.

[Simon Lutz]

Dear Ethan, that is exactly what I observe. And in version 5.2.6 this problem was not there. I do not know if it has somehting to do with the changes in src/mouse.c? Cheers, Simon

[Hiroki Motoyoshi]

I could see this phenomenon in my environment (MacOS, gnuplot 5.5) as well.
If you call "set terminal" more than once with x11, wxt, or qt as an argument, the "swallow" phenomena seem to occur depending on the number of calls.

I checked this behavior with the following script setting GNUTERM="dumb".

n = 10
do for [i=1:n] {
  set terminal wxt
  # set terminal x11
  # set terminal qt
}

print "sin"
plot sin(x) w l
pause -1

print "cos"
plot cos(x) w l
pause -1

print "tan"
plot tan(x) w l
pause -1

print "sin2"
plot sin(x) w l
pause -1

print "cos2"
plot cos(x) w l
pause -1

print "tan2"
plot tan(x) w l
pause -1

print "done"

If n=1, no "swallow" will occur. However, if n>2, (n-1) times "swallow" will occur.

The most suspicious part of the source code is the following part in the function event_reset in mouse.c.

/* This hack is necessary on some systems in order to prevent one
 * character of input from being swallowed when the plot window is
 * closed. But which systems, exactly, and in what circumstances?
 * FIXME (June 2020):
 * A change to readline.c:getc_wrapper may have made this unnecessary.
 */
if (paused_for_mouse || !interactive) {
    if (term && (!strncmp("x11",term->name,3)
             || !strncmp("wxt",term->name,3)
             || !strncmp("qt",term->name,2)))
        ungetc('\n',stdin);
}

This ungetc is called every time the "set terminal" command for x11, wxt, or qt is executed. The extra "\n" buffered by "ungetc" may cause the input to be "swallowed".

I hope this report gives you a hint of what to fix.

Thanks.

[Ethan Merritt]

You have it sort of backwards. The code fragment you show was added as a work-around for the swallow-one-character behavior. The idea was that if the program is known to sometimes swallow the next character, we can provide it with a harmless character to swallow. The next command should execute properly whether or not the extra '\n' is swallowed. At the time this work-around was adopted it did fix some cases of swallowing. But not all of them. And the ones it does not catch are the ones you are seeing now.

[Hiroki Motoyoshi]

Thank you for your explanation. I understand that the workaround with ungetc() has a long story.
I think the problem here is that when "set terminal" is called, ungetc() is also called. I don't understand why GE_reset (and ungetc()) is called when 'set terminal' is evaluated. If the workaround using ungetc() is to deal with the case of "when the plot window is closed", how about adding term_initialised condition to call ungetc() as in the attached patch? After applying the patch, the ungetc() is not called when setting terminal either x11, wxt, or qt, and it is called when closing the plot window.

[Ethan Merritt]

Can you give me a recipe for triggering this ungetc() before the terminal is initialized? I cannot reproduce it. Note that it can only be called if the flag paused_for_mouse is set, and I don't see how that can happen before the terminal is initialized.

I don't think your patch would hurt anything, but also I don't see how this addresses the "swallowing" problem. If a character is being lost, that would indicate we need to add an ungetc() somewhere, not remove one.

[Hiroki Motoyoshi]

I have checked the source code again and found that event_reset() is called by set_terminal() in "set.c". You can find the following code.

  5167 #ifdef USE_MOUSE_MOUSE
  5168 event_reset((void *)1); /* cancel zoombox etc.
  5169 #endif

Therefore, GE_reset is not relevant here.

Note that it can only be called if the flag paused_for_mouse is set

I don't think so. If (! interactive) is true, it can be called. Please check this code in mouse.c.

  2427      if (paused_for_mouse || !interactive) {
  2428          if (term && (!strncmp("x11",term->name,3)
  2429                   || !strncmp("wxt",term->name,3)
  2430                   || !strncmp("qt",term->name,2)))
  2431              ungetc('\n',stdin);
  2432      }

I checked using the debugger and found that paused_for_mouse is false and interractive is false when event_reset() called from set_terminal().

For summary,

1. The event_reset() is called from set_terminal() in every 'set terminal' command.
2. Since (! interactive) is true in running script, ungetc() is called for 'set terminal x11' (or wxt, or qt).

Finally, my solution is like this.

@@ -2425,7 +2424,7 @@ event_reset(struct gp_event_t *ge)
      * FIXME (June 2020):
      * A change to readline.c:getc_wrapper may have made this unnecessary.
      */
-    if (paused_for_mouse || !interactive) {
+    if (paused_for_mouse && !interactive) {
        if (term && (!strncmp("x11",term->name,3)
                 || !strncmp("wxt",term->name,3)
                 || !strncmp("qt",term->name,2)))

[Ethan Merritt]

I am still confused. What error case is this a solution for? The call to ungetc() was added to fix cases of swallowed characters. As the comment says, it may no longer be necessary during interactive use; I am not sure. But I am not aware of a case where it causes a swallowed character. How could it?

Are you saying that you can reproduce the original test case provided by Simon Lutz gnuplot pause.gnuplot and this patch fixes it? I cannot reproduce the original problem, but I am reasonably certain that removing the ungetc() from the general non-interactive case will break other existing scripts. See bug #2134.

Is it possible that the problem you are seeing is only present on OSX? Perhaps what is needed is something like

#ifdef OSX
if (paused_for_mouse)
#else
if (paused_for_mouse || !interactive)
#endif

Note: I don't know what is the correct symbol to test for in #ifdef.
If this does fix something for you, can you try to see whether it still works for the problem case reported in #2134?

[Hiroki Motoyoshi]

What error case is this a solution for?

For the bug (#2292) reported by Simon.

If this does fix something for you, can you try to see whether it still works for the problem case reported in #2134?

As you guessed, my solution in previous message brought back #2134. Sorry for the confusion.

But I am not aware of a case where it causes a swallowed character. How could it?

To be precise, this bug isn't caused by swallowing character, but rather by "\n" being held in the input buffer that you didn't enter. A "\n" is pushed into stdin buffer at each calling of "set terminal x11|wxt|qt" in non-interactive mode. Once "\n" is pushed and then "pause -1" is called (after plot command), fgetc() will read "\n" immediately from the input buffer before the user enters "\n". I think Simon was referring to this as "pause -1 swallows plots" because the plot disappears so quickly.

Are you saying that you can reproduce the original test case provided by Simon Lutz gnuplot pause.gnuplot and this patch fixes it?

Yes, I am. I could reproduce the original problem using pause.gnuplot by Simon on MacOS + gnuplot 5.5 and CentOS + gnuplot 5.2.7. I don't think this bug is specific to MacOS.

The way that ungetc() is called in "set terminal" is described in the previous message. The reset_event() is called only from set_terminal() or the event handler of window closing. I think the patch using term_initialised is still a candidate for a solution. This one does not seem to have any impact on #2134.

FYI, if you have the following code in ~/.gnuplot, you can see that many "\n" is entered to prompt even in interactive mode.

do for [i=1:10] {
  set terminal x11
}

[Ethan Merritt]

To be precise, this bug isn't caused by swallowing character, but rather by "\n" being held in the input buffer that you didn't enter. A "\n" is pushed into stdin buffer at each calling of "set terminal x11|wxt|qt" in non-interactive mode. Once "\n" is pushed and then "pause -1" is called (after plot command), fgetc() will read "\n" immediately from the input buffer before the user enters "\n". I think Simon was referring to this as "pause -1 swallows plots" because the plot disappears so quickly.

Thanks. I think I am now caught up to you. I agree that it makes sense to suppress the ungetc() if the call came directly from "set terminal". I will look at the term_initialised patch tomorrow to see if I can find any corner cases where it causes a problem.

[Simon Lutz]

May I ask if this problem will be solved, e.g., in Gnuplot version 5.5?
(It is still present in 5.4.1.)
Thank you very much and all the best!

[Ethan Merritt]

Hiroki Motoyoshi's patch was committed for version 5.5 on 11 Nov 2020. Does that solve the problem for you or not? If it does, then it can be back-ported for version 5.4.2 (expected May/June). If not, then I guess we're still waiting for a solution.