#1885 wxt terminal segmentation Fault

[Anonymous]

Hi There,

I'm experiencing some "random" segmentation faults when I'm using a scriptfile and when calling gnuplot from a c++ application (through popen() ).

Thanks for the attention.

[Karl Ratzsch]

Reproduces here (ubuntu16.10, gnuplot5.0cvs, also5.1cvs) only with the large canvas size (1200, 800).

set term wxt size 1200,800
plot "-" using 1:2

segfaults after 5-6 tries with the supplied dataset, same with "plot sin(x)" but not as frequently (1 in 15 or so).

No problems with qt, or wxt and smaller canvas. gnuplot is compiled with wtx3.0

[Ethan Merritt]

I can't reproduce it here. I tried scripting a loop and invoking your test script 50 times or so using gnuplot 5.0.3, 5.0.5 or current cvs. I also tried increasing the size of the wxt terminal.

Any chance you can catch a stack trace to provide a hint where the problem occurs?

[Karl Ratzsch]

Not today. I have gnuplot compiled without the wx-single-threaded option. Will also try if that makes a difference

[Ethan Merritt]

I'm not using single-threaded, so that's not the explanation.
FWIW I have wxgtk 3.0.2

[Karl Ratzsch]

gnuplot: ../../../../src/cairo-path-fixed.c:366: _cairo_path_fixed_last_op: Assertion `buf->num_ops != 0' failed.
Abgebrochen (Speicherabzug geschrieben)

I got this report only once in a few dozen tries.

(stupid question from a guy who is not so experienced on the lower levels: Throwing a stack trace is written into the respective program to do in case of an exception, so if i don't see one, that means the crash was so bad the exception didn't catch (or rather there was no exception for this kind of error). Is that about right?)

[Ethan Merritt]

In general a full trace helps because it lets you see what higher level command in the gnuplot core code caused some bad condition in a lower level routine (some cairo library piece in this case). Seeing only the latter doesn't tell us much, or anyhow doesn't offer much of a clue where in the gnuplot code to look.

Having said that, in this particular case the failure is an assertion rather than a "real" segfault. My opinion is that leaving assertions enabled in a production version of the code is a really bad idea. I suffered through at least one distro version of wxt that had the debug assertions enabled in the packaged version and I never did get it to work reliably with gnuplot. Whether it would have worked any better with the assertions disabled is of course unknown but for sure they did not help anything.

As to how to trap exceptions, that depends on what tools you have easily available. If I have a test case that reliably show the fault, I run the program inside gdb or valgrind and wait for the fault to trigger. Neither of those tools requires anything "written into the program".

[Karl Ratzsch]

OK, got this with

 gdb --args gnuplot inputfile.gp
 run
 bt

I'm in unknown water here ...

Thread 1 "gnuplot" received signal SIGSEGV, Segmentation fault.
0xb7295730 in cairo_matrix_transform_point () from /usr/lib/i386-linux-gnu/libcairo.so.2
(gdb) bt
#0  0xb7295730 in cairo_matrix_transform_point () from /usr/lib/i386-linux-gnu/libcairo.so.2
#1  0xb72826a0 in ?? () from /usr/lib/i386-linux-gnu/libcairo.so.2
#2  0xb7279a5b in cairo_move_to () from /usr/lib/i386-linux-gnu/libcairo.so.2
#3  0x0815bd57 in gp_cairo_draw_point (plot=0xb256d144, x1=16991, y1=6066, style=0)
    at wxterminal/gp_cairo.c:1006
#4  0x08149322 in wxtPanel::wxt_cairo_exec_command (this=0xb256cf30, command=...)
    at wxterminal/wxt_gui.cpp:2960
#5  0x08154fd3 in wxtPanel::wxt_cairo_refresh (this=0xb256cf30) at wxterminal/wxt_gui.cpp:2887
#6  0x08156243 in wxt_text () at wxterminal/wxt_gui.cpp:2077
#7  0x0812fbc4 in term_end_plot () at term.c:556
#8  0x08135185 in term_end_plot () at term.c:548
#9  0x080971ee in do_plot (plots=0x8248520, pcount=1) at graphics.c:928
#10 0x080bb861 in eval_plots () at plot2d.c:3388
#11 0x0806aaa7 in command () at command.c:625
#12 do_line () at command.c:420
#13 0x080a625b in load_file (fp=0x8247360, name=0x8246fe0 "inputfile.gp", calltype=4) at misc.c:404
#14 0x08059b0c in main (argc=<optimized out>, argv=0xbffff108) at plot.c:654

[Ethan Merritt]

Hmm. If I'm reading that correctly, it crashed during a refresh operation after the main program was already done with the plot and trying to exit. So it probably doesn't have anything to do with what was actually in the plot or anything in the gnuplot core code. The [x1;y1] coordinates are reasonable for a wxt window with oversampling enabled, so it's strange that the cairo command in progress is a simple cairo_move_to(). I guess the problem is something to do with the timing and order in which wxt and/or cairo is releasing resources of a plot that is in the process of being rendered. That's consistent with only seeing the problem some fraction of the time. It's only a guess, but perhaps it becomes more frequent with a larger canvas size just because the plot takes fractionally longer to render.

I don't think that compiling it single-threaded will help because the command in question is already inside a mutex to protect threads from interfering with each other (wxterminal/wxt_gui.cpp:2077). But you could try that anyhow. I don't have any immediate ideas on what else you could try beyond that, and I can't experiment myself because I can't reproduce the problem.

[Karl Ratzsch]

Yes, still crashes as frequently after building gnuplot --with-wx-single-threaded. I guess the workaround will have to be using the qt terminal for the moment.

I found another symptom: Changing the wxt terminal size interactively does not work reliably unless i close the terminal window before. The outer window size does not change and/or gnuplot crashes. Funnily, the same is true for x11 (no crashes, yet), but not for qt. Other bug?

[Nate Eldredge]

I've been seeing similar random crashes with gnuplot 5.2.4 (as well as earlier versions) on Ubuntu 18.04. I can reproduce with gnuplot -e 'set term wxt; plot sin(x)'. I'm using Xvfb to test because my headless machine is faster. I get crashes on about 2-3% of runs. I have lots of core files that suggest memory or heap corruption (pointers with impossible values, etc). I can reproduce on several different machines, so it's not hardware.

For me, --with-wx-single-threaded fixes it, and in that configuration I was able to do about 30000 trials with no crashes. So, Karl's crashes under single-threaded may either be some other bug that's either been fixed since then, or that I just can't reproduce.

There do seem to be some serious problems with the threaded wxt implementation. valgrind found a use-after-free violation, that I think is due to improper locking when calling gdk. One thread freed something from within gdk_window_process_updates and the other freed the same pointer from within gdk_window_process_all_updates. Indeed, with a debugger I was able to catch both threads within these respective functions at the same time, and one of them hadn't called gdk_threads_enter. This seems bad according to https://www.geany.org/manual/gtk/gtk-faq/x482.html.

I don't have any experience programming for gtk or wx so I don't feel qualified to come up with a proper fix or audit the code's thread safety. But it does seem to me that multithread wx ought to be either fixed (by someone who knows how) or disabled by default.

I am happy to provide more details if anyone is interested.

[Ethan Merritt]

Your findings are consistent with what others have reported. The default has already been changed to single-threaded in the code repository. Unless individual distros override the default that will be what people see in 5.2.5 when it is released later this year.

[Nate Eldredge]

Good to hear, thanks!

By the way, I also tried the original test case (set term wxt size 1200,800; plot "inputfile" using 1:2) under 5.2.4 with wx-single-threaded. 24000 iterations and no segfaults. However, I did get a couple instances of the following error:

The program 'gnuplot' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadWindow (invalid Window parameter)'.
  (Details: serial 1287 error_code 3 request_code 38 minor_code 0)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the --sync command line
   option to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)
Gnuplot exiting abnormally. Trying to execute exit handlers anyway.

So perhaps this is a separate issue to investigate. It could possibly be related to my use of Xvfb.