Menu
▾
▴
1 Attachments
#1401 changes needed in order to support wxWidgets 3.0
hi, if the recent cvs version is complied with wxwidgets 3.0 installed on the computer, the resulting binary crashed with the first plot to the wxt terminal. (Crash report is appended.)
This happens regardless of wxwidgets 2.8 also being installed or not. Only after removing the 3.0 package can i make a working build with wxt.
Is wxwidgets 3.0 supposed to work?
Googling on that error message suggests that it is recent breakage in some system library rather than something specific to gnuplot. For instance this thread
http://stackoverflow.com/questions/23199699/glib-critical-source-id-xxx-was-not-found-when-attempting-to-remove-it
blames it on glib 2.39
What glib version do you have?
I´ve got glib 2.42.
Judging from the Glib changelog cited for example here
https://groups.google.com/forum/#!topic/vim_dev/Pmxb8swH1yE
it seems like there is some change in gnuplot (or wxt?) required to fix this, but it seems there are a number of followup fixes in glib, too.
Btw., the warning now only appears if i do something with the menubar in the wxt terminal. Running through all.dem does not trigger it.
~~~~~~
Another fix for the wxt terminal linked against wxWidgets 3.0.
wxWidgets + wxgtk version 2.8 were fine with the sequence
delete cairo_bitmap;
cairo_bitmap = new wxBitmap(*image);
but version 3.0 hits a use-after-free that can be attributed to a
UI-triggered event invoking OnPaint(), which tries to access the old
bitmap before the new one has been assigned. Omitting the delete
"cures" this, but then there is a memory leak. Empirical tests seem
to show that it suffices to defer the delete operation until after
the assignment operation. A proper interlock might be better yet.
~~~~~~~
This may be the last broken piece. The deferred deletion fix is in 5.0 and 5.1. I'd rather use a mutex or something, but this fix does seem to work on my test machine.
I'm adding to the conversation here for reference.
The following:
implies -lX11 be added to linkage because wx-config can't anticipate the inclusion of XInitThreads(). Hence, several of us are having to manually add such a thing when doing ./configure. The -lX11 really should be done as part of configure, if I'm understanding the issue correctly.
Interestingly, I found a similar discussion in the Qt realm:
https://stackoverflow.com/questions/31952711/threading-pyqt-crashes-with-unknown-request-in-queue-while-dequeuing
It's interesting, in particular:
"The X11 windowing functions are apparently not threadsafe unless explicitly set to be so, and for whatever reason PyQt doesn't automatically set them to be."
The solution for Qt is a bunch of application options, namely QtCore.Qt.AA_X11InitThreads discussed here:
http://pyqt.sourceforge.net/Docs/PyQt4/qt.html#ApplicationAttribute-enum
Is there any sort of equivalent methodology for wxWidgets?
And here is a discussion about wxWidgets not allowing graphics from a secondary thread (Qt is the same way):
http://wxwidgets.10942.n7.nabble.com/wxWidget3-0-1-gtk-Error-this-is-a-multi-threaded-client-and-XInitThreads-has-not-been-called-td87717.html
Is it possible that somewhere in the WXT terminal the secondary thread is doing something graphics related?
Last edit: Dan Sebald 2017-07-05
Attached is a patch that will include X11 library in the WX libraries if XInitThreads() is required and X11 library is available (inherently in LIBRARIES_FOR_X).
The problem is that wxgtk stupidly insists on the XInitThreads business even if the program doesn't use X11. So making it conditional on LIBRARIES_FOR_X doesn't resolve the original problem if you configure like this:
(--without-gd is there because the gd configure tool actually gets this right and pulls in -lX11 if libgd wants it. If the wxgtk configure tool did the same we wouldn't have this problem).
That configuration seems to work for me. I have
(What release do you have, if it is 2.9 maybe it is only that version that has a XInitThreads dependency.)
If I use that configuration and make any change like the following:
wx_gui compiles. So that line is not being included.
I see your point. But, the user is requesting no X11, the consequence of which is a crash. We could drop the "&& defined(X11)" above in which case configuration will succeed but compilation then fail. That's probably no better, so in addition to that we could place an error in configure.ac when wx_needs_xinitthreads is "yes" and X11 is not present.
Given the single-thread test suggested through the discussion list, in which XInitThreads is not required for plot success, I'm leaning toward the initial problem here being that GUI operations are done in a secondary thread, which wxWidgets documentation suggests is precarious.
Reading the comments in wx_gui.cpp, namely:
I understand now that this macro
/ Create a new application object /
IMPLEMENT_APP_NO_MAIN(wxtApp)
means that gnuplot's main() is used as the main() for wxWidgets app. In other words, the WXT terminal essentially makes the whole of gnuplot executable a wxWidgets application. As such, since the wxWidgets main() is suppressed, there is no GUI event loop in the main thread any more. Hence the secondary thread and relaunching wxTheApp->OnRun() within that thread. But we've noted the potential problems with using GUI commands in a secondary thread.
I think that eventually WXT has to be moved out as its own outboard driver similar to what Qt terminal does. Qt terminal establishes a QDataStream between the Qt terminal driver and the outboard gnuplot_qt. That will take a big hunk of code outside gnuplot into "gnuplot_wxt".
Short term, there really is no good solution. It's happenstance that the addition of XInitThreads() enables the wxWidgets secondary thread to handle GUI graphics. However, outside of X-windows, will the secondary thread approach work? (Is there any major OS not using X? Other than Windows, which apparently has a GUI event loop in every main().) As mentioned at the start, forcing wxWidgets into single thread allows plotting without XInitThreads. However, I've found a segmentation fault when trying to do multiple WXT plot windows.
Refactoring wxt into an outboard driver sounds like a lot of work, and there's no guarantee that it would be any less problematic than the current configuration. Basically the current design worked fine through wxWidgets version 2.8. Whether the breakage since then is due to a fundamental design change in wx or "merely" to a lot of bugs, I don't know. Either way the bottom line is for linux you're better off sticking with version 2.8 or dropping wxt altogether and using the qt terminal instead.
FWIW:
Since you ask:
X11 is one of 7 backends for wxWidgets. OSX uses cocoa or carbon. Windows has its own. According to posts on the wxWidgets discussion forum, there were plans to develop a Wayland backend that would not require X11 but the developers working on it left for other projects.
On windows, wxt terminal works fine on wxWidgets 3.0.2.
Bastain suggested wxt terminal is to be default on 5.2.
http://gnuplot.10905.n7.nabble.com/default-terminal-for-version-5-2-on-Windows-td20671.html
Some demos on qt terminal on my enviorments, charactors are desplaied in correct way.
And thus I do not use qt terminal on gnuplot on windows. (I use wxt or windows terminals.)
As T.M. mentions, WXT is much Windows like in look and feel. It has nice fonts too, very clean. The thing is, varied gnuplot terminals has the ability to sort of match other applications. For example, Octave uses a Qt GUI now and selecting gnuplot's Qt terminal makes it appear that the GUI and the plots it creates are the same application (when they really aren't). I can imagine someone writing an application in wxWidgets and wanting to integrate gnuplot WXT plots for a seamless match.
Yes, it would be a lot of work. The good things are:
1) The function naming convention is pretty good so that it's clear what goes in the terminal API (functions wxt_space_etc) and what goes in an outboard program (functions wxCamelEtc).
2) All code within WXT_MULTITHREADED gets dropped because outboard program won't be doing any multi-threaded code, at least not in the way it is currently used. So a the wxt_MutexGuiEnter()...wxt_MutexGuiLeave() simply get tossed.
3) Going back to something more in tune with the conventional IMPLEMENT_APP(wxtApp) as opposed to IMPLEMENT_APP_NO_MAIN(wxtApp) would make programming a little easier. (I think back to the person who originally programmed Qt terminal and how quickly the author seemed to do it by following the X terminal as a pattern.)
4) The nicest result is that WXT and Qt terminals can then run in the same session, switching back and forth.
The bad things are:
1) Even though there is heavy use of wxt_command_push() to currently put action into the secondary thread--such that this wxt_command_push() could be replaced with something like wxt_command_send() to shuffle it across a dataStream to the outboard program--it's not 100% consistent as some of the wxt_space_etc routines directly manipulate frame info and wxWidgets non-gui info that should reside in the outboard program. In other words it's currently a mish-mash of approaches with liberal use of wxt_MutexGuiLeave() scattered about, so some of those wxt_space_etc routines have to have their contents moved over to the outboard program.
2) Definitely would destabilize things for months, the mouse code always being the most fickle.
I'm attaching a ZIP file of a very rudimentary re-arrangement of the code in wxterminal subdirectory moving in the direction of adding a gnuplot_wxt outboard driver. It doesn't do a whole lot, sort of a "hello world" type of thing. But it is a step in that direction if we want to persue this further.
The only sort of "diff" from the repository is in
gnuplot-rearrange_wxterminal_compilation-djs2017jul08.patch
which is the Makefile describing the files in the project. In incorporates file name changes and the project files for gnuplot_wxt.
For consistency with other project naming conventions, think of it this way.
wxt_gui.cpp ---> wxt_term.cpp
wxt_gui.cpp ---> gnuplot_wxt.cpp
wxt_gui.h ---> gnuplot_wxt.h
That is, wxt_gui.cpp currently holds all the code, so I replicated that into two files wxt_term.cpp which behaves pretty much the same way as current, i.e., WXT terminal is integrated into gnuplot still (i.e., no code removed).
I then commented out various parts of gnuplot_wxt.cpp to make it compile and create the program gnuplot_wxt. The main conceptual change is going from
IMPLEMENT_APP_NO_MAIN(wxtApp)
to
IMPLEMENT_APP(wxtApp)
so that gnuplot_wxt has a main() that runs a GUI event loop.
The way to build this is
There is no communication between wxt_term and gnuplot_wxt. Just that sample empty plot to verify that the GUI runs (EDIT) withOUT XInitThreads().
I know it's not a huge step, but it is a start to get the ball rolling. When I find some time again, I'll try establishing a wxt_term-to-gnuplot_wxt connection and put a few "send command" transmissions next to the API calls so that we begin to see the original plot along side a partial plot in gnuplot_wxt. (No sense in tossing away the current plots until gnuplot_wxt is more mature.)
Last edit: Dan Sebald 2017-07-09
wxt works fine with wxWidgets 3.0 on Windows as is. It is also the default terminal on Windows since a long time and people have gotten used to the fact that an "inboard" driver can e.g. still rotate splots in persist mode. An outboard driver surely has its benefits, but also has severe drawbacks. As can be seen with the qt terminal, communication between the core and the driver can be difficult to get right - look at the amount of code needed for qt_waitforinput! Yet, for qt it still doesn't work perfectly yet (on Windows). Also the qt terminal on Windows feels slow to start, because the Qt libraries have not been "pre-loaded".
So I am strongly arguing to keep the state of things as is - at least in Windows.
I forget what the motivation for persist mode is, but why exit if one wants essentially the full featured rotation, etc.? The outboard driver is the model consistent with all other terminals--the historical terminal driver. The amount of code in qt_waitforinput has a lot to do with mousing issues. Perhaps it can be done better. I don't seen any reason that outboard client/server communications can't be made as fast as inboard, as far as the user drag-and-rotate perception is concerned. Qt isn't the issue. The plan isn't to get rid of WXT, but improve it and take the bulky inboard code associated with WXT rendering out of gnuplot... make it leaner again. As I see it, the advantages of removing the code, properly doing graphics in a main thread thereby avoiding inclusion of XInitThreads(), and being able to swap back and forth between WXT, X11 and Qt terminals are pretty big compared to rotatable persist plots.
You misunderstand. Things aren't broken on Windows, but will be with these changes. You can use qt and wxt in the same session there. Note that I am only talking about qt since that is the only outboard driver on Windows. wxt has been inboard ever since its introduction in version 4.2 in 2006. I think 11 years is "historic" enough.
Btw. the slowness of qt start-up (not communication!) stems from the fact that starting a process is expensive if the shared libs are huge and not yet in memory. But I agree that that could be solved by loading gnuplot_qt early. Just some extra work.
So whatever you do, please do not break the Windows builds. We certainly can get things right on Windows even after changing to an outboard driver, but that just feels like a lot of work without real need.
How will the Windows version be broken by making the WXT driver outboard? wxWidgets framework is designed to be cross-platform. The sockets functions are part of the framework, so we shouldn't have to struggle to make that work.
I would have said something long ago had I known that WXT were to be integrated the way it was, and the result of going to an outboard model back then would have probably saved a lot of people a lot of work. There looks to be some tricky code in there for interrupt signal handling
I don't think any of the mutex/sigint code is needed if the model is an outboard driver running with a single thread. In fact, the second comment above points this out explicitly, and I don't know why when the person wrote that comment that there wasn't some thought to overhauling the driver.
As for Qt and the slowly loading libraries, that's Windows I guess. However, you may be encountering some "bug" I've read about in Octave whereby on Windows sometimes Qt takes on the order of ten seconds to load because of some font cache complications:
https://savannah.gnu.org/bugs/?45458
The issue that comment refers to will be a problem regardless of how the primary/secondary threads are assigned. The program needs to respond to both mouse events and to keyboard input. Locking out either one while waiting for the other causes the program to jam. If anything it's a trickier problem for the outboard drivers.
Also Qt is designed to be cross-platform. Stil - as mentioned before - there's a lot of platform-specific code required to simultaneously handle input from stdin and pipes, window messages, console events, the qt event queue, the X11 event queue and then in addition the wxt event queue . None of that is handled by wxWidgets or Qt. (Yet the use of the cross-platform-sockets requires us to link the main executable to Qt or wxWidgets.)
I appreciate your efforts to fix wxt on non-Windows platforms. All I am saying is that the outboard driver will not work without a lot of platform-specific work, not only for handling events. Someone will need to write that code to continue to have a working wxt terminal on Windows if we continue in that direction.
I'd prefer to come up with a cleaner solution for handling multiple input sources than what we do know first. Using threads might be a possibility, but we would have to choose a library for that, too. Not all platforms have posix threads.
I'm not following your reasoning on this Bastian. How is it that the outboard driver is a lot of platform-specific code? The outboard driver should be almost completely wxWidgets, single-threaded, GUI in the main thread--the only thread. Doing so greatly simplifies things.
Let's use Qt as an example of this:
The WIN32 within qt_term.cpp do not count because that is not outboard driver; it's the inboard interface. The WIN32 in gnuplot_qt counts, and there is only one difference--the location of the data directory for Windows. The WIN32 in QtGnuplotWindow.cpp counts, but it has to do with "raise", the feature that Ethan has been suggesting for ages should be done differently. There looks to be hardly any platform-specific code in the Qt outboard driver/application--as is expected of a framework.
A posix thread library for wxWidgets? Multiple input sources? We get away from that by going to outboard driver which needs no additional threads, Windows or otherwise. Because the wxWidgets design doesn't follow the suggested guidelines (link above somewhere), it has made programming difficult. And going forward, as shown by the XInitThreads, it's creating problems working with evolving toolkits. 95% of the code remains the same for Windows because that is already in single-threaded mode, i.e., the code that would be discarded (mutex/thread) is already excluded in Windows.
The usual point of view is that a single process with threads is lighter-weight than separate processes with interprocess communication via pipes or shared memory. You seem to be arguing the opposite. And we would absolutely not get away from having to handle multiple input sources. One way or another, the user-visible program must respond both to events from the plot window (mousing, resize, tool widget operations) and to events from the originating process (keyboard, input pipe, system signals).
I am mostly discussing code like the one in qt_term.cpp! That is the difficult part. An outboard driver without an interface on the gnuplot core side is hardly usable. I do not see how I could that make more clear.
Lightweight in what sense? Less memory? Certainly gnuplot is bigger when the WXT code is included within it. What I'm arguing is that the outboard driver for WXT is the model consistent with X11 (the original interface) and Qt. I recall on the discussion list going back to 90s or 00s about the terminal model that gnuplot uses and not deviating from that.
I never said anything about discarding responding to events from user-visible program. Is there some difference between WXT and Qt that I'm missing? I create a plot in Qt and can use the mouse to zoom, type 'u' to unzoom, etc. Seems to work. Is there something deficient about Qt? Qt must be using some type of socket for duplex communication.
A point can be made clearer by explaining. I look at qt_term.cpp and all these WIN32 pre-processor conditionals are all quite simple, except for the qt_waitforinput(). However, qt_waitforinput() isn't strikingly less complex than the current wxt_waitforinput(). I did say at the start of this, "mouse code always being the most fickle". There's no getting away from that.
In any case, the complexity of qt_waitforinput() doesn't look near in comparison to swapping the sigint handler in and out and dealing with mutexes because of threads.
What is preventing WXT from being more like Qt in design?