From: Mojca M. <moj...@gm...> - 2014-02-07 10:50:14
|
On Sun, Feb 2, 2014 at 8:32 PM, sfeam wrote: > Hi all, > > I recall from several past threads on the list that some people have been > avoiding the qt terminal because its rendering speed, particularly for > interactive 3D plots, was slower than x11 or wxt. Qt was also highly problematic on Mac when resizing windows (no matter how trivial the plot was) because it tried to redraw everything for almost every pixel of a difference (a typical resize would cause redrawing everything some twenty times and cause weird flickering, while X11 was nicely waiting a while (a few seconds?) before attempting to redraw anything). I wanted to test, but it seems that Qt terminal is broken on Mac now. (I'm using Qt4 because Qt5 wasn't ported to MacPorts yet, even though I could switch to system libraries if necessary.) It says: > plot sin(x) Could not connect gnuplot_qt "" . Starting a new one And then it opens two icons for the GUI, none of which is functional. I then need to use Ctrl+C to be able to type anything. And when I exit gnuplot, I need to manually exit from both Qt that have been opened (there are only icons hinting that a program is running and two gnuplot_qt programs are displayed in task manager, but no actual window with the plot is displayed). In the old gnuplot both gnuplot_qt programs would exit while now they both remain open. If I close one of gnuplot_qt while gnuplot is still running, I get Qt terminal communication error: select() error 9 Bad file descriptor which is probably expected. Gnuplot 4.6 works fine with the same installation of Qt (apart from the fact that forking causes it to open two windows and should probably be avoided on Mac at least, but that has always been the case and it's just a minor annoyance, while the latest version doesn't work at all). Mojca |
From: Mojca M. <moj...@gm...> - 2014-02-07 11:37:32
|
On Fri, Feb 7, 2014 at 11:50 AM, Mojca Miklavec wrote: > > I wanted to test, but it seems that Qt terminal is broken on Mac now. The commit that broke gnuplot on Mac seems to be the following one: Author: sfeam <sfeam> Date: Sat Oct 26 05:44:10 2013 +0000 explicit construction and destruction of qt_term data +2013-10-26 Thomas Bleher + + * src/qtterminal/qt_term.cpp: Change the Qt terminal to only create its + data when initialized (to avoid the Static Initialization Order Fiasco) + and to destroy its data in a gnuplot atexit handler. + but it's not a trivial commit and it would take me slightly more time to figure out how it works and what exactly goes wrong. Mojca |
From: Thomas B. <Tho...@gm...> - 2014-02-07 21:03:35
|
* Mojca Miklavec <moj...@gm...> [2014-02-07 12:37]: > On Fri, Feb 7, 2014 at 11:50 AM, Mojca Miklavec wrote: > > > > I wanted to test, but it seems that Qt terminal is broken on Mac now. > > The commit that broke gnuplot on Mac seems to be the following one: > > Author: sfeam <sfeam> > Date: Sat Oct 26 05:44:10 2013 +0000 > > explicit construction and destruction of qt_term data > > > +2013-10-26 Thomas Bleher > + > + * src/qtterminal/qt_term.cpp: Change the Qt terminal to only create its > + data when initialized (to avoid the Static Initialization Order Fiasco) > + and to destroy its data in a gnuplot atexit handler. > + > > but it's not a trivial commit and it would take me slightly more time > to figure out how it works and what exactly goes wrong. Hmm, this is strange. I have no idea how this patch could have broken the Mac build. As a short explanation: before this patch, several variables in qt_term.cpp were initialized statically. For types that have a constructor, this means that their constructor is called before main(). The order of initialization between different files is unspecified. So if one constructor depends on a static field in another file that is also initialized by a constructor, things may crash. This happened on Windows, where some Qt variable was initialized before the Qt library itself. This patch changed the code so that all variables that depend on Qt are initialized only after main() has started (so at a time when Qt is guaranteed to have been initialized). I don't have a Mac, so I can't test anything, unfortunately. Two things come to mind that you could look at: - Does this happen with the newest version of Qt4? It may be a bug that has already been fixed. - Does it work if you start gnuplot_qt manually? The qt terminal can connect to already running programs. To do this: - start gnuplot_qt - determine its pid - start gnuplot - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is replaced by the numerical pid of the gnuplot_qt process - Try to plot something, e.g. "plot x" Hope this helps you. Thomas |
From: Bastian M. <bma...@we...> - 2014-02-11 08:21:48
|
Triggered by the current discussion, I tried to build the qt terminal on Windows using two configurations: MSVC 2008 and Qt 4.8.5 (binary), and MSVC 2012 and Qt 5.2.1 (binary). In both cases, I replaced the waitforinput code by getchar(): if (options != TERM_ONLY_CHECK_MOUSING) return getchar(); else return NUL; Otherwise I would get error messages like this: "Qt terminal communication error: select() error 0 No error". But this was expected as discussed in http://sourceforge.net/p/gnuplot/patches/645/ I did not yet test Jérôme's proposed solution, though. Also, I had to apply the "C89" patch which can be found at http://sourceforge.net/p/gnuplot/patches/484/ The behaviour of the result seems to be similar to that on Mac: Without changes, gnuplot_qt gets started (once), but gnuplot cycles forever trying to get font info. Adding sleep() after StartDetached() and/or increasing the timeout in qt_connectToServer() helps sometimes, but does not reliably solve the problem. Using Thomas' proposed scheme below, the plot window shows up correctly and I can do `plot`s, but mousing is of course inactive. Bastian Am 07.02.2014 22:03, schrieb Thomas Bleher: > > I don't have a Mac, so I can't test anything, unfortunately. > Two things come to mind that you could look at: > > - Does this happen with the newest version of Qt4? It may be a bug that > has already been fixed. > > - Does it work if you start gnuplot_qt manually? The qt terminal can > connect to already running programs. To do this: > - start gnuplot_qt > - determine its pid > - start gnuplot > - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is > replaced by the numerical pid of the gnuplot_qt process > - Try to plot something, e.g. "plot x" > > Hope this helps you. > Thomas > |
From: Mojca M. <moj...@gm...> - 2014-02-08 02:01:15
|
On Fri, Feb 7, 2014 at 10:03 PM, Thomas Bleher wrote: > * Mojca Miklavec [2014-02-07 12:37]: >> On Fri, Feb 7, 2014 at 11:50 AM, Mojca Miklavec wrote: >> > >> > I wanted to test, but it seems that Qt terminal is broken on Mac now. >> >> The commit that broke gnuplot on Mac seems to be the following one: >> >> Author: sfeam <sfeam> >> Date: Sat Oct 26 05:44:10 2013 +0000 >> >> explicit construction and destruction of qt_term data > > Hmm, this is strange. I have no idea how this patch could have broken > the Mac build. Maybe it just helped a deeper problem surface now. The implementation of the Qt terminal never really worked properly on Macs. I might be wrong, but I would blame forking. Gnuplot starts two separate Qt programs/windows, the proper one and a "fake/nonworking" one. Trying to print anything crashes gnuplot (but crash during printing is not really a showstopper) etc. On Qt 4 and Mac OS X >= 10.7 one can hide the second fake window. On Qt 5 I have no clue how to hide it (so users end up with two icons per running gnuplot), but at least printing doesn't crash. It would be great if someone could figure out how to avoid forking (and try to avoid it at least on mac) if that is possible at all. I'm not skilled enough to understand how gnuplot_qt and forking works to be able to change it. > As a short explanation: before this patch, several variables in > qt_term.cpp were initialized statically. For types that have a > constructor, this means that their constructor is called before main(). > The order of initialization between different files is unspecified. So > if one constructor depends on a static field in another file that is > also initialized by a constructor, things may crash. > This happened on Windows, where some Qt variable was initialized before > the Qt library itself. > > This patch changed the code so that all variables that depend on Qt are > initialized only after main() has started (so at a time when Qt is > guaranteed to have been initialized). > > I don't have a Mac, so I can't test anything, unfortunately. > Two things come to mind that you could look at: > > - Does this happen with the newest version of Qt4? It may be a bug that > has already been fixed. Yes. I'm using 4.8.5. (Unfortunately the maintainer of Qt 4 gave up on Qt 5 for some reason – apparently too much work and too many problems when he tried to create a package and nobody else is willing to step in. But I did successfully link gnuplot against a different Qt installation and I could try to repeat that now to check whether current solution works in 5.2. But the sole fact that it fails to work with Qt 4 makes the current situation unacceptable for a release in my opinion.) > - Does it work if you start gnuplot_qt manually? The qt terminal can > connect to already running programs. To do this: > - start gnuplot_qt > - determine its pid > - start gnuplot > - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is > replaced by the numerical pid of the gnuplot_qt process > - Try to plot something, e.g. "plot x" Yes, that works. Mojca |
From: Daniel J S. <dan...@ie...> - 2014-02-08 06:57:07
|
(Ethan, hope you are following below.) On 02/07/2014 08:01 PM, Mojca Miklavec wrote: > On Fri, Feb 7, 2014 at 10:03 PM, Thomas Bleher wrote: >> * Mojca Miklavec [2014-02-07 12:37]: >>> On Fri, Feb 7, 2014 at 11:50 AM, Mojca Miklavec wrote: >>>> >>>> I wanted to test, but it seems that Qt terminal is broken on Mac now. >>> >>> The commit that broke gnuplot on Mac seems to be the following one: >>> >>> Author: sfeam<sfeam> >>> Date: Sat Oct 26 05:44:10 2013 +0000 >>> >>> explicit construction and destruction of qt_term data >> >> Hmm, this is strange. I have no idea how this patch could have broken >> the Mac build. > > Maybe it just helped a deeper problem surface now. The implementation > of the Qt terminal never really worked properly on Macs. I might be > wrong, but I would blame forking. Gnuplot starts two separate Qt > programs/windows, the proper one and a "fake/nonworking" one. Trying > to print anything crashes gnuplot (but crash during printing is not > really a showstopper) etc. On Qt 4 and Mac OS X>= 10.7 one can hide > the second fake window. On Qt 5 I have no clue how to hide it (so > users end up with two icons per running gnuplot), but at least > printing doesn't crash. > > It would be great if someone could figure out how to avoid forking > (and try to avoid it at least on mac) if that is possible at all. I'm > not skilled enough to understand how gnuplot_qt and forking works to > be able to change it. Others have asked for gnuplot to not use fork() for OSX. I think it is ultimately up to someone with an OSX machine to do this. If we could figure out how to do this with threads instead of fork, maybe that is a solution. Fork is a pretty straightforward concept--the existing process is duplicated (with a different PID) and in one process the "fork()" function returns one value, and in the second process it returns a different value. So, removing some lines of code: pid_t pid = fork(); if (pid < 0) PARENT fprintf(stderr, "Forking error\n"); else if (pid == 0) // Child: start the GUI { CHILD // Make sure the forked copy doesn't trash the history file cancel_history(); // Start the gnuplot_qt program QString filename = getenv("GNUPLOT_DRIVER_DIR"); if (filename.isEmpty()) filename = QT_DRIVER_DIR; filename += "/gnuplot_qt"; execlp(filename.toUtf8().data(), "gnuplot_qt", (char*)NULL); fprintf(stderr, "Expected Qt driver: %s\n", filename.toUtf8().data()); perror("Exec failed"); exit(EXIT_FAILURE); } The parent process will get either PID + or PID -1. The child process gets the PID 0. For example, there is no way the line labeled PARENT will ever happen in the child process because if fork() fails the test "if (pid < 0)" will not happen (because there is no such process). The parent process fails the second test ("else if") and continues onward. The child process on the other hand passes the test and proceeds to swap its process with gnuplot_qt via execlp. Mac OSX doesn't like fork, or at least it doesn't like how gnuplot is doing the fork. Is there a better way of doing this than fork? Say threads, or spawn? Before addressing that, let's see if we can make fork() work in OSX, because apparently it does have fork()...but the code we have might not be "fork-safe". Here is the best I can find on this issue: http://stackoverflow.com/questions/18854708/why-is-it-prohivited-to-use-fork-without-exec-in-mac https://mikeash.com/pyblog/friday-qa-2012-01-20-fork-safety.html The second reference is pretty good. If I'm understanding correctly, gnuplot is supposed to not be doing anything that requires a core OS function between the fork() and the exec##() family of functions. I believe the following hunk may be the problem: QString filename = getenv("GNUPLOT_DRIVER_DIR"); if (filename.isEmpty()) filename = QT_DRIVER_DIR; filename += "/gnuplot_qt"; There are a few lines here that are either requesting some system memory or are requesting some environment information...either of which could fall under core OS function. I believe this line: // Make sure the forked copy doesn't trash the history file cancel_history(); is OK because all this routine does is set this variable static char *expanded_history_filename; to NULL. Since it is static it is a memory location within the process and not requiring any sort of OS functionality. SO, with that in mind, Mojca could you try to figure out if a simple mod will make this work under OSX? Try moving that filename hunk of code before the fork, say (I'm probably being sloppy and ignoring some more direct way of creating a character string from a QString): // Start the GUI application void execGnuplotQt() { char *chfilename; // Prep the child filename before issuing the fork // and avoid any non-fork-safe OS action. QString filename = getenv("GNUPLOT_DRIVER_DIR"); if (filename.isEmpty()) filename = QT_DRIVER_DIR; filename += "/gnuplot_qt"; chfilename = malloc(filename.length() + 1); if (chfilename) { *chfilename = filename.toUtf8().data(); // Fork the GUI and exec the gnuplot_qt program in the child process pid_t pid = fork(); if (pid < 0) fprintf(stderr, "Forking error\n"); else if (pid == 0) // Child: start the GUI { // Make sure the forked copy doesn't trash the history file cancel_history(); // Start the gnuplot_qt program execlp(filename.toUtf8().data(), "gnuplot_qt", (char*)NULL); fprintf(stderr, "Expected Qt driver: %s\n", filename.toUtf8().data()); perror("Exec failed"); exit(EXIT_FAILURE); } } free(chfilename); qt_localServerName = "qtgnuplot" + QString::number(pid); qt_gnuplot_qtStarted = true; } Give that a try. You may need to do a bit of debugging to make it compile. Let us know if that solves the common Mac OSX all-caps error. Dan |
From: Daniel J S. <dan...@ie...> - 2014-02-08 07:04:16
|
On 02/08/2014 12:57 AM, Daniel J Sebald wrote: > chfilename = malloc(filename.length() + 1); > if (chfilename) > { > *chfilename = filename.toUtf8().data(); You may need a strcpy there...it's been a while since I've done any C-level memory copying. :-) Dan |
From: sfeam <sf...@us...> - 2014-02-08 18:27:16
|
On Saturday, 08 February 2014 12:57:00 AM Daniel J Sebald wrote: > > Others have asked for gnuplot to not use fork() for OSX. I think it is > ultimately up to someone with an OSX machine to do this. If we could > figure out how to do this with threads instead of fork, maybe that is a > solution. I think you have that backwards. As I understand it, OSX does not like running the graphics display loop in a separate thread but is OK with running it in a separate process. So fork=good, separate thread=bad. As you might expect, the source code for QProcess::startDetached() (at least in the linux version) basically consists of fork+exec. So the change to calling the Qt library routine rather than doing the fork+exec in-line should not have made any functional difference. Of course if the OSX Qt implementation does something wacky in QProcess::startDetached() that's another thing altogether. Mojca: - Do you have GNUPLOT_DRIVER_DIR set to the current build directory? I could imagine that if the new executable forks a copy of an old gnuplot_qt that strange things might happen. If you run with the following trivial patch, does the reported process ID make sense? --- gnuplot/src/qtterminal/qt_term.cpp 2014-01-26 13:57:10.687171903 -0800 +++ gnuplot-cvs/src/qtterminal/qt_term.cpp 2014-02-08 10:25:19.819905362 -0800 @@ -211,9 +211,10 @@ void execGnuplotQt() qint64 pid; qt->gnuplot_qtStarted = QProcess::startDetached(filename, QStringList(), QString(), &pid); - if (qt->gnuplot_qtStarted) + if (qt->gnuplot_qtStarted) { qt->localServerName = "qtgnuplot" + QString::number(pid); - else + qDebug() << "started detached process " << qt->localServerName; + } else qDebug() << "Could not start gnuplot_qt with path" << filename; } Ethan |
From: Daniel J S. <dan...@ie...> - 2014-02-08 20:47:45
|
On 02/08/2014 12:28 PM, sfeam wrote: > On Saturday, 08 February 2014 12:57:00 AM Daniel J Sebald wrote: >> >> Others have asked for gnuplot to not use fork() for OSX. I think it is >> ultimately up to someone with an OSX machine to do this. If we could >> figure out how to do this with threads instead of fork, maybe that is a >> solution. > > I think you have that backwards. > As I understand it, OSX does not like running the graphics display > loop in a separate thread but is OK with running it in a separate process. > So fork=good, separate thread=bad. Ah, you're correct. I remember something like that now, but it is for Qt. Qt only allows graphics in the main thread. Other threads not, hence there needs to be a bit of communication between threads. If that restriction goes back to some OS's restrictions, I'm not sure. Let's avoid such a rework because it doesn't fit the gnuplot term paradigm. I've seen a lot of talk about the fork issue and OSX, so I assumed there was more to it, but apparently it's not such an issue--no more than other flavors of Unix. > As you might expect, the source code for QProcess::startDetached() > (at least in the linux version) basically consists of fork+exec. > So the change to calling the Qt library routine rather than doing the > fork+exec in-line should not have made any functional difference. > Of course if the OSX Qt implementation does something wacky > in QProcess::startDetached() that's another thing altogether. Yes, Qt probably does such a thing behind the scenes. I've looked at the Qt source. The file is called qprocess_unix.cpp, and best I can tell is * If the OS is QNX, then startDetached -> doSpawn and ::doSpawn disallows multithreaded programs from forking (I assume somewhere down the road Qnix spawn is the equivalent of fork/exec) * If the OS is Unix, then startDetached sets up a safe pipe to catch the start of the child startDetached then does fork startDetached then calls a memset (I don't think that requires OS) startDetached then does some signal setup and closes the started pipes startDetached then does a second fork (I'm not quite sure why) startDetached then does all kinds of manipulations of QStrings, argv[] for the process, etc. But it does look to me that all these variables are on the stack. startDetached then calls qt_safe_execv The only OSX-specific code within all of startDetached is this: #ifdef Q_OS_MAC argv[i + 1] = ::strdup(arguments.at(i).toUtf8().constData()); #else argv[i + 1] = ::strdup(arguments.at(i).toLocal8Bit().constData()); #endif so nothing special there. * The qt_safe_exec## set of functions are inline calls of the form EINTR_LOOP(ret, ::execve(filename, argv, envp)); and EINTR_LOOP is nothing special either, just loops until the process exits without having been interrupted So, with all that, I'm surprised that Qt does so much manipulation of data in between the fork and the exec## family of functions and presumably OSX doesn't complain. gnuplot wasn't doing much different with "QString filename" on the stack, so I'm guessing the getenv("GNUPLOT_DRIVER_DIR") between the fork and exec was what OSX complained about in old versions of the code. OK, great. Now which version of gnuplot will have the portable QProcess::startDetached so that I may tell folks who want to know? I see from the ChangeLog this just made it into the source code with the past three weeks. Dan |
From: Mojca M. <moj...@gm...> - 2014-02-08 22:40:21
|
On Sat, Feb 8, 2014 at 7:28 PM, sfeam wrote: > > I think you have that backwards. > As I understand it, OSX does not like running the graphics display > loop in a separate thread but is OK with running it in a separate process. > So fork=good, separate thread=bad. Or maybe Qt on OS X simply has problems with both. > Mojca: - Do you have GNUPLOT_DRIVER_DIR set to the current > build directory? I didn't set that variable anywhere. But I use ./configure --prefix=$PWD/inst make && make install ./inst/bin/gnuplot > I could imagine that if the new executable forks > a copy of an old gnuplot_qt that strange things might happen. The version of gnuplot_qt that is running is the proper one (unless "make install" doesn't work properly). > If you run with the following trivial patch, does the reported > process ID make sense? Yes, it does: Terminal type set to 'qt' gnuplot> plot sin(x) started detached process "qtgnuplot18712" Could not connect gnuplot_qt "" . Starting a new one started detached process "qtgnuplot18714" The two running gnuplot_qt processes have exactly the right PID (and one of them is sometimes reporting 32 EB of shared memory ;), but it doesn't seem to work. > --- gnuplot/src/qtterminal/qt_term.cpp 2014-01-26 13:57:10.687171903 -0800 > +++ gnuplot-cvs/src/qtterminal/qt_term.cpp 2014-02-08 10:25:19.819905362 -0800 > @@ -211,9 +211,10 @@ void execGnuplotQt() > > qint64 pid; > qt->gnuplot_qtStarted = QProcess::startDetached(filename, QStringList(), QString(), &pid); > - if (qt->gnuplot_qtStarted) > + if (qt->gnuplot_qtStarted) { > qt->localServerName = "qtgnuplot" + QString::number(pid); > - else > + qDebug() << "started detached process " << qt->localServerName; > + } else > qDebug() << "Could not start gnuplot_qt with path" << filename; > } Mojca |
From: Daniel J S. <dan...@ie...> - 2014-02-08 22:59:31
|
On 02/08/2014 04:40 PM, Mojca Miklavec wrote: > On Sat, Feb 8, 2014 at 7:28 PM, sfeam wrote: >> >> I think you have that backwards. >> As I understand it, OSX does not like running the graphics display >> loop in a separate thread but is OK with running it in a separate process. >> So fork=good, separate thread=bad. > > Or maybe Qt on OS X simply has problems with both. > >> Mojca: - Do you have GNUPLOT_DRIVER_DIR set to the current >> build directory? > > I didn't set that variable anywhere. But I use > ./configure --prefix=$PWD/inst > make&& make install > ./inst/bin/gnuplot > >> I could imagine that if the new executable forks >> a copy of an old gnuplot_qt that strange things might happen. > > The version of gnuplot_qt that is running is the proper one (unless > "make install" doesn't work properly). > >> If you run with the following trivial patch, does the reported >> process ID make sense? > > Yes, it does: > > Terminal type set to 'qt' > gnuplot> plot sin(x) > started detached process "qtgnuplot18712" > Could not connect gnuplot_qt "" . Starting a new one > started detached process "qtgnuplot18714" > > The two running gnuplot_qt processes have exactly the right PID (and > one of them is sometimes reporting 32 EB of shared memory ;), but it > doesn't seem to work. Well, it's encouraging that the processes for the two created qt_gnuplot instance exist in the OS space. That's further along than you were having with the old version of the code where OSX aborted the process. Now it seems that gnuplot isn't keeping track of the ID in the proper way to access the child process it is creating. That is, > Could not connect gnuplot_qt "" . Starting a new one I'm assuming there should be something meaningful within those double quotes. The problem lies in qt_connectToServer() or the call to qt_connectToServer(). Looking at the code, it seems that the printout above might be in error. I see the following: if (connectToWidget) { qDebug() << "Could not connect to widget" << qt_option->Widget << ". Starting a QtGnuplotApplication"; qt_option->Widget = QString(); qt_connectToServer(qt->localServerName); } // The gnuplot_qt program could not be reached: try to start a new one else { qDebug() << "Could not connect gnuplot_qt" << qt_option->Widget << ". Starting a new one"; execGnuplotQt(); qt_connectToServer(qt->localServerName, false); } Doesn't that seem like the second case (else) shouldn't be using qt_option-<Widget but the localServerName string in your case? (Since you didn't specify to use an existing Widget.) So, I'm guessing that the qt_option->Widget is a null string hence: > Could not connect gnuplot_qt "" . Starting a new one has an empty string when it should be reporting > Could not connect gnuplot_qt "qtgnuplot18712" . Starting a new one The second part of this is that I don't think "qtgnuplot18712" has any meaning to the OS. The OS will probably understand "18712". I'll defer to you and Ethan on that one. Dan |
From: Mojca M. <moj...@gm...> - 2014-02-08 23:26:59
|
On Sat, Feb 8, 2014 at 11:59 PM, Daniel J Sebald wrote: > On 02/08/2014 04:40 PM, Mojca Miklavec wrote: >> On Sat, Feb 8, 2014 at 7:28 PM, sfeam wrote: >>> >>> If you run with the following trivial patch, does the reported >>> process ID make sense? >> >> Yes, it does: >> >> Terminal type set to 'qt' >> gnuplot> plot sin(x) >> started detached process "qtgnuplot18712" >> Could not connect gnuplot_qt "" . Starting a new one >> started detached process "qtgnuplot18714" >> >> The two running gnuplot_qt processes have exactly the right PID (and >> one of them is sometimes reporting 32 EB of shared memory ;), but it >> doesn't seem to work. > > Well, it's encouraging that the processes for the two created qt_gnuplot > instance exist in the OS space. That's further along than you were having > with the old version of the code where OSX aborted the process. Can you please explain what you meant with this? What exactly is mean with "the old version"? (Which version is the old version here?) And what kind of aborting are you talking about? > Now it seems that gnuplot isn't keeping track of the ID in the proper way to > access the child process it is creating. That is, > > >> Could not connect gnuplot_qt "" . Starting a new one > > > I'm assuming there should be something meaningful within those double > quotes. The problem lies in qt_connectToServer() or the call to > qt_connectToServer(). > > Looking at the code, it seems that the printout above might be in error. I > see the following: > > if (connectToWidget) > { > qDebug() << "Could not connect to widget" << > qt_option->Widget << ". Starting a QtGnuplotApplication"; > qt_option->Widget = QString(); > qt_connectToServer(qt->localServerName); > } > // The gnuplot_qt program could not be reached: try to start > a new one > else > { > qDebug() << "Could not connect gnuplot_qt" << > qt_option->Widget << ". Starting a new one"; > execGnuplotQt(); > qt_connectToServer(qt->localServerName, false); > } > > Doesn't that seem like the second case (else) shouldn't be using > qt_option-<Widget but the localServerName string in your case? If I change that to qDebug() << "Could not connect gnuplot_qt" << qt->localServerName << ". Starting a new one"; I indeed get gnuplot> plot sin(x) started detached process "qtgnuplot20164" Could not connect gnuplot_qt "qtgnuplot20164" . Starting a new one started detached process "qtgnuplot20166" so your observation makes sense. (But of course it doesn't affect functionality.) > The second part of this is that I don't think "qtgnuplot18712" has any > meaning to the OS. The OS will probably understand "18712". I'll defer to > you and Ethan on that one. Please note that GDB says 261 while((qt->socket.state() != QLocalSocket::ConnectedState) && (QDateTime::currentDateTime() < timeout)); (gdb) p qt->socket.state() $19 = QLocalSocket::ConnectedState So it probably does connect somehow. Mojca PS: It would be really nice if someone with sufficient knowledge of Qt would create a tiny app where gnuplot would already start as a Qt application similar to Terminal and simply open Qt windows from there (no need for support for wxWidgets or X11 in that mode). Something very similar to what wgnuplot.exe does on Windows. Yes, that would be another mode of operation, but it would be a very useful one and would make distribution of binaries for Mac a lot easier for example. |
From: Daniel J S. <dan...@ie...> - 2014-02-08 23:50:02
|
On 02/08/2014 05:26 PM, Mojca Miklavec wrote: > On Sat, Feb 8, 2014 at 11:59 PM, Daniel J Sebald wrote: >> On 02/08/2014 04:40 PM, Mojca Miklavec wrote: >>> On Sat, Feb 8, 2014 at 7:28 PM, sfeam wrote: >>>> >>>> If you run with the following trivial patch, does the reported >>>> process ID make sense? >>> >>> Yes, it does: >>> >>> Terminal type set to 'qt' >>> gnuplot> plot sin(x) >>> started detached process "qtgnuplot18712" >>> Could not connect gnuplot_qt "" . Starting a new one >>> started detached process "qtgnuplot18714" >>> >>> The two running gnuplot_qt processes have exactly the right PID (and >>> one of them is sometimes reporting 32 EB of shared memory ;), but it >>> doesn't seem to work. >> >> Well, it's encouraging that the processes for the two created qt_gnuplot >> instance exist in the OS space. That's further along than you were having >> with the old version of the code where OSX aborted the process. > > Can you please explain what you meant with this? What exactly is mean > with "the old version"? (Which version is the old version here?) And > what kind of aborting are you talking about? The posts from a day or two ago were referring to the gnuplot version before QProcess::startDetached was introduced, and now we are referring to the gnuplot version after QProcess::startDetached was introduced. From what I've seen elsewhere, the older (pre startDetached) used to launch the Qt child process and then get a complaint like the following from OSX: __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___ >> Doesn't that seem like the second case (else) shouldn't be using >> qt_option-<Widget but the localServerName string in your case? > > If I change that to > qDebug()<< "Could not connect gnuplot_qt"<< qt->localServerName > << ". Starting a new one"; > I indeed get > > gnuplot> plot sin(x) > started detached process "qtgnuplot20164" > Could not connect gnuplot_qt "qtgnuplot20164" . Starting a new one > started detached process "qtgnuplot20166" > > so your observation makes sense. (But of course it doesn't affect > functionality.) Chipping away at things then... >> The second part of this is that I don't think "qtgnuplot18712" has any >> meaning to the OS. The OS will probably understand "18712". I'll defer to >> you and Ethan on that one. > > Please note that GDB says > > 261 while((qt->socket.state() != QLocalSocket::ConnectedState)&& > (QDateTime::currentDateTime()< timeout)); > (gdb) p qt->socket.state() > $19 = QLocalSocket::ConnectedState > > So it probably does connect somehow. OK, hard for me to tell what is happening without a bit of trial-and-error. Dan |
From: sfeam <sf...@us...> - 2014-02-08 04:24:11
|
On Saturday, 08 February 2014 03:01:09 AM Mojca Miklavec wrote: > On Fri, Feb 7, 2014 at 10:03 PM, Thomas Bleher wrote: > > - Does it work if you start gnuplot_qt manually? The qt terminal can > > connect to already running programs. To do this: > > - start gnuplot_qt > > - determine its pid > > - start gnuplot > > - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is > > replaced by the numerical pid of the gnuplot_qt process > > - Try to plot something, e.g. "plot x" > > Yes, that works. > > Mojca That's kind of unexpected. In that case it seems that all you need to do is figure out what differs in those two initialization paths, and the fix should be evident. Could it be as simple as timing? Maybe just wait 1 second before leaving execGnuplotQt(). If that works we can worry about changing it to a proper test that the newly created process is now running. Ethan |
From: Mojca M. <moj...@gm...> - 2014-02-08 11:34:31
|
On Sat, Feb 8, 2014 at 5:26 AM, sfeam wrote: > On Saturday, 08 February 2014 03:01:09 AM Mojca Miklavec wrote: >> On Fri, Feb 7, 2014 at 10:03 PM, Thomas Bleher wrote: > >> > - Does it work if you start gnuplot_qt manually? The qt terminal can >> > connect to already running programs. To do this: >> > - start gnuplot_qt >> > - determine its pid >> > - start gnuplot >> > - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is >> > replaced by the numerical pid of the gnuplot_qt process >> > - Try to plot something, e.g. "plot x" >> >> Yes, that works. >> >> Mojca > > That's kind of unexpected. Oh, and to make it more fun: this works after the "problematic" commit. It doesn't work with the latest version from CVS any longer. I can try to figure out which version broke *that* ;) Mojca |
From: Jérôme L. <lod...@us...> - 2014-02-08 07:19:20
|
Le samedi 8 février 2014 03:01:09 Mojca Miklavec a écrit : > It would be great if someone could figure out how to avoid forking > (and try to avoid it at least on mac) if that is possible at all. I'm > not skilled enough to understand how gnuplot_qt and forking works to > be able to change it. Note that in the current CVS, forking is not used anymore to start gnuplot_qt. Instead, the Qt function QProcess::startDetached is used. |
From: Daniel J S. <dan...@ie...> - 2014-02-08 07:27:50
|
On 02/08/2014 01:05 AM, Jérôme Lodewyck wrote: > Le samedi 8 février 2014 03:01:09 Mojca Miklavec a écrit : >> It would be great if someone could figure out how to avoid forking >> (and try to avoid it at least on mac) if that is possible at all. I'm >> not skilled enough to understand how gnuplot_qt and forking works to >> be able to change it. > > Note that in the current CVS, forking is not used anymore to start gnuplot_qt. > Instead, the Qt function QProcess::startDetached is used. Oh man... :-) |
From: Mojca M. <moj...@gm...> - 2014-02-08 11:25:50
|
On Sat, Feb 8, 2014 at 8:05 AM, Jérôme Lodewyck wrote: > Le samedi 8 février 2014 03:01:09 Mojca Miklavec a écrit : >> It would be great if someone could figure out how to avoid forking >> (and try to avoid it at least on mac) if that is possible at all. I'm >> not skilled enough to understand how gnuplot_qt and forking works to >> be able to change it. > > Note that in the current CVS, forking is not used anymore to start gnuplot_qt. > Instead, the Qt function QProcess::startDetached is used. I'm sorry. I was concentrating on the two two versions just before and after the "problematic" commit. I actually remember seeing QProcess::startDetached, but I was working on the old copy from October when testing. I will nevertheless try some of Daniel's hints. Mojca |
From: sfeam <sf...@us...> - 2014-02-08 23:44:08
|
On Saturday, 08 February 2014 04:59:23 PM Daniel J Sebald wrote: > > Looking at the code, it seems that the printout above might be in error. > I see the following: You are right. The debug statement dumps the wrong string. Mojca: Please apply the following additional patch and repeat your experiment: %%%%%%%%%%%%%%%%% --- gnuplot/src/qtterminal/qt_term.cpp 2014-01-26 13:57:10.687171903 -0800 +++ gnuplot-cvs/src/qtterminal/qt_term.cpp 2014-02-08 15:33:45.873131386 -0800 @@ -273,7 +274,7 @@ void qt_connectToServer(const QString& s // The gnuplot_qt program could not be reached: try to start a new one else { - qDebug() << "Could not connect gnuplot_qt" << qt_option->Widget << ". Starting a new one"; + qDebug() << "Could not connect gnuplot_qt" << server << ". Starting a new one"; execGnuplotQt(); qt_connectToServer(qt->localServerName, false); } %%%%%%%%%%%%%%%%%% Now here's an interesting thing. I can get approximately the same behavior under linux that you report under OSX by deliberately setting GNUPLOT_DRIVER_DIR to an incorrect directory. So to be on the safe side, could you please set this environmental variable even though it should be redundant? Another possible test: Change the timeout value on line 259 from qt->socket.waitForConnected(200); to qt->socket.waitForConnected(-1); This could potentially cause gnuplot to hang, but it also might work :-) Ethan |
From: Mojca M. <moj...@gm...> - 2014-02-09 00:01:26
|
On Sun, Feb 9, 2014 at 12:45 AM, sfeam wrote: > On Saturday, 08 February 2014 04:59:23 PM Daniel J Sebald wrote: >> >> Looking at the code, it seems that the printout above might be in error. >> I see the following: > > You are right. The debug statement dumps the wrong string. > > Mojca: > > Please apply the following additional patch and repeat your experiment: > %%%%%%%%%%%%%%%%% > --- gnuplot/src/qtterminal/qt_term.cpp 2014-01-26 13:57:10.687171903 -0800 > +++ gnuplot-cvs/src/qtterminal/qt_term.cpp 2014-02-08 15:33:45.873131386 -0800 > @@ -273,7 +274,7 @@ void qt_connectToServer(const QString& s > // The gnuplot_qt program could not be reached: try to start a new one > else > { > - qDebug() << "Could not connect gnuplot_qt" << qt_option->Widget << ". Starting a new one"; > + qDebug() << "Could not connect gnuplot_qt" << server << ". Starting a new one"; > execGnuplotQt(); > qt_connectToServer(qt->localServerName, false); > } > %%%%%%%%%%%%%%%%%% Yes, this does the right thing (of course it only affects reporting the right value, not the functionality). > Now here's an interesting thing. I can get approximately the same behavior > under linux that you report under OSX by deliberately setting GNUPLOT_DRIVER_DIR > to an incorrect directory. So to be on the safe side, could you please set > this environmental variable even though it should be redundant? I tried setting it to all values I could think of ($PWD/src, $PWD/inst/bin, $PWD/inst/libexec/gnuplot/4.7). It did switch to a different "gnuplot_qt" binary in the first case, but no change. > Another possible test: > Change the timeout value on line 259 from > qt->socket.waitForConnected(200); > to > qt->socket.waitForConnected(-1); > > This could potentially cause gnuplot to hang, but it also might work :-) Gnuplot runs at 99% CPU with or without that change (just gnuplot, gnuplot_qt doesn't consume any CPU at all after the first few seconds). Mojca |
From: sfeam <sf...@us...> - 2014-02-09 00:17:16
|
On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote: > > Another possible test: > > Change the timeout value on line 259 from > > qt->socket.waitForConnected(200); > > to > > qt->socket.waitForConnected(-1); > > > > This could potentially cause gnuplot to hang, but it also might work > > Gnuplot runs at 99% CPU with or without that change (just gnuplot, > gnuplot_qt doesn't consume any CPU at all after the first few > seconds). gnuplot is spinning CPU cycles while waiting for a timeout? The OSX implementation must really suck. Anyhow if you can see the gnuplot_qt process but the waitForConnected fails to return that's probably a huge clue to what's gone wrong. Let me think about this for a while. I'm cc-ing Jérôme Lodewyck. Maybe he can decipher the clue. Ethan > On Saturday, 08 February 2014 03:01:09 AM Mojca Miklavec wrote: > > On Fri, Feb 7, 2014 at 10:03 PM, Thomas Bleher wrote: > > > - Does it work if you start gnuplot_qt manually? The qt terminal can > > > > > > connect to already running programs. To do this: > > > - start gnuplot_qt > > > - determine its pid > > > - start gnuplot > > > - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is > > > > > > replaced by the numerical pid of the gnuplot_qt process > > > > > > - Try to plot something, e.g. "plot x" > > > > Yes, that works. > > > > Mojca > > That's kind of unexpected. > In that case it seems that all you need to do is figure out what > differs in those two initialization paths, and the fix should be evident. > Could it be as simple as timing? > Maybe just wait 1 second before leaving execGnuplotQt(). > If that works we can worry about changing it to a proper test that > the newly created process is now running. > > Ethan |
From: Mojca M. <moj...@gm...> - 2014-02-09 11:11:52
|
On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote: > On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote: >> > Another possible test: >> > Change the timeout value on line 259 from >> > qt->socket.waitForConnected(200); >> > to >> > qt->socket.waitForConnected(-1); >> > >> > This could potentially cause gnuplot to hang, but it also might work >> >> Gnuplot runs at 99% CPU with or without that change (just gnuplot, >> gnuplot_qt doesn't consume any CPU at all after the first few >> seconds). > > gnuplot is spinning CPU cycles while waiting for a timeout? > The OSX implementation must really suck. > Anyhow if you can see the gnuplot_qt process but the waitForConnected > fails to return that's probably a huge clue to what's gone wrong. > Let me think about this for a while. > > I'm cc-ing Jérôme Lodewyck. Maybe he can decipher the clue. I don't know if that's a clue or not, but if I run gdb and manually press "n", only a single gnuplot_qt is started and gnuplot actually returns to the console (if I simply run gnuplot, it runs at 99% CPU "forever"). The first "plot sin(x)" doesn't show anything, but when I plot something for the second time, the plot is actually shown. So maybe there's just a problem of wrong timing/synchronisation/initialisation somewhere after all. The fact is that if I actually get to this point (by manually stepping inside gdb), it works a lot better than it did with forking: - printing doesn't crash - there is no need for the dirty hack removeDockIcon() TransformProcessType(&psn, kProcessTransformToBackgroundApplication); to hide the nofunctional window Mojca |
From: Mojca M. <moj...@gm...> - 2014-02-09 11:29:41
|
On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote: > On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote: >> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote: >>> > Another possible test: >>> > Change the timeout value on line 259 from >>> > qt->socket.waitForConnected(200); >>> > to >>> > qt->socket.waitForConnected(-1); >>> > >>> > This could potentially cause gnuplot to hang, but it also might work >>> >>> Gnuplot runs at 99% CPU with or without that change (just gnuplot, >>> gnuplot_qt doesn't consume any CPU at all after the first few >>> seconds). >> >> gnuplot is spinning CPU cycles while waiting for a timeout? >> The OSX implementation must really suck. >> Anyhow if you can see the gnuplot_qt process but the waitForConnected >> fails to return that's probably a huge clue to what's gone wrong. >> Let me think about this for a while. >> >> I'm cc-ing Jérôme Lodewyck. Maybe he can decipher the clue. > > I don't know if that's a clue or not, but if I run gdb and manually > press "n", only a single gnuplot_qt is started and gnuplot actually > returns to the console (if I simply run gnuplot, it runs at 99% CPU > "forever"). The first "plot sin(x)" doesn't show anything, but when I > plot something for the second time, the plot is actually shown. > > So maybe there's just a problem of wrong > timing/synchronisation/initialisation somewhere after all. > > The fact is that if I actually get to this point (by manually stepping > inside gdb), it works a lot better than it did with forking: > - printing doesn't crash > - there is no need for the dirty hack removeDockIcon() > TransformProcessType(&psn, kProcessTransformToBackgroundApplication); > to hide the nofunctional window And indeed if I change the timeout: --- a/src/qtterminal/qt_term.cpp +++ b/src/qtterminal/qt_term.cpp @@ -251,7 +252,7 // The QLocalSocket::waitForConnected does not respect the time out argument when the // gnuplot_qt application is not yet started. To wait for it, we need to implement the timeout ourselves - QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000); + QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000); do { qt->socket.connectToServer(server); it suddenly almost starts working. I'm saying almost because the first plot doesn't work, but the second one does. Terminal type set to 'qt' gnuplot> plot sin(x) # nothing can be seen started detached process "qtgnuplot31706" gnuplot> plot cos(x) # works gnuplot> It seems that gnuplot gives up too quickly. And if it does give up, the second attempt to connect doesn't work properly, ends up with two instances of gnuplot_qt running (none of them gets closed etc). According to a gdb a lot of that "infinite cycling" (when not properly connected) happens with while (!receivedFontPropos) qt->socket.waitForReadyRead(1000); while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t)) } The following is "sampling" of the running gnuplot at 99% CPU: Call graph: 2510 Thread_2393718 DispatchQueue_1: com.apple.main-thread (serial) + 2510 start (in gnuplot) + 52 [0x100011244] + 2510 main (in gnuplot) + 2796 [0x1000aa62c] plot.c:672 + 2510 com_line (in gnuplot) + 144 [0x100025140] command.c:323 + 2510 do_line (in gnuplot) + 1066 [0x10002559a] command.c:411 + 2510 command (in gnuplot) + 131 [0x100026283] command.c:616 + 2510 plot_command (in gnuplot) + 206 [0x100028d1e] command.c:1559 + 2510 plotrequest (in gnuplot) + 1763 [0x1000abf83] plot2d.c:273 + 2510 eval_plots (in gnuplot) + 27828 [0x1000c00c4] plot2d.c:3259 + 2510 do_plot (in gnuplot) + 131 [0x100062843] graphics.c:517 + 2510 term_start_plot (in gnuplot) + 63 [0x10011476f] term.c:557 + 2510 qt_graphics (in gnuplot) + 669 [0x1001da68d] qt_term.cpp:480 + 1582 qt_sendFont() (in gnuplot) + 772 [0x1001da264] qt_term.cpp:424 + ! 962 QLocalSocket::bytesAvailable() const (in QtNetwork) + 42 [0x1022520fa] + ! : 646 QAbstractSocket::bytesAvailable() const (in QtNetwork) + 16 [0x102245210] + ! : | 646 QIODevice::bytesAvailable() const (in QtCore) + 0,20,... [0x1030389a0,0x1030389b4,...] + ! : 248 QAbstractSocket::bytesAvailable() const (in QtNetwork) + 0,16,... [0x102245200,0x102245210,...] + ! : 68 DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 [0x10226ca8a] + ! 295 QLocalSocket::bytesAvailable() const (in QtNetwork) + 16 [0x1022520e0] + ! : 295 QIODevice::bytesAvailable() const (in QtCore) + 0,85,... [0x1030389a0,0x1030389f5,...] + ! 236 QLocalSocket::bytesAvailable() const (in QtNetwork) + 6,16,... [0x1022520d6,0x1022520e0,...] + ! 89 DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 [0x10226ca8a] + 342 qt_sendFont() (in gnuplot) + 737 [0x1001da241] qt_term.cpp:423 + ! 195 QLocalSocket::waitForReadyRead(int) (in QtNetwork) + 6,51,... [0x1022526c6,0x1022526f3,...] + ! 147 QLocalSocket::waitForReadyRead(int) (in QtNetwork) + 18 [0x1022526d2] + ! 147 QLocalSocket::state() const (in QtNetwork) + 0,8 [0x10224aed0,0x10224aed8] + 180 qt_sendFont() (in gnuplot) + 748,772,... [0x1001da24c,0x1001da264,...] qt_term.cpp:424 + 118 qt_sendFont() (in gnuplot) + 737,713,... [0x1001da241,0x1001da229,...] qt_term.cpp:423 + 110 DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) + 0 [0x1001e38b6] + 97 DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) + 0 [0x1001e38ec] + 47 qt_sendFont() (in gnuplot) + 695 [0x1001da217] qt_term.cpp:421 + 34 qt_sendFont() (in gnuplot) + 999 [0x1001da347] qt_term.cpp:437 2510 Thread_2393731 DispatchQueue_2: com.apple.libdispatch-manager (serial) + 2510 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff8c328316] + 2510 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 [0x7fff8c329786] + 2510 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8b5e27e6] 2510 Thread_2393735: QProcessManager 2510 thread_start (in libsystem_c.dylib) + 13 [0x7fff89671b75] 2510 _pthread_start (in libsystem_c.dylib) + 335 [0x7fff8966e8bf] 2510 QThreadPrivate::start(void*) (in QtCore) + 504 [0x102fad238] 2510 QProcessManager::run() (in QtCore) + 168 [0x10307d7b8] 2510 __select (in libsystem_kernel.dylib) + 10 [0x7fff8b5e1df2] Total number in stack (recursive counted multiple, when >=5): Sort by top of stack, same collapsed (when >= 5): __select (in libsystem_kernel.dylib) 2510 kevent (in libsystem_kernel.dylib) 2510 QIODevice::bytesAvailable() const (in QtCore) 941 qt_sendFont() (in gnuplot) 379 QAbstractSocket::bytesAvailable() const (in QtNetwork) 248 QLocalSocket::bytesAvailable() const (in QtNetwork) 236 QLocalSocket::waitForReadyRead(int) (in QtNetwork) 195 DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) 157 QLocalSocket::state() const (in QtNetwork) 147 DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) 110 DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) 97 Mojca |
From: Daniel J S. <dan...@ie...> - 2014-02-09 18:20:04
|
On 02/09/2014 05:29 AM, Mojca Miklavec wrote: > On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote: >> On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote: >>> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote: >>>>> Another possible test: >>>>> Change the timeout value on line 259 from >>>>> qt->socket.waitForConnected(200); >>>>> to >>>>> qt->socket.waitForConnected(-1); >>>>> >>>>> This could potentially cause gnuplot to hang, but it also might work >>>> >>>> Gnuplot runs at 99% CPU with or without that change (just gnuplot, >>>> gnuplot_qt doesn't consume any CPU at all after the first few >>>> seconds). >>> >>> gnuplot is spinning CPU cycles while waiting for a timeout? >>> The OSX implementation must really suck. >>> Anyhow if you can see the gnuplot_qt process but the waitForConnected >>> fails to return that's probably a huge clue to what's gone wrong. >>> Let me think about this for a while. >>> >>> I'm cc-ing Jérôme Lodewyck. Maybe he can decipher the clue. >> >> I don't know if that's a clue or not, but if I run gdb and manually >> press "n", only a single gnuplot_qt is started and gnuplot actually >> returns to the console (if I simply run gnuplot, it runs at 99% CPU >> "forever"). The first "plot sin(x)" doesn't show anything, but when I >> plot something for the second time, the plot is actually shown. >> >> So maybe there's just a problem of wrong >> timing/synchronisation/initialisation somewhere after all. >> >> The fact is that if I actually get to this point (by manually stepping >> inside gdb), it works a lot better than it did with forking: >> - printing doesn't crash >> - there is no need for the dirty hack removeDockIcon() >> TransformProcessType(&psn, kProcessTransformToBackgroundApplication); >> to hide the nofunctional window > > And indeed if I change the timeout: > > --- a/src/qtterminal/qt_term.cpp > +++ b/src/qtterminal/qt_term.cpp > @@ -251,7 +252,7 > > // The QLocalSocket::waitForConnected does not respect the > time out argument when the > // gnuplot_qt application is not yet started. To wait for it, > we need to implement the timeout ourselves > - QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000); > + QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000); > do > { > qt->socket.connectToServer(server); > > it suddenly almost starts working. I'm saying almost because the first > plot doesn't work, but the second one does. > > Terminal type set to 'qt' > gnuplot> plot sin(x) # nothing can be seen > started detached process "qtgnuplot31706" > gnuplot> plot cos(x) # works > gnuplot> > > It seems that gnuplot gives up too quickly. And if it does give up, > the second attempt to connect doesn't work properly, ends up with two > instances of gnuplot_qt running (none of them gets closed etc). > > According to a gdb a lot of that "infinite cycling" (when not properly > connected) happens with > > while (!receivedFontPropos) > qt->socket.waitForReadyRead(1000); > while (qt->socket.bytesAvailable()>= (int)sizeof(gp_event_t)) > } > > The following is "sampling" of the running gnuplot at 99% CPU: Trying my best to decipher, but I think it's getting close now. In a post that didn't make it here you added that you see this error: "Incorrect NSStringEncoding value 0x0000 detected." Looking at the debugger results below, it seems that qt_term.cpp is having problems at or near lines 423-424: > 748,772,... [0x1001da24c,0x1001da264,...] qt_term.cpp:424 > + 118 qt_sendFont() (in gnuplot) + > 737,713,... [0x1001da241,0x1001da229,...] qt_term.cpp:423 > + 110 and the code hunk around there is: while (!receivedFontPropos) { qt->socket.waitForReadyRead(1000); 423 while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t)) 424 { gp_event_t event; qt->socket.read((char*) &event, sizeof(gp_event_t)); // Here, we discard other events than fontprops. if ((event.type == GE_fontprops) && (event.par1 > 0) && (event.par2 > 0)) { receivedFontPropos = true; metric = QPair<int, int>(event.par1, event.par2); fontMetricCache[currentFont] = metric; break; } } } So, some comments: 1) Ethan was likely correct about the timeout. The Qt documentation indicates the default timeout is 30 seconds. 1 second seems rather short. Especially in this case, as font setting/info is a system library and it is quite common for system level features to take up to five seconds or so to complete especially the first time the are called and loaded resident. Mojca, for a temporary measure to take this issue out of the picture, consider setting the timeout to the default 30 seconds (i.e., remove the argument 1000 from the function or change 1000 to 30000). 2) With the error you pointed out regarding NSStringEncoding, it would seem there might be an added delay because something isn't correct about qt_term's requested font info string. Perhaps the system font library can't find the metric information that is being sought and it takes a while to prep/search the system database. 3) On the other hand, that error you indicated could be what happens when qt->socket.waitForReadyRead(1000); while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t)) fails. (Are you seeing that error when you add the really long wait period?) 4) When the above does fail (i.e., times out), what happens? It looks like QPair<int, int> metric; is uninitialized, or at least defaults to a zero pair. So, if the font info times out, what is the value of metric when used here? term->v_char = qt_oversampling*metric.first; term->h_char = qt_oversampling*metric.second; 5) Timeout should generally be treated a little more robustly especially given that Qt has some ready-made tools for displaying dialog boxes. Perhaps when timeout happens, display a warning dialog about canceling or continuing onward. Of course, that creates issues for gnuplot in non-interactive mode. In any case, this probably isn't critical with regard to the problems in functionality you are seeing. Dan > > Call graph: > 2510 Thread_2393718 DispatchQueue_1: com.apple.main-thread (serial) > + 2510 start (in gnuplot) + 52 [0x100011244] > + 2510 main (in gnuplot) + 2796 [0x1000aa62c] plot.c:672 > + 2510 com_line (in gnuplot) + 144 [0x100025140] command.c:323 > + 2510 do_line (in gnuplot) + 1066 [0x10002559a] command.c:411 > + 2510 command (in gnuplot) + 131 [0x100026283] command.c:616 > + 2510 plot_command (in gnuplot) + 206 [0x100028d1e] > command.c:1559 > + 2510 plotrequest (in gnuplot) + 1763 [0x1000abf83] > plot2d.c:273 > + 2510 eval_plots (in gnuplot) + 27828 > [0x1000c00c4] plot2d.c:3259 > + 2510 do_plot (in gnuplot) + 131 [0x100062843] > graphics.c:517 > + 2510 term_start_plot (in gnuplot) + 63 > [0x10011476f] term.c:557 > + 2510 qt_graphics (in gnuplot) + 669 > [0x1001da68d] qt_term.cpp:480 > + 1582 qt_sendFont() (in gnuplot) + 772 > [0x1001da264] qt_term.cpp:424 > + ! 962 QLocalSocket::bytesAvailable() const > (in QtNetwork) + 42 [0x1022520fa] > + ! : 646 QAbstractSocket::bytesAvailable() > const (in QtNetwork) + 16 [0x102245210] > + ! : | 646 QIODevice::bytesAvailable() > const (in QtCore) + 0,20,... [0x1030389a0,0x1030389b4,...] > + ! : 248 QAbstractSocket::bytesAvailable() > const (in QtNetwork) + 0,16,... [0x102245200,0x102245210,...] > + ! : 68 > DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 > [0x10226ca8a] > + ! 295 QLocalSocket::bytesAvailable() const > (in QtNetwork) + 16 [0x1022520e0] > + ! : 295 QIODevice::bytesAvailable() const > (in QtCore) + 0,85,... [0x1030389a0,0x1030389f5,...] > + ! 236 QLocalSocket::bytesAvailable() const > (in QtNetwork) + 6,16,... [0x1022520d6,0x1022520e0,...] > + ! 89 > DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 > [0x10226ca8a] > + 342 qt_sendFont() (in gnuplot) + 737 > [0x1001da241] qt_term.cpp:423 > + ! 195 QLocalSocket::waitForReadyRead(int) > (in QtNetwork) + 6,51,... [0x1022526c6,0x1022526f3,...] > + ! 147 QLocalSocket::waitForReadyRead(int) > (in QtNetwork) + 18 [0x1022526d2] > + ! 147 QLocalSocket::state() const (in > QtNetwork) + 0,8 [0x10224aed0,0x10224aed8] > + 180 qt_sendFont() (in gnuplot) + > 748,772,... [0x1001da24c,0x1001da264,...] qt_term.cpp:424 > + 118 qt_sendFont() (in gnuplot) + > 737,713,... [0x1001da241,0x1001da229,...] qt_term.cpp:423 > + 110 > DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) + 0 > [0x1001e38b6] > + 97 > DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) + 0 > [0x1001e38ec] > + 47 qt_sendFont() (in gnuplot) + 695 > [0x1001da217] qt_term.cpp:421 > + 34 qt_sendFont() (in gnuplot) + 999 > [0x1001da347] qt_term.cpp:437 > 2510 Thread_2393731 DispatchQueue_2: > com.apple.libdispatch-manager (serial) > + 2510 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff8c328316] > + 2510 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 > [0x7fff8c329786] > + 2510 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8b5e27e6] > 2510 Thread_2393735: QProcessManager > 2510 thread_start (in libsystem_c.dylib) + 13 [0x7fff89671b75] > 2510 _pthread_start (in libsystem_c.dylib) + 335 [0x7fff8966e8bf] > 2510 QThreadPrivate::start(void*) (in QtCore) + 504 [0x102fad238] > 2510 QProcessManager::run() (in QtCore) + 168 [0x10307d7b8] > 2510 __select (in libsystem_kernel.dylib) + 10 [0x7fff8b5e1df2] > > Total number in stack (recursive counted multiple, when>=5): > > Sort by top of stack, same collapsed (when>= 5): > __select (in libsystem_kernel.dylib) 2510 > kevent (in libsystem_kernel.dylib) 2510 > QIODevice::bytesAvailable() const (in QtCore) 941 > qt_sendFont() (in gnuplot) 379 > QAbstractSocket::bytesAvailable() const (in QtNetwork) 248 > QLocalSocket::bytesAvailable() const (in QtNetwork) 236 > QLocalSocket::waitForReadyRead(int) (in QtNetwork) 195 > DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) 157 > QLocalSocket::state() const (in QtNetwork) 147 > DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) 110 > DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) 97 > > Mojca > |
From: Yuriy K. <yu...@gm...> - 2014-02-12 09:24:58
|
Daniel J Sebald wrote: > On 02/09/2014 05:29 AM, Mojca Miklavec wrote: >> On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote: >>> On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote: >>>> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote: >>>>>> Another possible test: >>>>>> Change the timeout value on line 259 from >>>>>> qt->socket.waitForConnected(200); >>>>>> to >>>>>> qt->socket.waitForConnected(-1); >>>>>> >>>>>> This could potentially cause gnuplot to hang, but it also might work >>>>> Gnuplot runs at 99% CPU with or without that change (just gnuplot, >>>>> gnuplot_qt doesn't consume any CPU at all after the first few >>>>> seconds). >>>> gnuplot is spinning CPU cycles while waiting for a timeout? >>>> The OSX implementation must really suck. >>>> Anyhow if you can see the gnuplot_qt process but the waitForConnected >>>> fails to return that's probably a huge clue to what's gone wrong. >>>> Let me think about this for a while. >>>> >>>> I'm cc-ing Jérôme Lodewyck. Maybe he can decipher the clue. >>> I don't know if that's a clue or not, but if I run gdb and manually >>> press "n", only a single gnuplot_qt is started and gnuplot actually >>> returns to the console (if I simply run gnuplot, it runs at 99% CPU >>> "forever"). The first "plot sin(x)" doesn't show anything, but when I >>> plot something for the second time, the plot is actually shown. >>> >>> So maybe there's just a problem of wrong >>> timing/synchronisation/initialisation somewhere after all. >>> >>> The fact is that if I actually get to this point (by manually stepping >>> inside gdb), it works a lot better than it did with forking: >>> - printing doesn't crash >>> - there is no need for the dirty hack removeDockIcon() >>> TransformProcessType(&psn, kProcessTransformToBackgroundApplication); >>> to hide the nofunctional window >> And indeed if I change the timeout: >> >> --- a/src/qtterminal/qt_term.cpp >> +++ b/src/qtterminal/qt_term.cpp >> @@ -251,7 +252,7 >> >> // The QLocalSocket::waitForConnected does not respect the >> time out argument when the >> // gnuplot_qt application is not yet started. To wait for it, >> we need to implement the timeout ourselves >> - QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000); >> + QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000); >> do >> { >> qt->socket.connectToServer(server); >> >> it suddenly almost starts working. I'm saying almost because the first >> plot doesn't work, but the second one does. >> >> Terminal type set to 'qt' >> gnuplot> plot sin(x) # nothing can be seen >> started detached process "qtgnuplot31706" >> gnuplot> plot cos(x) # works >> gnuplot> >> >> It seems that gnuplot gives up too quickly. And if it does give up, >> the second attempt to connect doesn't work properly, ends up with two >> instances of gnuplot_qt running (none of them gets closed etc). >> >> According to a gdb a lot of that "infinite cycling" (when not properly >> connected) happens with >> >> while (!receivedFontPropos) >> qt->socket.waitForReadyRead(1000); >> while (qt->socket.bytesAvailable()>= (int)sizeof(gp_event_t)) >> } >> >> The following is "sampling" of the running gnuplot at 99% CPU: > > Trying my best to decipher, but I think it's getting close now. In a > post that didn't make it here you added that you see this error: > > "Incorrect NSStringEncoding value 0x0000 detected." > > Looking at the debugger results below, it seems that qt_term.cpp is > having problems at or near lines 423-424: > >> 748,772,... [0x1001da24c,0x1001da264,...] qt_term.cpp:424 >> + 118 qt_sendFont() (in gnuplot) + >> 737,713,... [0x1001da241,0x1001da229,...] qt_term.cpp:423 >> + 110 > > and the code hunk around there is: > > while (!receivedFontPropos) > { > qt->socket.waitForReadyRead(1000); > 423 while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Not sure, but this looks very wrong. If this condition ever triggers (there are less than sizeof(...) bytes in socket), it will result in infinite busy-loop (if there are *any* bytes in socket, waitForReadyRead will instantly return, receivedFontProps is not changed, bytesAvailable check will fail again, etc). I think same apply to similar code in qt_waitforinput(). (However, it is unlikely to be related to discussed bug (I expect it can only trigger on gnuplot and gnuplot_qt version/ABI/architecture mismatch). Still, it feels a way too fragile; IMO, there should be at very least mutual validation of sizeof(gp_event_t) at startup). > 424 { > gp_event_t event; > qt->socket.read((char*) &event, sizeof(gp_event_t)); > // Here, we discard other events than fontprops. > if ((event.type == GE_fontprops) && (event.par1 > 0) && (event.par2 > > 0)) > { > receivedFontPropos = true; > metric = QPair<int, int>(event.par1, event.par2); > fontMetricCache[currentFont] = metric; > break; > } > } > } > > So, some comments: > > 1) Ethan was likely correct about the timeout. The Qt documentation > indicates the default timeout is 30 seconds. 1 second seems rather > short. Especially in this case, as font setting/info is a system > library and it is quite common for system level features to take up to > five seconds or so to complete especially the first time the are called > and loaded resident. Mojca, for a temporary measure to take this issue > out of the picture, consider setting the timeout to the default 30 > seconds (i.e., remove the argument 1000 from the function or change 1000 > to 30000). > > 2) With the error you pointed out regarding NSStringEncoding, it would > seem there might be an added delay because something isn't correct about > qt_term's requested font info string. Perhaps the system font library > can't find the metric information that is being sought and it takes a > while to prep/search the system database. > > 3) On the other hand, that error you indicated could be what happens when > > qt->socket.waitForReadyRead(1000); > while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t)) > > fails. (Are you seeing that error when you add the really long wait > period?) > > 4) When the above does fail (i.e., times out), what happens? It looks like > > QPair<int, int> metric; > > is uninitialized, or at least defaults to a zero pair. So, if the font > info times out, what is the value of metric when used here? > > term->v_char = qt_oversampling*metric.first; > term->h_char = qt_oversampling*metric.second; > > 5) Timeout should generally be treated a little more robustly especially > given that Qt has some ready-made tools for displaying dialog boxes. > Perhaps when timeout happens, display a warning dialog about canceling > or continuing onward. Of course, that creates issues for gnuplot in > non-interactive mode. In any case, this probably isn't critical with > regard to the problems in functionality you are seeing. > > Dan > > >> Call graph: >> 2510 Thread_2393718 DispatchQueue_1: com.apple.main-thread (serial) >> + 2510 start (in gnuplot) + 52 [0x100011244] >> + 2510 main (in gnuplot) + 2796 [0x1000aa62c] plot.c:672 >> + 2510 com_line (in gnuplot) + 144 [0x100025140] command.c:323 >> + 2510 do_line (in gnuplot) + 1066 [0x10002559a] command.c:411 >> + 2510 command (in gnuplot) + 131 [0x100026283] command.c:616 >> + 2510 plot_command (in gnuplot) + 206 [0x100028d1e] >> command.c:1559 >> + 2510 plotrequest (in gnuplot) + 1763 [0x1000abf83] >> plot2d.c:273 >> + 2510 eval_plots (in gnuplot) + 27828 >> [0x1000c00c4] plot2d.c:3259 >> + 2510 do_plot (in gnuplot) + 131 [0x100062843] >> graphics.c:517 >> + 2510 term_start_plot (in gnuplot) + 63 >> [0x10011476f] term.c:557 >> + 2510 qt_graphics (in gnuplot) + 669 >> [0x1001da68d] qt_term.cpp:480 >> + 1582 qt_sendFont() (in gnuplot) + 772 >> [0x1001da264] qt_term.cpp:424 >> + ! 962 QLocalSocket::bytesAvailable() const >> (in QtNetwork) + 42 [0x1022520fa] >> + ! : 646 QAbstractSocket::bytesAvailable() >> const (in QtNetwork) + 16 [0x102245210] >> + ! : | 646 QIODevice::bytesAvailable() >> const (in QtCore) + 0,20,... [0x1030389a0,0x1030389b4,...] >> + ! : 248 QAbstractSocket::bytesAvailable() >> const (in QtNetwork) + 0,16,... [0x102245200,0x102245210,...] >> + ! : 68 >> DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 >> [0x10226ca8a] >> + ! 295 QLocalSocket::bytesAvailable() const >> (in QtNetwork) + 16 [0x1022520e0] >> + ! : 295 QIODevice::bytesAvailable() const >> (in QtCore) + 0,85,... [0x1030389a0,0x1030389f5,...] >> + ! 236 QLocalSocket::bytesAvailable() const >> (in QtNetwork) + 6,16,... [0x1022520d6,0x1022520e0,...] >> + ! 89 >> DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) + 0 >> [0x10226ca8a] >> + 342 qt_sendFont() (in gnuplot) + 737 >> [0x1001da241] qt_term.cpp:423 >> + ! 195 QLocalSocket::waitForReadyRead(int) >> (in QtNetwork) + 6,51,... [0x1022526c6,0x1022526f3,...] >> + ! 147 QLocalSocket::waitForReadyRead(int) >> (in QtNetwork) + 18 [0x1022526d2] >> + ! 147 QLocalSocket::state() const (in >> QtNetwork) + 0,8 [0x10224aed0,0x10224aed8] >> + 180 qt_sendFont() (in gnuplot) + >> 748,772,... [0x1001da24c,0x1001da264,...] qt_term.cpp:424 >> + 118 qt_sendFont() (in gnuplot) + >> 737,713,... [0x1001da241,0x1001da229,...] qt_term.cpp:423 >> + 110 >> DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) + 0 >> [0x1001e38b6] >> + 97 >> DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) + 0 >> [0x1001e38ec] >> + 47 qt_sendFont() (in gnuplot) + 695 >> [0x1001da217] qt_term.cpp:421 >> + 34 qt_sendFont() (in gnuplot) + 999 >> [0x1001da347] qt_term.cpp:437 >> 2510 Thread_2393731 DispatchQueue_2: >> com.apple.libdispatch-manager (serial) >> + 2510 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff8c328316] >> + 2510 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 >> [0x7fff8c329786] >> + 2510 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8b5e27e6] >> 2510 Thread_2393735: QProcessManager >> 2510 thread_start (in libsystem_c.dylib) + 13 [0x7fff89671b75] >> 2510 _pthread_start (in libsystem_c.dylib) + 335 [0x7fff8966e8bf] >> 2510 QThreadPrivate::start(void*) (in QtCore) + 504 [0x102fad238] >> 2510 QProcessManager::run() (in QtCore) + 168 [0x10307d7b8] >> 2510 __select (in libsystem_kernel.dylib) + 10 [0x7fff8b5e1df2] >> >> Total number in stack (recursive counted multiple, when>=5): >> >> Sort by top of stack, same collapsed (when>= 5): >> __select (in libsystem_kernel.dylib) 2510 >> kevent (in libsystem_kernel.dylib) 2510 >> QIODevice::bytesAvailable() const (in QtCore) 941 >> qt_sendFont() (in gnuplot) 379 >> QAbstractSocket::bytesAvailable() const (in QtNetwork) 248 >> QLocalSocket::bytesAvailable() const (in QtNetwork) 236 >> QLocalSocket::waitForReadyRead(int) (in QtNetwork) 195 >> DYLD-STUB$$QIODevice::bytesAvailable() const (in QtNetwork) 157 >> QLocalSocket::state() const (in QtNetwork) 147 >> DYLD-STUB$$QLocalSocket::waitForReadyRead(int) (in gnuplot) 110 >> DYLD-STUB$$QLocalSocket::bytesAvailable() const (in gnuplot) 97 |