gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

Just another observation before I test further: it wasn't just a
problem of first-time plot. It was also *sometimes* a problem of the
first plot after closing the plotting window. But that was
semi-random. Sometimes it worked and sometimes it didn't. This could
have been related to whether a new plot was a different one or not or
maybe related to timing.

I would say that it's a problem that
    if (!parent->isVisible())
is hidden inside
    if (s != viewport->size())
because the reverse can be true: c may well be equal to
viewport->size(), but parent may not be visible. I don't know the code
well enough, but it looks like parent is not visible whenever I close
the plotting window (or during the initial plot).

Mojca

Re: Reworked qt terminal is much faster – and fails to work on Mac

[sfeam <sf...@us...>]

On Wednesday, 12 February 2014 10:01:39 PM Mojca Miklavec wrote:
> 
> However if I revert the following commit from
>     2014-01-29 Jérôme Lodewyck Postpone show() in QtGnuplotWindow
> then the first plot displays just fine and then basically everything
> works as expected. 
> Mojca

Based on this and our off-line back and forth debugging, I am pretty
sure that the attached patch will fix your problem that the first plot
is not drawn.  

It is, as you say, a partial reversion of the 2014-01-29 patch.
I have made it conditional on #ifdef __APPLE__ but see below.

 Jérôme:
On Mojca's machine the viewport size always seems to be equal to
the window size on entry.  This means that the resize is not performed
the first time in the existing code.  Couldn't this happen on linux too?

	Ethan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/12/2014 10:14 PM, sfeam wrote:
> On Wednesday, 12 February 2014 08:50:18 PM Daniel J Sebald wrote:
>> On 02/12/2014 02:49 PM, Thomas Bleher wrote:
>>> * Daniel J Sebald<dan...@ie...>   [2014-02-12 06:36]:
>>>> In any case, I suggest adding the sleep as Ethan has done.  There is
>>>> no reason that should be running at 100%.  In fact, there is
>>>> probably a preferred way to do this without polling loops.  I
>>>> learned a little bit about Qt working on Octave and Qt has this
>>>> paradigm of signals and slots and the developers suggest adhering to
>>>> the concept otherwise can get kind of dodgy (not in this simple
>>>> case...but cases where widget IDs are floating about).
>>>>
>>>> signal:  something that a Qt object emits
>>>> slot:  the destination of the signal which can
>>>>          be of any number, e.g., five other
>>>>          objects could connect a slot to a signal
>>>>
>>>> Anyhow, I don't have time to look at this right now, but if one
>>>> looks at the documentation for a Qt socket:
>>>>
>>>> http://qt-project.org/doc/qt-4.8/qlocalsocket.html#connectToServer
>>>>
>>>> it indicates that a signal is emitted when the connection is
>>>> complete and there is a signal emitted when there is an error.  So
>>>> the proper thing to do is to first make connections to the socket
>>>> sort of like the following ("success" and "failed" are custom member
>>>> functions):
>>>>
>>>> connect (createdsocket, SIGNAL (connected ()), watcher, SLOT (success ()));
>>>> connect (createdsocket, SIGNAL (error ()), watcher, SLOT (failed ()));
>>>>
>>>> and then tell the socket to attempt to connect to the server:
>>>>
>>>> createdsocket->connectToServer (name)
>>>>
>>>> There is no need to check in a loop for anything.  Either the socket
>>>> will successfully connect and emit "connected" at which point
>>>> "success()" will get called or the socket will timeout and emit
>>>> "error" at which point "failed()" will get called.
>>>>
>>>> One can get very creative about connections made, the number of
>>>> slots watching a signal, doing this dynamically, etc.  So instead of
>>>> a recursive routine, it might be multiple connections, or
>>>> dynamically reconnect/disconnect in the "failed()" slot.  Etc.
>>>
>>> signals and slots are indeed very nice, but they need a Qt event loop to
>>> work correctly. I don't think it is possible to add the Qt event loop to
>>> gnuplot without major surgery. (Well, you can start a local event loop
>>> using QEventLoop inside a function, but in this case it's just more work
>>> to get the same result we already have).
>>
>> Good point, but we'll see if we can make it work somehow.  It's worth at
>> least a couple hours time to test whether it is feasible.  It would seem
>> that the place to start the Qt event loop along with setting up initial
>> signals/slots would be upon calling "term qt".  The good thing is that
>> the graphics is done in the satellite Qt program.  The bits accessible
>> to gnuplot core don't need graphics, so perhaps that Qt event loop could
>> be run in a different, non-main thread without too much work.
>>
>> Local event loop might work, but that seems like a lot of overhead to
>> communicate every time with a remote program with any sort of high
>> bandwidth.  Local event loops are what dialog boxes can and often use.
>> Overhead isn't an issue there.
>
> I don't agree that an event loop is needed for this particular purpose.
> We don't have a case of asynchronous processes sending each other
> signals at random times.  We just want some way for the daughter
> process to signal back to the parent process "I'm ready now, you can
> proceed".   And we want some way for the parent to wait for this
> signal without burning CPU cycles.   One would think from its name that
> QLocalSocket::waitForConnected() provides exactly this service, but
> evidently it doesn't.    At least in linux it should be possible to have the
> parent call sigwait(), and the daughter call kill(parent_pid, SIGUSR1).
> No need to involve Qt in this.

Well, if there were some way to communicate with the Qt satellite 
program without using any Qt code in the terminal that might be fine. 
But I think a lot of these networking and timer Qt function might need a 
Qt event loop running in order for them to work properly:

"
http://qt-project.org/wiki/ThreadsEventsQObjects

What requires a running event loop?

This isn’t an exhaustive list, but if you have the overall picture, you 
should be able to guess which classes require a running event loop.

     Widgets painting and interaction: [don't have that]
     Timers: [not directly, but if we are asking something to timeout 
maybe internally]
     Networking: all low-level Qt networking classes (QTcpSocket, 
QUdpSocket, QTcpServer, etc.) are asynchronous by design...  [maybe, I'm 
not sure]
"

So the question is whether some of these Qt calls are not working 
properly because there is no event loop.

So, some options might be:

1) Figure out some way of not using Qt in qt_term to access qtgnuplot1234.

2) Set up Qt properly and use the Qt functions.  In addition, the 
signals/slots would get rid of the while loops.  There are easy ways to 
put Qt into a sleep state (via mutex) rather than polling, etc.

Dan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[sfeam <sf...@us...>]

On Wednesday, 12 February 2014 08:50:18 PM Daniel J Sebald wrote:
> On 02/12/2014 02:49 PM, Thomas Bleher wrote:
> > * Daniel J Sebald<dan...@ie...>  [2014-02-12 06:36]:
> >> In any case, I suggest adding the sleep as Ethan has done.  There is
> >> no reason that should be running at 100%.  In fact, there is
> >> probably a preferred way to do this without polling loops.  I
> >> learned a little bit about Qt working on Octave and Qt has this
> >> paradigm of signals and slots and the developers suggest adhering to
> >> the concept otherwise can get kind of dodgy (not in this simple
> >> case...but cases where widget IDs are floating about).
> >>
> >> signal:  something that a Qt object emits
> >> slot:  the destination of the signal which can
> >>         be of any number, e.g., five other
> >>         objects could connect a slot to a signal
> >>
> >> Anyhow, I don't have time to look at this right now, but if one
> >> looks at the documentation for a Qt socket:
> >>
> >> http://qt-project.org/doc/qt-4.8/qlocalsocket.html#connectToServer
> >>
> >> it indicates that a signal is emitted when the connection is
> >> complete and there is a signal emitted when there is an error.  So
> >> the proper thing to do is to first make connections to the socket
> >> sort of like the following ("success" and "failed" are custom member
> >> functions):
> >>
> >> connect (createdsocket, SIGNAL (connected ()), watcher, SLOT (success ()));
> >> connect (createdsocket, SIGNAL (error ()), watcher, SLOT (failed ()));
> >>
> >> and then tell the socket to attempt to connect to the server:
> >>
> >> createdsocket->connectToServer (name)
> >>
> >> There is no need to check in a loop for anything.  Either the socket
> >> will successfully connect and emit "connected" at which point
> >> "success()" will get called or the socket will timeout and emit
> >> "error" at which point "failed()" will get called.
> >>
> >> One can get very creative about connections made, the number of
> >> slots watching a signal, doing this dynamically, etc.  So instead of
> >> a recursive routine, it might be multiple connections, or
> >> dynamically reconnect/disconnect in the "failed()" slot.  Etc.
> >
> > signals and slots are indeed very nice, but they need a Qt event loop to
> > work correctly. I don't think it is possible to add the Qt event loop to
> > gnuplot without major surgery. (Well, you can start a local event loop
> > using QEventLoop inside a function, but in this case it's just more work
> > to get the same result we already have).
> 
> Good point, but we'll see if we can make it work somehow.  It's worth at 
> least a couple hours time to test whether it is feasible.  It would seem 
> that the place to start the Qt event loop along with setting up initial 
> signals/slots would be upon calling "term qt".  The good thing is that 
> the graphics is done in the satellite Qt program.  The bits accessible 
> to gnuplot core don't need graphics, so perhaps that Qt event loop could 
> be run in a different, non-main thread without too much work.
> 
> Local event loop might work, but that seems like a lot of overhead to 
> communicate every time with a remote program with any sort of high 
> bandwidth.  Local event loops are what dialog boxes can and often use. 
> Overhead isn't an issue there.

I don't agree that an event loop is needed for this particular purpose.
We don't have a case of asynchronous processes sending each other
signals at random times.  We just want some way for the daughter
process to signal back to the parent process "I'm ready now, you can
proceed".   And we want some way for the parent to wait for this
signal without burning CPU cycles.   One would think from its name that
QLocalSocket::waitForConnected() provides exactly this service, but 
evidently it doesn't.    At least in linux it should be possible to have the
parent call sigwait(), and the daughter call kill(parent_pid, SIGUSR1).
No need to involve Qt in this.

	Ethan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/12/2014 02:49 PM, Thomas Bleher wrote:
> * Daniel J Sebald<dan...@ie...>  [2014-02-12 06:36]:
>> In any case, I suggest adding the sleep as Ethan has done.  There is
>> no reason that should be running at 100%.  In fact, there is
>> probably a preferred way to do this without polling loops.  I
>> learned a little bit about Qt working on Octave and Qt has this
>> paradigm of signals and slots and the developers suggest adhering to
>> the concept otherwise can get kind of dodgy (not in this simple
>> case...but cases where widget IDs are floating about).
>>
>> signal:  something that a Qt object emits
>> slot:  the destination of the signal which can
>>         be of any number, e.g., five other
>>         objects could connect a slot to a signal
>>
>> Anyhow, I don't have time to look at this right now, but if one
>> looks at the documentation for a Qt socket:
>>
>> http://qt-project.org/doc/qt-4.8/qlocalsocket.html#connectToServer
>>
>> it indicates that a signal is emitted when the connection is
>> complete and there is a signal emitted when there is an error.  So
>> the proper thing to do is to first make connections to the socket
>> sort of like the following ("success" and "failed" are custom member
>> functions):
>>
>> connect (createdsocket, SIGNAL (connected ()), watcher, SLOT (success ()));
>> connect (createdsocket, SIGNAL (error ()), watcher, SLOT (failed ()));
>>
>> and then tell the socket to attempt to connect to the server:
>>
>> createdsocket->connectToServer (name)
>>
>> There is no need to check in a loop for anything.  Either the socket
>> will successfully connect and emit "connected" at which point
>> "success()" will get called or the socket will timeout and emit
>> "error" at which point "failed()" will get called.
>>
>> One can get very creative about connections made, the number of
>> slots watching a signal, doing this dynamically, etc.  So instead of
>> a recursive routine, it might be multiple connections, or
>> dynamically reconnect/disconnect in the "failed()" slot.  Etc.
>
> signals and slots are indeed very nice, but they need a Qt event loop to
> work correctly. I don't think it is possible to add the Qt event loop to
> gnuplot without major surgery. (Well, you can start a local event loop
> using QEventLoop inside a function, but in this case it's just more work
> to get the same result we already have).

Good point, but we'll see if we can make it work somehow.  It's worth at 
least a couple hours time to test whether it is feasible.  It would seem 
that the place to start the Qt event loop along with setting up initial 
signals/slots would be upon calling "term qt".  The good thing is that 
the graphics is done in the satellite Qt program.  The bits accessible 
to gnuplot core don't need graphics, so perhaps that Qt event loop could 
be run in a different, non-main thread without too much work.

Local event loop might work, but that seems like a lot of overhead to 
communicate every time with a remote program with any sort of high 
bandwidth.  Local event loops are what dialog boxes can and often use. 
Overhead isn't an issue there.

Dan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/12/2014 03:24 AM, Yuriy Kaminskiy wrote:
> Daniel J Sebald wrote:

>> Looking at the debugger results below, it seems that qt_term.cpp is
>> having problems at or near lines 423-424:
>>
>>> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>>>       +                       118 qt_sendFont()  (in gnuplot) +
>>> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>>>       +                       110
>>
>> and the code hunk around there is:
>>
>> while (!receivedFontPropos)
>> {
>>     qt->socket.waitForReadyRead(1000);
>> 423  while (qt->socket.bytesAvailable()>= (int)sizeof(gp_event_t))
>                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Not sure, but this looks very wrong. If this condition ever triggers (there are
> less than sizeof(...) bytes in socket), it will result in infinite busy-loop (if
> there are *any* bytes in socket, waitForReadyRead will instantly return,
> receivedFontProps is not changed, bytesAvailable check will fail again, etc).
>
> I think same apply to similar code in qt_waitforinput().
>
> (However, it is unlikely to be related to discussed bug (I expect it can only
> trigger on gnuplot and gnuplot_qt version/ABI/architecture mismatch). Still, it
> feels a way too fragile; IMO, there should be at very least mutual validation of
> sizeof(gp_event_t) at startup).

I agree.  In previous posts I didn't put much thought into "good" fixes 
for these loops because, as you say, it seems too fragile.  If I have 
time this weekend I'll see if I can put something together using 
signals/slots.  It shouldn't be difficult.

Dan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

On Mon, Feb 10, 2014 at 1:57 AM, sfeam wrote:
> On Sunday, 09 February 2014 12:29:34 PM Mojca Miklavec wrote:
...
>> Total number in stack (recursive counted multiple, when >=5):
>>
>> Sort by top of stack, same collapsed (when >= 5):
>>         __select  (in libsystem_kernel.dylib)        2510
>>         kevent  (in libsystem_kernel.dylib)        2510
>>         QIODevice::bytesAvailable() const  (in QtCore)        941
>>         qt_sendFont()  (in gnuplot)        379
>>         QAbstractSocket::bytesAvailable() const  (in QtNetwork)        248
>>         QLocalSocket::bytesAvailable() const  (in QtNetwork)        236
>>         QLocalSocket::waitForReadyRead(int)  (in QtNetwork)        195
>>         DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork)        157
>>         QLocalSocket::state() const  (in QtNetwork)        147
>>         DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot)        110
>>         DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot)        97
>>
>> Mojca
>
> Finally a trace that makes sense in that it involves a recent change.
> This bit belongs to the patch:
>
> 2014-01-26  Jérôme Lodewyck  <lod...@us...>
>
>         * src/qtterminal/qt_term.cpp: Implement font metric caching. This solves
>         a flickering issue when a large number of font changes are called (for
>         example when rotating the world plot in world2.dem).
>
>         * src/qtterminal/QtGnuplotInstance.*: New public function that sends a
>         command to gnuplot and blocks until it receives the answer.
>
> Can you back out just that one set of changes from current CVS and see
> if that makes your qt terminal work?

I don't know if it makes much sense to debug infinite loops when it's
clear that the loops happened when gnuplot wasn't even connected
properly/initialized at startup (the code was executed with badly
initialized gnuplot under assumption that connection was successful).

But if I revert that patch alone, the first plot still doesn't work properly.


However if I revert the following commit from
    2014-01-29 Jérôme Lodewyck Postpone show() in QtGnuplotWindow
then the first plot displays just fine and then basically everything
works as expected.

Mojca

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Thomas B. <Tho...@gm...>]

* Daniel J Sebald <dan...@ie...> [2014-02-12 06:36]:
> In any case, I suggest adding the sleep as Ethan has done.  There is
> no reason that should be running at 100%.  In fact, there is
> probably a preferred way to do this without polling loops.  I
> learned a little bit about Qt working on Octave and Qt has this
> paradigm of signals and slots and the developers suggest adhering to
> the concept otherwise can get kind of dodgy (not in this simple
> case...but cases where widget IDs are floating about).
> 
> signal:  something that a Qt object emits
> slot:  the destination of the signal which can
>        be of any number, e.g., five other
>        objects could connect a slot to a signal
> 
> Anyhow, I don't have time to look at this right now, but if one
> looks at the documentation for a Qt socket:
> 
> http://qt-project.org/doc/qt-4.8/qlocalsocket.html#connectToServer
> 
> it indicates that a signal is emitted when the connection is
> complete and there is a signal emitted when there is an error.  So
> the proper thing to do is to first make connections to the socket
> sort of like the following ("success" and "failed" are custom member
> functions):
> 
> connect (createdsocket, SIGNAL (connected ()), watcher, SLOT (success ()));
> connect (createdsocket, SIGNAL (error ()), watcher, SLOT (failed ()));
> 
> and then tell the socket to attempt to connect to the server:
> 
> createdsocket->connectToServer (name)
> 
> There is no need to check in a loop for anything.  Either the socket
> will successfully connect and emit "connected" at which point
> "success()" will get called or the socket will timeout and emit
> "error" at which point "failed()" will get called.
> 
> One can get very creative about connections made, the number of
> slots watching a signal, doing this dynamically, etc.  So instead of
> a recursive routine, it might be multiple connections, or
> dynamically reconnect/disconnect in the "failed()" slot.  Etc.

signals and slots are indeed very nice, but they need a Qt event loop to
work correctly. I don't think it is possible to add the Qt event loop to
gnuplot without major surgery. (Well, you can start a local event loop
using QEventLoop inside a function, but in this case it's just more work
to get the same result we already have).

Thomas

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Ethan A M. <merritt@u.washington.edu>]

On Wednesday, 12 February, 2014 13:24:38 Yuriy Kaminskiy wrote:
> > 423  while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Not sure, but this looks very wrong. If this condition ever triggers (there are
> less than sizeof(...) bytes in socket), it will result in infinite busy-loop (if
> there are *any* bytes in socket, waitForReadyRead will instantly return,
> receivedFontProps is not changed, bytesAvailable check will fail again, etc).
> 
> I think same apply to similar code in qt_waitforinput().
> 
> (However, it is unlikely to be related to discussed bug (I expect it can only
> trigger on gnuplot and gnuplot_qt version/ABI/architecture mismatch). Still, it
> feels a way too fragile; 


You are correct.  I'll work up a fix.

> IMO, there should be at very least mutual validation of
> sizeof(gp_event_t) at startup).

Maybe, but there are other things that are more likely to go wrong if there
is a version mismatch.

	Ethan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

On Wed, Feb 12, 2014 at 8:25 AM, sfeam wrote:
>
> I have gone ahead and placed a variant of that modification into CVS.
/.../
> Let's see what the Windows/OSX people report.

With the latest version from trunk the second plot works fine. The
first plot doesn't work at all, so there are apparently more fixes
needed.

Given the large amount of different suggestion I'm not exactly sure
what I should try. (Something is telling me that Dan is right about
signals and slots, but I'm not sure how to implement that.)

Which version of gnuplot will have the new qt functionality/patches (=
the code that we are discussing about)?

Mojca

PS: I would be grateful if the patch for TransformProcessType on OS X
would be applied before the sources start diverting too much and one
would need to create the patch again.

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Yuriy K. <yu...@gm...>]

Daniel J Sebald wrote:
> On 02/09/2014 05:29 AM, Mojca Miklavec wrote:
>> On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote:
>>> On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote:
>>>> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote:
>>>>>> Another possible test:
>>>>>> Change the timeout value on line 259 from
>>>>>>                 qt->socket.waitForConnected(200);
>>>>>> to
>>>>>>                 qt->socket.waitForConnected(-1);
>>>>>>
>>>>>> This could potentially cause gnuplot to hang, but it also might work
>>>>> Gnuplot runs at 99% CPU with or without that change (just gnuplot,
>>>>> gnuplot_qt doesn't consume any CPU at all after the first few
>>>>> seconds).
>>>> gnuplot is spinning CPU cycles while waiting for a timeout?
>>>> The OSX implementation must really suck.
>>>> Anyhow if you can see the gnuplot_qt process but the waitForConnected
>>>> fails to return that's probably a huge clue to what's gone wrong.
>>>> Let me think about this for a while.
>>>>
>>>> I'm cc-ing Jérôme Lodewyck.  Maybe he can decipher the clue.
>>> I don't know if that's a clue or not, but if I run gdb and manually
>>> press "n", only a single gnuplot_qt is started and gnuplot actually
>>> returns to the console (if I simply run gnuplot, it runs at 99% CPU
>>> "forever"). The first "plot sin(x)" doesn't show anything, but when I
>>> plot something for the second time, the plot is actually shown.
>>>
>>> So maybe there's just a problem of wrong
>>> timing/synchronisation/initialisation somewhere after all.
>>>
>>> The fact is that if I actually get to this point (by manually stepping
>>> inside gdb), it works a lot better than it did with forking:
>>> - printing doesn't crash
>>> - there is no need for the dirty hack removeDockIcon()
>>>    TransformProcessType(&psn, kProcessTransformToBackgroundApplication);
>>>    to hide the nofunctional window
>> And indeed if I change the timeout:
>>
>> --- a/src/qtterminal/qt_term.cpp
>> +++ b/src/qtterminal/qt_term.cpp
>> @@ -251,7 +252,7
>>
>>          // The QLocalSocket::waitForConnected does not respect the
>> time out argument when the
>>          // gnuplot_qt application is not yet started. To wait for it,
>> we need to implement the timeout ourselves
>> -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
>> +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
>>          do
>>          {
>>                  qt->socket.connectToServer(server);
>>
>> it suddenly almost starts working. I'm saying almost because the first
>> plot doesn't work, but the second one does.
>>
>> Terminal type set to 'qt'
>> gnuplot>  plot sin(x) # nothing can be seen
>> started detached process  "qtgnuplot31706"
>> gnuplot>  plot cos(x) # works
>> gnuplot>
>>
>> It seems that gnuplot gives up too quickly. And if it does give up,
>> the second attempt to connect doesn't work properly, ends up with two
>> instances of gnuplot_qt running (none of them gets closed etc).
>>
>> According to a gdb a lot of that "infinite cycling" (when not properly
>> connected) happens with
>>
>> while (!receivedFontPropos)
>>      qt->socket.waitForReadyRead(1000);
>>      while (qt->socket.bytesAvailable()>= (int)sizeof(gp_event_t))
>> }
>>
>> The following is "sampling" of the running gnuplot at 99% CPU:
> 
> Trying my best to decipher, but I think it's getting close now.  In a 
> post that didn't make it here you added that you see this error:
> 
> "Incorrect NSStringEncoding value 0x0000 detected."
> 
> Looking at the debugger results below, it seems that qt_term.cpp is 
> having problems at or near lines 423-424:
> 
>> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>>      +                       118 qt_sendFont()  (in gnuplot) +
>> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>>      +                       110
> 
> and the code hunk around there is:
> 
> while (!receivedFontPropos)
> {
>    qt->socket.waitForReadyRead(1000);
> 423  while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Not sure, but this looks very wrong. If this condition ever triggers (there are
less than sizeof(...) bytes in socket), it will result in infinite busy-loop (if
there are *any* bytes in socket, waitForReadyRead will instantly return,
receivedFontProps is not changed, bytesAvailable check will fail again, etc).

I think same apply to similar code in qt_waitforinput().

(However, it is unlikely to be related to discussed bug (I expect it can only
trigger on gnuplot and gnuplot_qt version/ABI/architecture mismatch). Still, it
feels a way too fragile; IMO, there should be at very least mutual validation of
sizeof(gp_event_t) at startup).

> 424  {
>      gp_event_t event;
>      qt->socket.read((char*) &event, sizeof(gp_event_t));
>      // Here, we discard other events than fontprops.
>      if ((event.type == GE_fontprops) && (event.par1 > 0) && (event.par2 
>  > 0))
>      {
>        receivedFontPropos = true;
>        metric = QPair<int, int>(event.par1, event.par2);
>        fontMetricCache[currentFont] = metric;
>        break;
>      }
>    }
> }
> 
> So, some comments:
> 
> 1) Ethan was likely correct about the timeout.  The Qt documentation 
> indicates the default timeout is 30 seconds.  1 second seems rather 
> short.  Especially in this case, as font setting/info is a system 
> library and it is quite common for system level features to take up to 
> five seconds or so to complete especially the first time the are called 
> and loaded resident.  Mojca, for a temporary measure to take this issue 
> out of the picture, consider setting the timeout to the default 30 
> seconds (i.e., remove the argument 1000 from the function or change 1000 
> to 30000).
> 
> 2) With the error you pointed out regarding NSStringEncoding, it would 
> seem there might be an added delay because something isn't correct about 
> qt_term's requested font info string.  Perhaps the system font library 
> can't find the metric information that is being sought and it takes a 
> while to prep/search the system database.
> 
> 3) On the other hand, that error you indicated could be what happens when
> 
> qt->socket.waitForReadyRead(1000);
> while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
> 
> fails.  (Are you seeing that error when you add the really long wait 
> period?)
> 
> 4) When the above does fail (i.e., times out), what happens?  It looks like
> 
> QPair<int, int> metric;
> 
> is uninitialized, or at least defaults to a zero pair.  So, if the font 
> info times out, what is the value of metric when used here?
> 
> 	term->v_char = qt_oversampling*metric.first;
> 	term->h_char = qt_oversampling*metric.second;
> 
> 5) Timeout should generally be treated a little more robustly especially 
> given that Qt has some ready-made tools for displaying dialog boxes. 
> Perhaps when timeout happens, display a warning dialog about canceling 
> or continuing onward.  Of course, that creates issues for gnuplot in 
> non-interactive mode.  In any case, this probably isn't critical with 
> regard to the problems in functionality you are seeing.
> 
> Dan
> 
> 
>> Call graph:
>>      2510 Thread_2393718   DispatchQueue_1: com.apple.main-thread  (serial)
>>      + 2510 start  (in gnuplot) + 52  [0x100011244]
>>      +   2510 main  (in gnuplot) + 2796  [0x1000aa62c]  plot.c:672
>>      +     2510 com_line  (in gnuplot) + 144  [0x100025140]  command.c:323
>>      +       2510 do_line  (in gnuplot) + 1066  [0x10002559a]  command.c:411
>>      +         2510 command  (in gnuplot) + 131  [0x100026283]  command.c:616
>>      +           2510 plot_command  (in gnuplot) + 206  [0x100028d1e]
>> command.c:1559
>>      +             2510 plotrequest  (in gnuplot) + 1763  [0x1000abf83]
>>   plot2d.c:273
>>      +               2510 eval_plots  (in gnuplot) + 27828
>> [0x1000c00c4]  plot2d.c:3259
>>      +                 2510 do_plot  (in gnuplot) + 131  [0x100062843]
>> graphics.c:517
>>      +                   2510 term_start_plot  (in gnuplot) + 63
>> [0x10011476f]  term.c:557
>>      +                     2510 qt_graphics  (in gnuplot) + 669
>> [0x1001da68d]  qt_term.cpp:480
>>      +                       1582 qt_sendFont()  (in gnuplot) + 772
>> [0x1001da264]  qt_term.cpp:424
>>      +                       ! 962 QLocalSocket::bytesAvailable() const
>>   (in QtNetwork) + 42  [0x1022520fa]
>>      +                       ! : 646 QAbstractSocket::bytesAvailable()
>> const  (in QtNetwork) + 16  [0x102245210]
>>      +                       ! : | 646 QIODevice::bytesAvailable()
>> const  (in QtCore) + 0,20,...  [0x1030389a0,0x1030389b4,...]
>>      +                       ! : 248 QAbstractSocket::bytesAvailable()
>> const  (in QtNetwork) + 0,16,...  [0x102245200,0x102245210,...]
>>      +                       ! : 68
>> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
>> [0x10226ca8a]
>>      +                       ! 295 QLocalSocket::bytesAvailable() const
>>   (in QtNetwork) + 16  [0x1022520e0]
>>      +                       ! : 295 QIODevice::bytesAvailable() const
>> (in QtCore) + 0,85,...  [0x1030389a0,0x1030389f5,...]
>>      +                       ! 236 QLocalSocket::bytesAvailable() const
>>   (in QtNetwork) + 6,16,...  [0x1022520d6,0x1022520e0,...]
>>      +                       ! 89
>> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
>> [0x10226ca8a]
>>      +                       342 qt_sendFont()  (in gnuplot) + 737
>> [0x1001da241]  qt_term.cpp:423
>>      +                       ! 195 QLocalSocket::waitForReadyRead(int)
>> (in QtNetwork) + 6,51,...  [0x1022526c6,0x1022526f3,...]
>>      +                       ! 147 QLocalSocket::waitForReadyRead(int)
>> (in QtNetwork) + 18  [0x1022526d2]
>>      +                       !   147 QLocalSocket::state() const  (in
>> QtNetwork) + 0,8  [0x10224aed0,0x10224aed8]
>>      +                       180 qt_sendFont()  (in gnuplot) +
>> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>>      +                       118 qt_sendFont()  (in gnuplot) +
>> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>>      +                       110
>> DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot) + 0
>> [0x1001e38b6]
>>      +                       97
>> DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot) + 0
>> [0x1001e38ec]
>>      +                       47 qt_sendFont()  (in gnuplot) + 695
>> [0x1001da217]  qt_term.cpp:421
>>      +                       34 qt_sendFont()  (in gnuplot) + 999
>> [0x1001da347]  qt_term.cpp:437
>>      2510 Thread_2393731   DispatchQueue_2:
>> com.apple.libdispatch-manager  (serial)
>>      + 2510 _dispatch_mgr_thread  (in libdispatch.dylib) + 54  [0x7fff8c328316]
>>      +   2510 _dispatch_mgr_invoke  (in libdispatch.dylib) + 923
>> [0x7fff8c329786]
>>      +     2510 kevent  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e27e6]
>>      2510 Thread_2393735: QProcessManager
>>        2510 thread_start  (in libsystem_c.dylib) + 13  [0x7fff89671b75]
>>          2510 _pthread_start  (in libsystem_c.dylib) + 335  [0x7fff8966e8bf]
>>            2510 QThreadPrivate::start(void*)  (in QtCore) + 504  [0x102fad238]
>>              2510 QProcessManager::run()  (in QtCore) + 168  [0x10307d7b8]
>>                2510 __select  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e1df2]
>>
>> Total number in stack (recursive counted multiple, when>=5):
>>
>> Sort by top of stack, same collapsed (when>= 5):
>>          __select  (in libsystem_kernel.dylib)        2510
>>          kevent  (in libsystem_kernel.dylib)        2510
>>          QIODevice::bytesAvailable() const  (in QtCore)        941
>>          qt_sendFont()  (in gnuplot)        379
>>          QAbstractSocket::bytesAvailable() const  (in QtNetwork)        248
>>          QLocalSocket::bytesAvailable() const  (in QtNetwork)        236
>>          QLocalSocket::waitForReadyRead(int)  (in QtNetwork)        195
>>          DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork)        157
>>          QLocalSocket::state() const  (in QtNetwork)        147
>>          DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot)        110
>>          DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot)        97

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/12/2014 01:25 AM, sfeam wrote:
> On Tuesday, 11 February 2014 10:15:24 PM Thomas Bleher wrote:
>> * Ethan A Merritt<sf...@us...>  [2014-02-11 21:35]:
>>
>>> To see why, change it to this:
>>> 	qDebug()<<  "qt_connectToServer "<<  server;
>>>          do
>>>          {
>>>                  qt->socket.connectToServer(server);
>>>                  if (!qt->socket.waitForConnected(TIMEOUT))
>>>                          qDebug()<<  qt->socket.errorString();
>>>          	usleep(50000);
>>> 	}
>>>          while(...)
>>>
>>> Regardless of the value of TIMEOUT, including -1 or blank,  the
>>> waitForConnected returns immediately with an error:
>>>           "QLocalSocket::connectToServer: Invalid name"
>>
>> [Note: I haven't looked at this in detail, so these are just my guesses]
>>
>> I think the error is probably a race condition:
>> - gnuplot starts the external gnuplot_qt process
>> - Immediately afterwards it tries to connect to the socket from
>>    gnuplot_qt
>> - Depending on scheduling, gnuplot_qt may have had a chance to run or
>>    not. But in any case it needs to do some initialization first.
>>
>> So my guess is that the "Invalid name" error stems from the fact that
>> during the first iteration of the loop, the named pipe (which is created
>> by gnuplot_qt) is simply not there yet.
>
> That makes sense, but the first attempt to connect _always_ fails,
> even if I explicitly wait for several seconds before invoking waitForConnected().
> So I think it's not just a timing race.  There's something that doesn't
> get initialized until the first connection attempt. So the first attempt fails
> but then the next attempt succeeds.
>
> I have gone ahead and placed a variant of that modification into CVS.
> I think the worst it can do is hang a session that was going to fail anyhow.
> We can still fix the source of the initial failure if we ever find it.
>
>> I also looked through the recent changes to the gnuplot code just now,
>> and noticed that no QApplication object is created anymore in gnuplot.
>> According to the docs, waitForConnected() should also work without an
>> event loop, but I think it would still be interesting to see if the high
>> CPU usage goes away if QCoreApplication is replaced by QApplication.
>> Just my 2cents.
>> Thomas
>
> I don't see any difference here, but then I wasn't seeing CPU-churning
> to begin with.  Let's see what the Windows/OSX people report.
>
> 	Ethan

Here looks like a potential bug:

// Called before a plot to connect to the terminal window, if needed
void qt_connectToServer()
{
	if (!qt)
		return;
	ensureOptionsCreated();

	// Determine to which server we should connect
	bool connectToWidget = !qt_option->Widget.isEmpty();
	QString server = connectToWidget ? qt_option->Widget : qt->localServerName;

	if (qt->socket.state() == QLocalSocket::ConnectedState)
	{
		// Check if we are already connected to the correct server
		if (qt->socket.serverName() == server)
			return;

		// Otherwise disconnect
		qt->socket.disconnectFromServer();
		while (qt->socket.state() == QLocalSocket::ConnectedState)
			qt->socket.waitForDisconnected(1000);
	}

	// Start the gnuplot_qt helper program if not already started
	if (!connectToWidget && !qt->gnuplot_qtStarted)
		execGnuplotQt();

	// Connect to the server, or local server if not available.
	qt_connectToServer(server);
}

At the top of this function is:

	QString server = connectToWidget ? qt_option->Widget : qt->localServerName;

Potentially, localServerName could be an empty string or some valid 
server name.  If localServerName happens to be empty, it's probably 
because the server wasn't connected yet or there was an error in 
connecting.  If that is the case then this test will be true:

	// Start the gnuplot_qt helper program if not already started
	if (!connectToWidget && !qt->gnuplot_qtStarted)
		execGnuplotQt();

But, inside execGnuplotQt() is where the name of qt->localServerName is 
set, so if this line executed "server" will no longer be pertinent. 
Perhaps the first time through "server" is empty and after the valid 
qtgnuplot#### is made the code is requesting for a server with empty string.

Perhaps the code should be more along the lines:

	// Start the gnuplot_qt helper program if not already started
	if (!connectToWidget && !qt->gnuplot_qtStarted)
	{
		execGnuplotQt();
		server = qt->localServerName;
	}

	if (!server.isEmpty())
	{
		// Connect to the server, or local server if not available.
		qt_connectToServer(server);
	}

That isn't very well organized either.

Dan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[sfeam <sf...@us...>]

On Tuesday, 11 February 2014 10:15:24 PM Thomas Bleher wrote:
> * Ethan A Merritt <sf...@us...> [2014-02-11 21:35]:
>
> > To see why, change it to this:
> > 	qDebug() << "qt_connectToServer " << server;
> >         do
> >         {
> >                 qt->socket.connectToServer(server);
> >                 if (!qt->socket.waitForConnected(TIMEOUT))
> >                         qDebug() << qt->socket.errorString();
> >         	usleep(50000);
> > 	}
> >         while(...)
> > 
> > Regardless of the value of TIMEOUT, including -1 or blank,  the
> > waitForConnected returns immediately with an error:
> >          "QLocalSocket::connectToServer: Invalid name" 
> 
> [Note: I haven't looked at this in detail, so these are just my guesses]
> 
> I think the error is probably a race condition:
> - gnuplot starts the external gnuplot_qt process
> - Immediately afterwards it tries to connect to the socket from
>   gnuplot_qt
> - Depending on scheduling, gnuplot_qt may have had a chance to run or
>   not. But in any case it needs to do some initialization first.
> 
> So my guess is that the "Invalid name" error stems from the fact that
> during the first iteration of the loop, the named pipe (which is created
> by gnuplot_qt) is simply not there yet.

That makes sense, but the first attempt to connect _always_ fails,
even if I explicitly wait for several seconds before invoking waitForConnected().
So I think it's not just a timing race.  There's something that doesn't
get initialized until the first connection attempt. So the first attempt fails
but then the next attempt succeeds.

I have gone ahead and placed a variant of that modification into CVS.
I think the worst it can do is hang a session that was going to fail anyhow.
We can still fix the source of the initial failure if we ever find it.

> I also looked through the recent changes to the gnuplot code just now,
> and noticed that no QApplication object is created anymore in gnuplot.
> According to the docs, waitForConnected() should also work without an
> event loop, but I think it would still be interesting to see if the high
> CPU usage goes away if QCoreApplication is replaced by QApplication.
> Just my 2cents.
> Thomas

I don't see any difference here, but then I wasn't seeing CPU-churning
to begin with.  Let's see what the Windows/OSX people report.

	Ethan


>   
> > Without the usleep() it spews hundreds or even thousands of these errors
> > before fallling through to the rest of the code.  The usleep reduces this
> > to a manageable number of error reports. 
> > So the first connection attempts always fail, even under linux, and the
> > failure is immediate.
> > Why is it an invalid name?  I don't know.
> > How does it eventually succeed?  I don't know that either.
> > 
> > Since TIMEOUT is ignored this is becomes a CPU-burning loop,
> > which is what Mojca originally reported.  The usleep should fix that, and a 
> > large enough manual timeout as in  
> >      QDateTime timeout = QDateTime::currentDateTime().addMSecs(30000);
> > may in fact be necessary.  But it would be nice to understand the real
> > reason why the initial connection attempts fail, because maybe we can
> > test and wait on that condition directly and avoid the useless loop here.
> > 
> > 	Ethan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/11/2014 03:15 PM, Thomas Bleher wrote:
> * Ethan A Merritt<sf...@us...>  [2014-02-11 21:35]:
>> On Sunday, 09 February, 2014 16:57:36 sfeam wrote:
>>>> And indeed if I change the timeout:
>>>>
>>>> --- a/src/qtterminal/qt_term.cpp
>>>> +++ b/src/qtterminal/qt_term.cpp
>>>> @@ -251,7 +252,7
>>>>
>>>>          // The QLocalSocket::waitForConnected does not respect the
>>>> time out argument when the
>>>>          // gnuplot_qt application is not yet started. To wait for it,
>>>> we need to implement the timeout ourselves
>>>> -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
>>>> +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
>>>>          do
>>>>          {
>>>>                  qt->socket.connectToServer(server);
>>>>
>>>> it suddenly almost starts working. I'm saying almost because the first
>>>> plot doesn't work, but the second one does.
>>>
>>> Hence my suspicion that timeouts<  1000msec are not working.
>>> I suggest you do a global search and replace for timeouts and make
>>> sure they are all>1000 msec.
>>
>> I think I have uncovered part of the problem, but I do not understand
>> why it is failing.
>>
>> This loop:
>>          do
>>          {
>>                  qt->socket.connectToServer(server);
>>                  qt->socket.waitForConnected(200);
>>                  // TODO: yield CPU ?
>>          }
>> 	while(...)
>>
>> Does not work as intended.
>>
>> To see why, change it to this:
>> 	qDebug()<<  "qt_connectToServer "<<  server;
>>          do
>>          {
>>                  qt->socket.connectToServer(server);
>>                  if (!qt->socket.waitForConnected(TIMEOUT))
>>                          qDebug()<<  qt->socket.errorString();
>>          	usleep(50000);
>> 	}
>>          while(...)
>>
>> Regardless of the value of TIMEOUT, including -1 or blank,  the
>> waitForConnected returns immediately with an error:
>>           "QLocalSocket::connectToServer: Invalid name"
>
> [Note: I haven't looked at this in detail, so these are just my guesses]
>
> I think the error is probably a race condition:
> - gnuplot starts the external gnuplot_qt process
> - Immediately afterwards it tries to connect to the socket from
>    gnuplot_qt
> - Depending on scheduling, gnuplot_qt may have had a chance to run or
>    not. But in any case it needs to do some initialization first.
>
> So my guess is that the "Invalid name" error stems from the fact that
> during the first iteration of the loop, the named pipe (which is created
> by gnuplot_qt) is simply not there yet.

This is correct, but also how the software is designed.  It continues in 
a loop until it can find the pipe or timeout occurs.  I've printed out 
the server name on my linux box and there is a stream of:

.
.
.
Server:  "qtgnuplot13225"
Server:  "qtgnuplot13225"
Server:  "qtgnuplot13225"
Server:  "qtgnuplot13225"
gnuplot>

before the plot appears.  So it probably isn't too difficult for 1 
second to go by at the system level before an application is up and 
running.  So, I'd say make that timeout just a tad longer.

Notice that this is a recurrent function.  When the timeout does occur, 
eventually the logic gets to:

		if (connectToWidget)
		{
			qDebug() << "Could not connect to widget" << qt_option->Widget << ". 
Starting a QtGnuplotApplication";
			qt_option->Widget = QString();
			qt_connectToServer(qt->localServerName);
		}

so that calls this function a second time, but at the top of the 
function there is

	bool connectToWidget = (server != qt->localServerName);

So then what happens if there is a second failure to connect?  Well, 
then the logic becomes:

		else
		{
			qDebug() << "Could not connect gnuplot_qt" << qt_option->Widget << ". 
Starting a new one";
			execGnuplotQt();
			qt_connectToServer(qt->localServerName, false);
		}

and the argument is 'false' so will not try after the third time.

In any case, I suggest adding the sleep as Ethan has done.  There is no 
reason that should be running at 100%.  In fact, there is probably a 
preferred way to do this without polling loops.  I learned a little bit 
about Qt working on Octave and Qt has this paradigm of signals and slots 
and the developers suggest adhering to the concept otherwise can get 
kind of dodgy (not in this simple case...but cases where widget IDs are 
floating about).

signal:  something that a Qt object emits
slot:  the destination of the signal which can
        be of any number, e.g., five other
        objects could connect a slot to a signal

Anyhow, I don't have time to look at this right now, but if one looks at 
the documentation for a Qt socket:

http://qt-project.org/doc/qt-4.8/qlocalsocket.html#connectToServer

it indicates that a signal is emitted when the connection is complete 
and there is a signal emitted when there is an error.  So the proper 
thing to do is to first make connections to the socket sort of like the 
following ("success" and "failed" are custom member functions):

connect (createdsocket, SIGNAL (connected ()), watcher, SLOT (success ()));
connect (createdsocket, SIGNAL (error ()), watcher, SLOT (failed ()));

and then tell the socket to attempt to connect to the server:

createdsocket->connectToServer (name)

There is no need to check in a loop for anything.  Either the socket 
will successfully connect and emit "connected" at which point 
"success()" will get called or the socket will timeout and emit "error" 
at which point "failed()" will get called.

One can get very creative about connections made, the number of slots 
watching a signal, doing this dynamically, etc.  So instead of a 
recursive routine, it might be multiple connections, or dynamically 
reconnect/disconnect in the "failed()" slot.  Etc.

Dan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Thomas B. <Tho...@gm...>]

* Ethan A Merritt <sf...@us...> [2014-02-11 21:35]:
> On Sunday, 09 February, 2014 16:57:36 sfeam wrote:
> > > And indeed if I change the timeout:
> > > 
> > > --- a/src/qtterminal/qt_term.cpp
> > > +++ b/src/qtterminal/qt_term.cpp
> > > @@ -251,7 +252,7
> > > 
> > >         // The QLocalSocket::waitForConnected does not respect the
> > > time out argument when the
> > >         // gnuplot_qt application is not yet started. To wait for it,
> > > we need to implement the timeout ourselves
> > > -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
> > > +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
> > >         do
> > >         {
> > >                 qt->socket.connectToServer(server);
> > > 
> > > it suddenly almost starts working. I'm saying almost because the first
> > > plot doesn't work, but the second one does.
> > 
> > Hence my suspicion that timeouts < 1000msec are not working.
> > I suggest you do a global search and replace for timeouts and make
> > sure they are all >1000 msec.
> 
> I think I have uncovered part of the problem, but I do not understand
> why it is failing.
> 
> This loop:
>         do
>         {
>                 qt->socket.connectToServer(server);
>                 qt->socket.waitForConnected(200);
>                 // TODO: yield CPU ?
>         }
> 	while(...)
> 
> Does not work as intended.  
> 
> To see why, change it to this:
> 	qDebug() << "qt_connectToServer " << server;
>         do
>         {
>                 qt->socket.connectToServer(server);
>                 if (!qt->socket.waitForConnected(TIMEOUT))
>                         qDebug() << qt->socket.errorString();
>         	usleep(50000);
> 	}
>         while(...)
> 
> Regardless of the value of TIMEOUT, including -1 or blank,  the
> waitForConnected returns immediately with an error:
>          "QLocalSocket::connectToServer: Invalid name" 

[Note: I haven't looked at this in detail, so these are just my guesses]

I think the error is probably a race condition:
- gnuplot starts the external gnuplot_qt process
- Immediately afterwards it tries to connect to the socket from
  gnuplot_qt
- Depending on scheduling, gnuplot_qt may have had a chance to run or
  not. But in any case it needs to do some initialization first.

So my guess is that the "Invalid name" error stems from the fact that
during the first iteration of the loop, the named pipe (which is created
by gnuplot_qt) is simply not there yet.

I also looked through the recent changes to the gnuplot code just now,
and noticed that no QApplication object is created anymore in gnuplot.
According to the docs, waitForConnected() should also work without an
event loop, but I think it would still be interesting to see if the high
CPU usage goes away if QCoreApplication is replaced by QApplication.

Just my 2cents.
Thomas
  
> Without the usleep() it spews hundreds or even thousands of these errors
> before fallling through to the rest of the code.  The usleep reduces this
> to a manageable number of error reports. 
> So the first connection attempts always fail, even under linux, and the
> failure is immediate.
> Why is it an invalid name?  I don't know.
> How does it eventually succeed?  I don't know that either.
> 
> Since TIMEOUT is ignored this is becomes a CPU-burning loop,
> which is what Mojca originally reported.  The usleep should fix that, and a 
> large enough manual timeout as in  
>      QDateTime timeout = QDateTime::currentDateTime().addMSecs(30000);
> may in fact be necessary.  But it would be nice to understand the real
> reason why the initial connection attempts fail, because maybe we can
> test and wait on that condition directly and avoid the useless loop here.
> 
> 	Ethan

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Ethan A M. <sf...@us...>]

On Sunday, 09 February, 2014 16:57:36 sfeam wrote:
> > And indeed if I change the timeout:
> > 
> > --- a/src/qtterminal/qt_term.cpp
> > +++ b/src/qtterminal/qt_term.cpp
> > @@ -251,7 +252,7
> > 
> >         // The QLocalSocket::waitForConnected does not respect the
> > time out argument when the
> >         // gnuplot_qt application is not yet started. To wait for it,
> > we need to implement the timeout ourselves
> > -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
> > +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
> >         do
> >         {
> >                 qt->socket.connectToServer(server);
> > 
> > it suddenly almost starts working. I'm saying almost because the first
> > plot doesn't work, but the second one does.
> 
> Hence my suspicion that timeouts < 1000msec are not working.
> I suggest you do a global search and replace for timeouts and make
> sure they are all >1000 msec.

I think I have uncovered part of the problem, but I do not understand
why it is failing.

This loop:
        do
        {
                qt->socket.connectToServer(server);
                qt->socket.waitForConnected(200);
                // TODO: yield CPU ?
        }
	while(...)

Does not work as intended.  

To see why, change it to this:
	qDebug() << "qt_connectToServer " << server;
        do
        {
                qt->socket.connectToServer(server);
                if (!qt->socket.waitForConnected(TIMEOUT))
                        qDebug() << qt->socket.errorString();
        	usleep(50000);
	}
        while(...)

Regardless of the value of TIMEOUT, including -1 or blank,  the
waitForConnected returns immediately with an error:
         "QLocalSocket::connectToServer: Invalid name" 
Without the usleep() it spews hundreds or even thousands of these errors
before fallling through to the rest of the code.  The usleep reduces this
to a manageable number of error reports. 
So the first connection attempts always fail, even under linux, and the
failure is immediate.
Why is it an invalid name?  I don't know.
How does it eventually succeed?  I don't know that either.

Since TIMEOUT is ignored this is becomes a CPU-burning loop,
which is what Mojca originally reported.  The usleep should fix that, and a 
large enough manual timeout as in  
     QDateTime timeout = QDateTime::currentDateTime().addMSecs(30000);
may in fact be necessary.  But it would be nice to understand the real
reason why the initial connection attempts fail, because maybe we can
test and wait on that condition directly and avoid the useless loop here.

	Ethan

Re: Reworked qt terminal is much faster – Status on Windows

[Bastian M. <bma...@we...>]

Triggered by the current discussion, I tried to build the qt terminal on 
Windows using two configurations:
   MSVC 2008 and Qt 4.8.5 (binary), and
   MSVC 2012 and Qt 5.2.1 (binary).

In both cases, I replaced the waitforinput code by getchar():
	if (options != TERM_ONLY_CHECK_MOUSING)
		return getchar();
	else
		return NUL;
Otherwise I would get error messages like this:
   "Qt terminal communication error: select() error 0 No error".
But this was expected as discussed in
http://sourceforge.net/p/gnuplot/patches/645/
I did not yet test Jérôme's proposed solution, though.

Also, I had to apply the "C89" patch which can be found at
http://sourceforge.net/p/gnuplot/patches/484/

The behaviour of the result seems to be similar to that on Mac:

Without changes, gnuplot_qt gets started (once), but gnuplot cycles 
forever trying to get font info.

Adding sleep() after StartDetached() and/or increasing the timeout in 
qt_connectToServer() helps sometimes, but does not reliably solve the 
problem.

Using Thomas' proposed scheme below, the plot window shows up correctly 
and I can do `plot`s, but mousing is of course inactive.

   Bastian


Am 07.02.2014 22:03, schrieb Thomas Bleher:
>
> I don't have a Mac, so I can't test anything, unfortunately.
> Two things come to mind that you could look at:
>
> - Does this happen with the newest version of Qt4? It may be a bug that
>    has already been fixed.
>
> - Does it work if you start gnuplot_qt manually? The qt terminal can
>    connect to already running programs. To do this:
>    - start gnuplot_qt
>    - determine its pid
>    - start gnuplot
>    - call 'set terminal qt widget "qtgnuplot<pid>"', where <pid> is
>      replaced by the numerical pid of the gnuplot_qt process
>    - Try to plot something, e.g. "plot x"
>
> Hope this helps you.
> Thomas
>

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

On Mon, Feb 10, 2014 at 5:51 PM, sfeam wrote:
> On Monday, 10 February 2014 12:37:23 PM Mojca Miklavec wrote:
>> Also, now that there is no more forking involved, please delete
>>     src/qtterminal/qt_term_mac.m
>> as well as the corresponding lines in Makefiles.
>
> The program still forks.  The only change is that fork() is perforned via
> a Qt library routine rather than being invoked in-line.

Nonetheless that doesn't change my request to remove that code. Even
if forking is still present, it is now apparently done in such a way
that it doesn't confuse Qt any longer.

I'm attaching a patch. It nearly reverts the following commit:

2012-06-23  Jérôme lodewyck  <lod...@us...>

    * configure.in src/Makefile.am src/qtterminal/qt_term.cpp
    src/qtterminal/qt_term_mac.m: Proper handling of the qt terminal dock
    icons on Mac OS

(I'm saying nearly because some of the changes were already present in
the commit from 2012-01-17.)

The whole point of that code (mostly requested by me) was an ugly
workaround for problems related to the way forking was previously
implemented. I can do more testing, but it looks as if that particular
problem is gone.

Mojca

Re: Reworked qt terminal is much faster – and fails to work on Mac

[sfeam <sf...@us...>]

On Monday, 10 February 2014 12:37:23 PM Mojca Miklavec wrote:
> Also, now that there is no more forking involved, please delete
>     src/qtterminal/qt_term_mac.m
> as well as the corresponding lines in Makefiles.

The program still forks.  The only change is that fork() is perforned via
a Qt library routine rather than being invoked in-line.

	Ethan



> 
> Unless you plan to switch back to forking, the file is luckily no
> longer needed. It was only used to hide a non-functional qt window in
> the background (and didn't even work on Mac OS X <= 10.6 or with Qt
> 5). With the current approach that window/process is no longer
> present.
> 
> Mojca
> 
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

Also, now that there is no more forking involved, please delete
    src/qtterminal/qt_term_mac.m
as well as the corresponding lines in Makefiles.

Unless you plan to switch back to forking, the file is luckily no
longer needed. It was only used to hide a non-functional qt window in
the background (and didn't even work on Mac OS X <= 10.6 or with Qt
5). With the current approach that window/process is no longer
present.

Mojca

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Mojca M. <moj...@gm...>]

On Mon, Feb 10, 2014 at 1:57 AM, sfeam wrote:
> On Sunday, 09 February 2014 12:29:34 PM Mojca Miklavec wrote:
>> On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote:
>>
>> According to a gdb a lot of that "infinite cycling" (when not properly
>> connected) happens with
>>
>> while (!receivedFontPropos)
>>     qt->socket.waitForReadyRead(1000);
>>     while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
>> }
>
> It should not be possible for that while {} construction to gobble CPU.
> It only wakes up and checks the socket once per second.
> Unless (which I'm beginning to suspect) all the 1000 msec timeouts
> are acting as NOOPs on OSX.  Could it really be that Qt on OSX
> doesn't implement event-driven waits?

Sorry, I don't know the answer to that, but I can do some
quick-and-dirty test if you can suggest me what to test.

But in my opinion the problem is that qt isn't properly initialized by
the time it executes the "problematic" part of the code.

>> And indeed if I change the timeout:
>>
>> --- a/src/qtterminal/qt_term.cpp
>> +++ b/src/qtterminal/qt_term.cpp
>> @@ -251,7 +252,7
>>
>>         // The QLocalSocket::waitForConnected does not respect the
>> time out argument when the
>>         // gnuplot_qt application is not yet started. To wait for it,
>> we need to implement the timeout ourselves
>> -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
>> +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
>>         do
>>         {
>>                 qt->socket.connectToServer(server);
>>
>> it suddenly almost starts working. I'm saying almost because the first
>> plot doesn't work, but the second one does.
>
> Hence my suspicion that timeouts < 1000msec are not working.
> I suggest you do a global search and replace for timeouts and make
> sure they are all >1000 msec.

I'm not sure how to search for all timeouts. I need some help with that.

I see the following (but I'm not sure how to change it):

if (options == TERM_ONLY_CHECK_MOUSING) {
    timeout = &one_msec;
    one_msec.tv_sec = 0;
    one_msec.tv_usec = TERM_EVENT_POLL_TIMEOUT;
}

and I'm not sure how to reliable find all the others.

> Finally a trace that makes sense in that it involves a recent change.
> This bit belongs to the patch:
>
> 2014-01-26  Jérôme Lodewyck  <lod...@us...>
>
>         * src/qtterminal/qt_term.cpp: Implement font metric caching. This solves
>         a flickering issue when a large number of font changes are called (for
>         example when rotating the world plot in world2.dem).
>
>         * src/qtterminal/QtGnuplotInstance.*: New public function that sends a
>         command to gnuplot and blocks until it receives the answer.
>
> Can you back out just that one set of changes from current CVS and see
> if that makes your qt terminal work?

If I go back to the following version ...

Date:   Sat Jan 18 21:30:19 2014 +0000
    interactive color character art terminal using libcaca

(I didn't try to revert just a single commit, I simply went back to
the version before the one you claimed to be problematic.)

... then gnuplot doesn't get stuck at 99% CPU any longer. It returns
back to accepting input and the second plot works fine. (But please
note that this isn't really a problem as long as timeouts are
sufficiently large. I didn't check, but I suspect that the infinite
loop happens just because Qt isn't properly initialized at the time
when the code is trying to do something.)

Just to make it clear. The following happens:

Terminal type set to 'qt'
gnuplot> plot sin(x) # too short timeout, so another gnuplot_qt is started
Could not connect gnuplot_qt "" . Starting a new one
    # starts a new gnuplot_qt, but timeouts are still too short, so
nothing can be seen
gnuplot> plot cos(x) # the first plot that works ok
gnuplot> quit # only closes the second gnuplot_qt, one gnuplot_qt
needs to be closed manually

Independent on the font patch (which may not necessarily be bad, I
didn't check) the following are the main issues from my point of view:

1.) The following part of the code:

    while((qt->socket.state() != QLocalSocket::ConnectedState) &&
(QDateTime::currentDateTime() < timeout));
    // Still not connected...
    if ((qt->socket.state() != QLocalSocket::ConnectedState) && retry)

gives up too soon (1000 ms seems too short) and simply opens a new
"gnuplot_qt". This is a problem also because the old gnuplot_qt never
gets closed as a consequence. But this can be easily fixed by
increasing the timeout, at least the majority of problems would go
away.

2.) The second time when the same code gets executed (with
retry=false), it gives up too soon again. But now the problem is that
gnuplot blindly assumes that the code succeeded and connection is
alive. In my opinion there should be another check at the end of
qt_connectToServer (or somewhere else):

    if (qt->socket.state() != QLocalSocket::ConnectedState))

that would throw an error and prevent Qt from proceeding in case that
connection didn't succeed. Gnuplot shouldn't blindly assume that
connection succeeded the second time it tried.


Curiously, gnuplot_qt has about 40 font files open when I run just
"plot sin(x)". I don't understand why. For example:

/Library/Fonts/STIXIntSmReg.otf
/Library/Fonts/STIXSizThreeSymBol.otf
/Library/Fonts/STIXSizFiveSymReg.otf
/Library/Fonts/Microsoft/Marlett.ttf
/Library/Fonts/STIXSizOneSymBol.otf
/Library/Fonts/STIXSizFourSymReg.otf
/Library/Fonts/STIXSizFourSymBol.otf
/Library/Fonts/STIXIntUpSmReg.otf
/Library/Fonts/STIXSizThreeSymReg.otf
/Library/Fonts/STIXSizOneSymReg.otf
/Library/Fonts/STIXVar.otf
/Library/Fonts/STIXIntUpBol.otf
/Library/Fonts/STIXSizTwoSymReg.otf
/System/Library/Fonts/Symbol.ttf
/Library/Fonts/Microsoft/MS Reference Specialty.ttf
/Library/Fonts/Microsoft/Bookshelf Symbol 7.ttf
/Library/Fonts/STIXIntDReg.otf
/Library/Fonts/STIXNonUniBolIta.otf
/System/Library/Fonts/Apple Braille Pinpoint 8 Dot.ttf
/System/Library/Fonts/Apple Braille Outline 8 Dot.ttf
/Library/Fonts/STIXIntUpReg.otf
/System/Library/Fonts/ZapfDingbats.ttf
/Library/Fonts/STIXNonUniIta.otf
/Library/Fonts/STIXIntUpDBol.otf
/Library/Fonts/Hoefler Text Ornaments.ttf
/Library/Fonts/STIXNonUni.otf
/Library/Fonts/Bodoni Ornaments ITC TT/..namedfork/rsrc
/Library/Fonts/STIXNonUniBol.otf
/System/Library/Fonts/Apple Braille Outline 6 Dot.ttf
/Library/Fonts/STIXIntUpSmBol.otf
/System/Library/Fonts/Apple Braille Pinpoint 6 Dot.ttf
/System/Library/Fonts/Apple Color Emoji.ttf
/Library/Fonts/STIXIntSmBol.otf
/Library/Fonts/STIXSizTwoSymBol.otf
/Library/Fonts/STIXIntDBol.otf
/Library/Fonts/Type Embellishmnt One LET/..namedfork/rsrc
/System/Library/Fonts/Apple Braille.ttf
/System/Library/Fonts/LucidaGrande.ttc

----------------------

In summary. If I do both:
(a) revert to the version from Sat Jan 18 21:30:19 2014
(b) increase QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
then gnuplot works fine.

If I do just (b) on the latest version, then the first plot fails.
If I do just (a), the first plot fails, a second "gnuplot_qt" is
started (which is bad) and the second plot succeeds.

Mojca

Re: Patch for win64 in Octave

[Allin C. <cot...@wf...>]

On Sun, 9 Feb 2014, Petr Mikulik wrote:

> I've just noticed a patch for gnuplot "Add patch for win64 gnuplot" in
> Octave's mercurcial:
> 	http://hg.octave.org/mxe-octave/rev/598f4d2af02e
>
> Is this known?

See the thread starting with

http://comments.gmane.org/gmane.comp.graphics.gnuplot.devel/11357

Gnuplot CVS has now been patched for correct compilation on 64-bit 
Windows.

Allin Cottrell

Re: Reworked qt terminal is much faster – and fails to work on Mac

[sfeam <sf...@us...>]

On Sunday, 09 February 2014 12:29:34 PM Mojca Miklavec wrote:
> On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote:
> > On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote:
> >> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote:
> >>> > Another possible test:
> >>> > Change the timeout value on line 259 from
> >>> >                qt->socket.waitForConnected(200);
> >>> > to
> >>> >                qt->socket.waitForConnected(-1);
> >>> >
> >>> > This could potentially cause gnuplot to hang, but it also might work
> >>>
> >>> Gnuplot runs at 99% CPU with or without that change (just gnuplot,

That makes very little sense to me, but see below.

> >>> gnuplot_qt doesn't consume any CPU at a
> >> gnuplot is spinning CPU cycles while waiting for a timeout?
> >> The OSX implementation must really suck.
> >> Anyhow if you can see the gnuplot_qt process but the waitForConnected
> >> fails to return that's probably a huge clue to what's gone wrong.
> >> Let me think about this for a while.
> >>
> >> I'm cc-ing Jérôme Lodewyck.  Maybe he can decipher the clue.
> >
> > I don't know if that's a clue or not, but if I run gdb and manually
> > press "n", only a single gnuplot_qt is started and gnuplot actually
> > returns to the console (if I simply run gnuplot, it runs at 99% CPU
> > "forever"). The first "plot sin(x)" doesn't show anything, but when I
> > plot something for the second time, the plot is actually shown.
> >
> > So maybe there's just a problem of wrong
> > timing/synchronisation/initialisation somewhere after all.
> >
> > The fact is that if I actually get to this point (by manually stepping
> > inside gdb), it works a lot better than it did with forking:
> > - printing doesn't crash
> > - there is no need for the dirty hack removeDockIcon()
> >   TransformProcessType(&psn, kProcessTransformToBackgroundApplication);
> >   to hide the nofunctional window


> According to a gdb a lot of that "infinite cycling" (when not properly
> connected) happens with
> 
> while (!receivedFontPropos)
>     qt->socket.waitForReadyRead(1000);
>     while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
> }

It should not be possible for that while {} construction to gobble CPU.
It only wakes up and checks the socket once per second.
Unless (which I'm beginning to suspect) all the 1000 msec timeouts
are acting as NOOPs on OSX.  Could it really be that Qt on OSX 
doesn't implement event-driven waits? 

> And indeed if I change the timeout:
> 
> --- a/src/qtterminal/qt_term.cpp
> +++ b/src/qtterminal/qt_term.cpp
> @@ -251,7 +252,7
> 
>         // The QLocalSocket::waitForConnected does not respect the
> time out argument when the
>         // gnuplot_qt application is not yet started. To wait for it,
> we need to implement the timeout ourselves
> -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
> +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
>         do
>         {
>                 qt->socket.connectToServer(server);
> 
> it suddenly almost starts working. I'm saying almost because the first
> plot doesn't work, but the second one does.

Hence my suspicion that timeouts < 1000msec are not working.
I suggest you do a global search and replace for timeouts and make
sure they are all >1000 msec.

 
> Terminal type set to 'qt'
> gnuplot> plot sin(x) # nothing can be seen
> started detached process  "qtgnuplot31706"
> gnuplot> plot cos(x) # works
> gnuplot>
> 
> It seems that gnuplot gives up too quickly. And if it does give up,
> the second attempt to connect doesn't work properly, ends up with two
> instances of gnuplot_qt running (none of them gets closed etc).
> 
> The following is "sampling" of the running gnuplot at 99% CPU:
> 
> Call graph:
>     2510 Thread_2393718   DispatchQueue_1: com.apple.main-thread  (serial)
>     + 2510 start  (in gnuplot) + 52  [0x100011244]
>     +   2510 main  (in gnuplot) + 2796  [0x1000aa62c]  plot.c:672
>     +     2510 com_line  (in gnuplot) + 144  [0x100025140]  command.c:323
>     +       2510 do_line  (in gnuplot) + 1066  [0x10002559a]  command.c:411
>     +         2510 command  (in gnuplot) + 131  [0x100026283]  command.c:616
>     +           2510 plot_command  (in gnuplot) + 206  [0x100028d1e]
> command.c:1559
>     +             2510 plotrequest  (in gnuplot) + 1763  [0x1000abf83]
>  plot2d.c:273
>     +               2510 eval_plots  (in gnuplot) + 27828
> [0x1000c00c4]  plot2d.c:3259
>     +                 2510 do_plot  (in gnuplot) + 131  [0x100062843]
> graphics.c:517
>     +                   2510 term_start_plot  (in gnuplot) + 63
> [0x10011476f]  term.c:557
>     +                     2510 qt_graphics  (in gnuplot) + 669
> [0x1001da68d]  qt_term.cpp:480
>     +                       1582 qt_sendFont()  (in gnuplot) + 772
> [0x1001da264]  qt_term.cpp:424
>     +                       ! 962 QLocalSocket::bytesAvailable() const
>  (in QtNetwork) + 42  [0x1022520fa]
>     +                       ! : 646 QAbstractSocket::bytesAvailable()
> const  (in QtNetwork) + 16  [0x102245210]
>     +                       ! : | 646 QIODevice::bytesAvailable()
> const  (in QtCore) + 0,20,...  [0x1030389a0,0x1030389b4,...]
>     +                       ! : 248 QAbstractSocket::bytesAvailable()
> const  (in QtNetwork) + 0,16,...  [0x102245200,0x102245210,...]
>     +                       ! : 68
> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
> [0x10226ca8a]
>     +                       ! 295 QLocalSocket::bytesAvailable() const
>  (in QtNetwork) + 16  [0x1022520e0]
>     +                       ! : 295 QIODevice::bytesAvailable() const
> (in QtCore) + 0,85,...  [0x1030389a0,0x1030389f5,...]
>     +                       ! 236 QLocalSocket::bytesAvailable() const
>  (in QtNetwork) + 6,16,...  [0x1022520d6,0x1022520e0,...]
>     +                       ! 89
> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
> [0x10226ca8a]
>     +                       342 qt_sendFont()  (in gnuplot) + 737
> [0x1001da241]  qt_term.cpp:423
>     +                       ! 195 QLocalSocket::waitForReadyRead(int)
> (in QtNetwork) + 6,51,...  [0x1022526c6,0x1022526f3,...]
>     +                       ! 147 QLocalSocket::waitForReadyRead(int)
> (in QtNetwork) + 18  [0x1022526d2]
>     +                       !   147 QLocalSocket::state() const  (in
> QtNetwork) + 0,8  [0x10224aed0,0x10224aed8]
>     +                       180 qt_sendFont()  (in gnuplot) +
> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>     +                       118 qt_sendFont()  (in gnuplot) +
> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>     +                       110
> DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot) + 0
> [0x1001e38b6]
>     +                       97
> DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot) + 0
> [0x1001e38ec]
>     +                       47 qt_sendFont()  (in gnuplot) + 695
> [0x1001da217]  qt_term.cpp:421
>     +                       34 qt_sendFont()  (in gnuplot) + 999
> [0x1001da347]  qt_term.cpp:437
>     2510 Thread_2393731   DispatchQueue_2:
> com.apple.libdispatch-manager  (serial)
>     + 2510 _dispatch_mgr_thread  (in libdispatch.dylib) + 54  [0x7fff8c328316]
>     +   2510 _dispatch_mgr_invoke  (in libdispatch.dylib) + 923
> [0x7fff8c329786]
>     +     2510 kevent  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e27e6]
>     2510 Thread_2393735: QProcessManager
>       2510 thread_start  (in libsystem_c.dylib) + 13  [0x7fff89671b75]
>         2510 _pthread_start  (in libsystem_c.dylib) + 335  [0x7fff8966e8bf]
>           2510 QThreadPrivate::start(void*)  (in QtCore) + 504  [0x102fad238]
>             2510 QProcessManager::run()  (in QtCore) + 168  [0x10307d7b8]
>               2510 __select  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e1df2]
> 
> Total number in stack (recursive counted multiple, when >=5):
> 
> Sort by top of stack, same collapsed (when >= 5):
>         __select  (in libsystem_kernel.dylib)        2510
>         kevent  (in libsystem_kernel.dylib)        2510
>         QIODevice::bytesAvailable() const  (in QtCore)        941
>         qt_sendFont()  (in gnuplot)        379
>         QAbstractSocket::bytesAvailable() const  (in QtNetwork)        248
>         QLocalSocket::bytesAvailable() const  (in QtNetwork)        236
>         QLocalSocket::waitForReadyRead(int)  (in QtNetwork)        195
>         DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork)        157
>         QLocalSocket::state() const  (in QtNetwork)        147
>         DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot)        110
>         DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot)        97
> 
> Mojca

Finally a trace that makes sense in that it involves a recent change.
This bit belongs to the patch:

2014-01-26  Jérôme Lodewyck  <lod...@us...>

        * src/qtterminal/qt_term.cpp: Implement font metric caching. This solves
        a flickering issue when a large number of font changes are called (for
        example when rotating the world plot in world2.dem).

        * src/qtterminal/QtGnuplotInstance.*: New public function that sends a
        command to gnuplot and blocks until it receives the answer.

Can you back out just that one set of changes from current CVS and see
if that makes your qt terminal work?  If it doesn't, then add in the larger
timeouts and check again.   We can probably make this font-related stuff
work, but first let's try to disentangle it from any fork+exec+sync timing
problems.

	Ethan

Patch for win64 in Octave

[Petr M. <mi...@ph...>]

I've just noticed a patch for gnuplot "Add patch for win64 gnuplot" in 
Octave's mercurcial:
 	http://hg.octave.org/mxe-octave/rev/598f4d2af02e

Is this known?

Petr

Re: Reworked qt terminal is much faster – and fails to work on Mac

[Daniel J S. <dan...@ie...>]

On 02/09/2014 05:29 AM, Mojca Miklavec wrote:
> On Sun, Feb 9, 2014 at 12:11 PM, Mojca Miklavec wrote:
>> On Sun, Feb 9, 2014 at 1:19 AM, sfeam wrote:
>>> On Sunday, 09 February 2014 01:01:20 AM Mojca Miklavec wrote:
>>>>> Another possible test:
>>>>> Change the timeout value on line 259 from
>>>>>                 qt->socket.waitForConnected(200);
>>>>> to
>>>>>                 qt->socket.waitForConnected(-1);
>>>>>
>>>>> This could potentially cause gnuplot to hang, but it also might work
>>>>
>>>> Gnuplot runs at 99% CPU with or without that change (just gnuplot,
>>>> gnuplot_qt doesn't consume any CPU at all after the first few
>>>> seconds).
>>>
>>> gnuplot is spinning CPU cycles while waiting for a timeout?
>>> The OSX implementation must really suck.
>>> Anyhow if you can see the gnuplot_qt process but the waitForConnected
>>> fails to return that's probably a huge clue to what's gone wrong.
>>> Let me think about this for a while.
>>>
>>> I'm cc-ing Jérôme Lodewyck.  Maybe he can decipher the clue.
>>
>> I don't know if that's a clue or not, but if I run gdb and manually
>> press "n", only a single gnuplot_qt is started and gnuplot actually
>> returns to the console (if I simply run gnuplot, it runs at 99% CPU
>> "forever"). The first "plot sin(x)" doesn't show anything, but when I
>> plot something for the second time, the plot is actually shown.
>>
>> So maybe there's just a problem of wrong
>> timing/synchronisation/initialisation somewhere after all.
>>
>> The fact is that if I actually get to this point (by manually stepping
>> inside gdb), it works a lot better than it did with forking:
>> - printing doesn't crash
>> - there is no need for the dirty hack removeDockIcon()
>>    TransformProcessType(&psn, kProcessTransformToBackgroundApplication);
>>    to hide the nofunctional window
>
> And indeed if I change the timeout:
>
> --- a/src/qtterminal/qt_term.cpp
> +++ b/src/qtterminal/qt_term.cpp
> @@ -251,7 +252,7
>
>          // The QLocalSocket::waitForConnected does not respect the
> time out argument when the
>          // gnuplot_qt application is not yet started. To wait for it,
> we need to implement the timeout ourselves
> -       QDateTime timeout = QDateTime::currentDateTime().addMSecs(1000);
> +       QDateTime timeout = QDateTime::currentDateTime().addMSecs(10000);
>          do
>          {
>                  qt->socket.connectToServer(server);
>
> it suddenly almost starts working. I'm saying almost because the first
> plot doesn't work, but the second one does.
>
> Terminal type set to 'qt'
> gnuplot>  plot sin(x) # nothing can be seen
> started detached process  "qtgnuplot31706"
> gnuplot>  plot cos(x) # works
> gnuplot>
>
> It seems that gnuplot gives up too quickly. And if it does give up,
> the second attempt to connect doesn't work properly, ends up with two
> instances of gnuplot_qt running (none of them gets closed etc).
>
> According to a gdb a lot of that "infinite cycling" (when not properly
> connected) happens with
>
> while (!receivedFontPropos)
>      qt->socket.waitForReadyRead(1000);
>      while (qt->socket.bytesAvailable()>= (int)sizeof(gp_event_t))
> }
>
> The following is "sampling" of the running gnuplot at 99% CPU:

Trying my best to decipher, but I think it's getting close now.  In a 
post that didn't make it here you added that you see this error:

"Incorrect NSStringEncoding value 0x0000 detected."

Looking at the debugger results below, it seems that qt_term.cpp is 
having problems at or near lines 423-424:

> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>      +                       118 qt_sendFont()  (in gnuplot) +
> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>      +                       110

and the code hunk around there is:

while (!receivedFontPropos)
{
   qt->socket.waitForReadyRead(1000);
423  while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))
424  {
     gp_event_t event;
     qt->socket.read((char*) &event, sizeof(gp_event_t));
     // Here, we discard other events than fontprops.
     if ((event.type == GE_fontprops) && (event.par1 > 0) && (event.par2 
 > 0))
     {
       receivedFontPropos = true;
       metric = QPair<int, int>(event.par1, event.par2);
       fontMetricCache[currentFont] = metric;
       break;
     }
   }
}

So, some comments:

1) Ethan was likely correct about the timeout.  The Qt documentation 
indicates the default timeout is 30 seconds.  1 second seems rather 
short.  Especially in this case, as font setting/info is a system 
library and it is quite common for system level features to take up to 
five seconds or so to complete especially the first time the are called 
and loaded resident.  Mojca, for a temporary measure to take this issue 
out of the picture, consider setting the timeout to the default 30 
seconds (i.e., remove the argument 1000 from the function or change 1000 
to 30000).

2) With the error you pointed out regarding NSStringEncoding, it would 
seem there might be an added delay because something isn't correct about 
qt_term's requested font info string.  Perhaps the system font library 
can't find the metric information that is being sought and it takes a 
while to prep/search the system database.

3) On the other hand, that error you indicated could be what happens when

qt->socket.waitForReadyRead(1000);
while (qt->socket.bytesAvailable() >= (int)sizeof(gp_event_t))

fails.  (Are you seeing that error when you add the really long wait 
period?)

4) When the above does fail (i.e., times out), what happens?  It looks like

QPair<int, int> metric;

is uninitialized, or at least defaults to a zero pair.  So, if the font 
info times out, what is the value of metric when used here?

	term->v_char = qt_oversampling*metric.first;
	term->h_char = qt_oversampling*metric.second;

5) Timeout should generally be treated a little more robustly especially 
given that Qt has some ready-made tools for displaying dialog boxes. 
Perhaps when timeout happens, display a warning dialog about canceling 
or continuing onward.  Of course, that creates issues for gnuplot in 
non-interactive mode.  In any case, this probably isn't critical with 
regard to the problems in functionality you are seeing.

Dan


>
> Call graph:
>      2510 Thread_2393718   DispatchQueue_1: com.apple.main-thread  (serial)
>      + 2510 start  (in gnuplot) + 52  [0x100011244]
>      +   2510 main  (in gnuplot) + 2796  [0x1000aa62c]  plot.c:672
>      +     2510 com_line  (in gnuplot) + 144  [0x100025140]  command.c:323
>      +       2510 do_line  (in gnuplot) + 1066  [0x10002559a]  command.c:411
>      +         2510 command  (in gnuplot) + 131  [0x100026283]  command.c:616
>      +           2510 plot_command  (in gnuplot) + 206  [0x100028d1e]
> command.c:1559
>      +             2510 plotrequest  (in gnuplot) + 1763  [0x1000abf83]
>   plot2d.c:273
>      +               2510 eval_plots  (in gnuplot) + 27828
> [0x1000c00c4]  plot2d.c:3259
>      +                 2510 do_plot  (in gnuplot) + 131  [0x100062843]
> graphics.c:517
>      +                   2510 term_start_plot  (in gnuplot) + 63
> [0x10011476f]  term.c:557
>      +                     2510 qt_graphics  (in gnuplot) + 669
> [0x1001da68d]  qt_term.cpp:480
>      +                       1582 qt_sendFont()  (in gnuplot) + 772
> [0x1001da264]  qt_term.cpp:424
>      +                       ! 962 QLocalSocket::bytesAvailable() const
>   (in QtNetwork) + 42  [0x1022520fa]
>      +                       ! : 646 QAbstractSocket::bytesAvailable()
> const  (in QtNetwork) + 16  [0x102245210]
>      +                       ! : | 646 QIODevice::bytesAvailable()
> const  (in QtCore) + 0,20,...  [0x1030389a0,0x1030389b4,...]
>      +                       ! : 248 QAbstractSocket::bytesAvailable()
> const  (in QtNetwork) + 0,16,...  [0x102245200,0x102245210,...]
>      +                       ! : 68
> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
> [0x10226ca8a]
>      +                       ! 295 QLocalSocket::bytesAvailable() const
>   (in QtNetwork) + 16  [0x1022520e0]
>      +                       ! : 295 QIODevice::bytesAvailable() const
> (in QtCore) + 0,85,...  [0x1030389a0,0x1030389f5,...]
>      +                       ! 236 QLocalSocket::bytesAvailable() const
>   (in QtNetwork) + 6,16,...  [0x1022520d6,0x1022520e0,...]
>      +                       ! 89
> DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork) + 0
> [0x10226ca8a]
>      +                       342 qt_sendFont()  (in gnuplot) + 737
> [0x1001da241]  qt_term.cpp:423
>      +                       ! 195 QLocalSocket::waitForReadyRead(int)
> (in QtNetwork) + 6,51,...  [0x1022526c6,0x1022526f3,...]
>      +                       ! 147 QLocalSocket::waitForReadyRead(int)
> (in QtNetwork) + 18  [0x1022526d2]
>      +                       !   147 QLocalSocket::state() const  (in
> QtNetwork) + 0,8  [0x10224aed0,0x10224aed8]
>      +                       180 qt_sendFont()  (in gnuplot) +
> 748,772,...  [0x1001da24c,0x1001da264,...]  qt_term.cpp:424
>      +                       118 qt_sendFont()  (in gnuplot) +
> 737,713,...  [0x1001da241,0x1001da229,...]  qt_term.cpp:423
>      +                       110
> DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot) + 0
> [0x1001e38b6]
>      +                       97
> DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot) + 0
> [0x1001e38ec]
>      +                       47 qt_sendFont()  (in gnuplot) + 695
> [0x1001da217]  qt_term.cpp:421
>      +                       34 qt_sendFont()  (in gnuplot) + 999
> [0x1001da347]  qt_term.cpp:437
>      2510 Thread_2393731   DispatchQueue_2:
> com.apple.libdispatch-manager  (serial)
>      + 2510 _dispatch_mgr_thread  (in libdispatch.dylib) + 54  [0x7fff8c328316]
>      +   2510 _dispatch_mgr_invoke  (in libdispatch.dylib) + 923
> [0x7fff8c329786]
>      +     2510 kevent  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e27e6]
>      2510 Thread_2393735: QProcessManager
>        2510 thread_start  (in libsystem_c.dylib) + 13  [0x7fff89671b75]
>          2510 _pthread_start  (in libsystem_c.dylib) + 335  [0x7fff8966e8bf]
>            2510 QThreadPrivate::start(void*)  (in QtCore) + 504  [0x102fad238]
>              2510 QProcessManager::run()  (in QtCore) + 168  [0x10307d7b8]
>                2510 __select  (in libsystem_kernel.dylib) + 10  [0x7fff8b5e1df2]
>
> Total number in stack (recursive counted multiple, when>=5):
>
> Sort by top of stack, same collapsed (when>= 5):
>          __select  (in libsystem_kernel.dylib)        2510
>          kevent  (in libsystem_kernel.dylib)        2510
>          QIODevice::bytesAvailable() const  (in QtCore)        941
>          qt_sendFont()  (in gnuplot)        379
>          QAbstractSocket::bytesAvailable() const  (in QtNetwork)        248
>          QLocalSocket::bytesAvailable() const  (in QtNetwork)        236
>          QLocalSocket::waitForReadyRead(int)  (in QtNetwork)        195
>          DYLD-STUB$$QIODevice::bytesAvailable() const  (in QtNetwork)        157
>          QLocalSocket::state() const  (in QtNetwork)        147
>          DYLD-STUB$$QLocalSocket::waitForReadyRead(int)  (in gnuplot)        110
>          DYLD-STUB$$QLocalSocket::bytesAvailable() const  (in gnuplot)        97
>
> Mojca
>