Menu
▾
▴
gnuplot-beta
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 03:26:33
|
In attempting to modify the qt_term.cpp Qt terminal, I ran into some annoying problems with recursive calls, namely enhanced_recursion() and do_event(). First, let me summarize a few things: 1) There are a few uses of the global pointer term->. My preference would be to remove those, but I understand there needs to be a way to get some information back to gnuplot core. It seems to me that putting non-const pointers in the API is the best way to do that. But yes, it is sort of the same difference. 2) By placing the Qt terminal gnuplot_qt interface in a separate thread, a mutex/wait is necessary for any terminal function that a) modifies one of these global variables via term->, b) accesses anything globally via pointer such as a text string. The reason being that gnuplot core cannot modify things that the separate thread is just about to access or use anything that the separate thread has yet to modify. This mechanism works fine. In the cases where it is only objects passed into the thread not using pointers, the API call can return immediately because all those signals are queued up in the second thread and don't need any global access. So, if this works, I would probably go through and first copy the global strings into a QString and just send that via signal to the terminal slot. 3) So the global variables aren't so bad, but recursive calls back into gnuplot core (by the code in a separate thread) are trouble because that is not thread safe. If the emitter code emits a signal and then sits to wait for the slot to signal it has finished and wake up the emitter, a do_event() back into the core is going to issue another signal and wait a second time. The first slot call is going to either freeze or timeout because potentially the gnuplot core thread is waiting for two different things to finish. I've managed to move enhanced_recursion() into the emitter code (i.e., same thread as gnuplot core) and that works. Inelegant, but it works. But I raised the white flag with the do_event() callbacks that was coming from mouse code. I just commented that line out to test things (and get some feedback from Mojca.) So, with that, I'll ask if there is some way of redesigning enhanced_recursion() and do_event(). It really doesn't fit the terminal concept if term code is behaving that way, i.e., calling code that really isn't inside its domain. What is the role of enhanced_recursion()? What is the role of do_event()? Is there some way a result can be sent back via the API that indicates to repeat the last event? Dan |
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 03:36:45
|
On 02/23/2014 09:26 PM, Daniel J Sebald wrote: > What is the role of do_event()? > > Is there some way a result can be sent back via the API that indicates > to repeat the last event? Or, return an event pointer to the API that if zero means the gnuplot core should do nothing, but if it is a valid pointer to an event type then gnuplot core should issue that event. That would be much better than attempting to call core code from within the terminal and a separate thread. Dan |
|
From: sfeam <sf...@us...> - 2014-02-24 03:44:08
|
On Sunday, 23 February 2014 09:26:26 PM Daniel J Sebald wrote: > In attempting to modify the qt_term.cpp Qt terminal, I ran into some > annoying problems with recursive calls, namely enhanced_recursion() and > do_event(). > > First, let me summarize a few things: > > 1) There are a few uses of the global pointer term->. My preference > would be to remove those, but I understand there needs to be a way to > get some information back to gnuplot core. It seems to me that putting > non-const pointers in the API is the best way to do that. But yes, it > is sort of the same difference. I'm afraid I'm not following you. A few uses for term-> in what piece of code, exactly? > I've managed to move enhanced_recursion() into the emitter code (i.e., > same thread as gnuplot core) and that works. Inelegant, but it works. Here I'm really lost. enhanced_recursion() is part of the gnuplot core. It is shared by all terminals. What does this have to do with qt in particular? > So, with that, I'll ask if there is some way of redesigning > enhanced_recursion() and do_event(). These have nothing to do with each other, so I don't understand the question. Both are part of the core, and are shared by all terminals. > What is the role of enhanced_recursion()? This is the routine that interprets enhanced text markup strings. It's part of the core text processing. It is called whenever the core routines want to output a string in enhanced text mode. > What is the role of do_event()? do_event() is an asynchronous entry point in the core. Interactive terminals use it to request some action, e.g. replot, update mouse coords, zoom, respond to hot-key. > Is there some way a result can be sent back via the API that indicates > to repeat the last event? Sent from whom to whom? What sort of event? |
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 04:04:50
|
On 02/23/2014 09:43 PM, sfeam wrote:
> On Sunday, 23 February 2014 09:26:26 PM Daniel J Sebald wrote:
>> In attempting to modify the qt_term.cpp Qt terminal, I ran into some
>> annoying problems with recursive calls, namely enhanced_recursion() and
>> do_event().
>>
>> First, let me summarize a few things:
>>
>> 1) There are a few uses of the global pointer term->. My preference
>> would be to remove those, but I understand there needs to be a way to
>> get some information back to gnuplot core. It seems to me that putting
>> non-const pointers in the API is the best way to do that. But yes, it
>> is sort of the same difference.
>
> I'm afraid I'm not following you.
> A few uses for term-> in what piece of code, exactly?
There aren't many cases, but here is one example in qt_term.cpp:
// Called just before a plot is going to be displayed.
void QtTerminalInterface::qt_graphics(unsigned int v_char)
{
ensureOptionsCreated();
out << GEDesactivate;
qt_flushOutBuffer();
connectToServer();
// Set text encoding
if (!(codec = qt_encodingToCodec(encoding)))
codec = QTextCodec::codecForLocale();
// Set font
currentFontSize = qt_optionFontSize;
currentFontName = qt_option->FontName;
// Set plot size
if (qt_setSize)
{
term->xmax = qt_oversampling*qt_setWidth;
term->ymax = qt_oversampling*qt_setHeight;
qt_setSize = false;
}
In a separate thread, the "term->xmax =" will be done asynchronously.
Hence a mutex/wait is needed to make sure the code in the separate
thread has updated term->xmax and term->max before the core thread can
continue onward.
>> I've managed to move enhanced_recursion() into the emitter code (i.e.,
>> same thread as gnuplot core) and that works. Inelegant, but it works.
>
> Here I'm really lost. enhanced_recursion() is part of the gnuplot core.
> It is shared by all terminals. What does this have to do with qt
> in particular?
It has to do with the fact I've placed the bulk of the qt terminal in a
separate thread (which has an execution loop to ensure proper Qt
behavior...that's the theory anyway).
>> So, with that, I'll ask if there is some way of redesigning
>> enhanced_recursion() and do_event().
>
> These have nothing to do with each other, so I don't understand the
> question. Both are part of the core, and are shared by all terminals.
What they share is the fact that they go back to the core code and can
recursively issue further API calls before the active API returns.
>> What is the role of enhanced_recursion()?
>
> This is the routine that interprets enhanced text markup strings.
> It's part of the core text processing. It is called whenever the
> core routines want to output a string in enhanced text mode.
What you described sounds more like enhanced_writec(), whereas
enhanced_recursion() is issued by the terminal, at least it is for the
Qt terminal.
>> What is the role of do_event()?
>
> do_event() is an asynchronous entry point in the core.
> Interactive terminals use it to request some action, e.g.
> replot, update mouse coords, zoom, respond to hot-key.
>
>> Is there some way a result can be sent back via the API that indicates
>> to repeat the last event?
>
> Sent from whom to whom? What sort of event?
Add a second variable to waitforinput, say:
qt_waitforinput(int options, gp_event_t* event)
and the gnuplot core does something like:
gp_event_t *event_request = 0;
term->waitforinput(options, &event_request);
if (event_request)
do_event(event_request);
That way there are no recursions...I'm assuming (hoping) that there
isn't recursions inside of recursions.
Dan
|
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 04:24:55
|
On 02/23/2014 10:04 PM, Daniel J Sebald wrote:
> Add a second variable to waitforinput, say:
>
> qt_waitforinput(int options, gp_event_t* event)
>
> and the gnuplot core does something like:
>
> gp_event_t *event_request = 0;
> term->waitforinput(options,&event_request);
> if (event_request)
> do_event(event_request);
>
> That way there are no recursions...I'm assuming (hoping) that there
> isn't recursions inside of recursions.
I suppose I could do something like the above inside the Qt terminal, so
long as it is done on the gnuplot core side of things:
MAIN PROGRAM | QTHREAD
|
gnuplot core | QtTerminalInterface
QtTerminalEmitter | (active event loop)
(inactive event loop) |
|
Place do_event() here |
when other thread is |
complete |
Dan
|
|
From: sfeam <sf...@us...> - 2014-02-24 04:36:10
|
On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
> On 02/23/2014 09:43 PM, sfeam wrote:
> > On Sunday, 23 February 2014 09:26:26 PM Daniel J Sebald wrote:
> >> In attempting to modify the qt_term.cpp Qt terminal, I ran into some
> >> annoying problems with recursive calls, namely enhanced_recursion() and
> >> do_event().
> >>
> >> First, let me summarize a few things:
> >>
> >> 1) There are a few uses of the global pointer term->. My preference
> >> would be to remove those, but I understand there needs to be a way to
> >> get some information back to gnuplot core. It seems to me that putting
> >> non-const pointers in the API is the best way to do that. But yes, it
> >> is sort of the same difference.
> >
> > I'm afraid I'm not following you.
> > A few uses for term-> in what piece of code, exactly?
>
> There aren't many cases, but here is one example in qt_term.cpp:
>
> // Called just before a plot is going to be displayed.
> void QtTerminalInterface::qt_graphics(unsigned int v_char)
> {
> ensureOptionsCreated();
> out << GEDesactivate;
> qt_flushOutBuffer();
> connectToServer();
>
> // Set text encoding
> if (!(codec = qt_encodingToCodec(encoding)))
> codec = QTextCodec::codecForLocale();
>
> // Set font
> currentFontSize = qt_optionFontSize;
> currentFontName = qt_option->FontName;
>
> // Set plot size
> if (qt_setSize)
> {
> term->xmax = qt_oversampling*qt_setWidth;
> term->ymax = qt_oversampling*qt_setHeight;
> qt_setSize = false;
> }
>
> In a separate thread, the "term->xmax =" will be done asynchronously.
> Hence a mutex/wait is needed to make sure the code in the separate
> thread has updated term->xmax and term->max before the core thread can
> continue onward.
Ah. Now I'm with you.
Yeah, this is the piece of code that has changed the most in qt
because nothing seems to work properly on both linux and OSX.
You are quite correct that term->foo should not be referenced in
this part of the terminal driver. It is supposed to return the revised
font information via an event GP_fontprops. And it _was_ doing that
at one point. I've now lost track of all the work-arounds and what
exactly they fixed, but certainly it would be good if you can get
back to the original intent. You can look at other terminal
drivers as a model if needed.
> >> So, with that, I'll ask if there is some way of redesigning
> >> enhanced_recursion() and do_event().
> >
> > These have nothing to do with each other, so I don't understand the
> > question. Both are part of the core, and are shared by all terminals.
>
> What they share is the fact that they go back to the core code and can
> recursively issue further API calls before the active API returns.
>
>
> >> What is the role of enhanced_recursion()?
> >
> > This is the routine that interprets enhanced text markup strings.
> > It's part of the core text processing. It is called whenever the
> > core routines want to output a string in enhanced text mode.
>
> What you described sounds more like enhanced_writec(), whereas
> enhanced_recursion() is issued by the terminal, at least it is for the
> Qt terminal.
All of those enhanced_foo() routines are part of the text output layer.
enhanced_recursion() in particular is shared by all terminals and
lives in term.c. It is called whenever an enhanced text string is
output, and then calls itself recursively as the name suggests to
handle embedded fragments of the text markup. E.g.
"{top_{sub1_{sub2_{sub3}}}}"
Each left curly bracket triggers a new level of recursion.
Please leave it alone.
> >> What is the role of do_event()?
> >
> > do_event() is an asynchronous entry point in the core.
> > Interactive terminals use it to request some action, e.g.
> > replot, update mouse coords, zoom, respond to hot-key.
> >
> >> Is there some way a result can be sent back via the API that indicates
> >> to repeat the last event?
> >
> > Sent from whom to whom? What sort of event?
>
> Add a second variable to waitforinput, say:
>
> qt_waitforinput(int options, gp_event_t* event)
>
> and the gnuplot core does something like:
>
> gp_event_t *event_request = 0;
> term->waitforinput(options, &event_request);
> if (event_request)
> do_event(event_request);
>
> That way there are no recursions...I'm assuming (hoping) that there
> isn't recursions inside of recursions.
Sorry, I'm not following this at all.
You can't change waitforinput() for just qt;
it's an API shared by all the interactive terminals.
Anyhow, what's wrong with recursion?
I posted an example before of a hot key definition that
necessarily triggers recursion and works correctly on
all the terminals I tried. If there is a piece of the qt
event code that is not re-entry safe, let's fix that first
and then worry about whether something else is also problematic.
Ethan
|
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 05:22:51
|
On 02/23/2014 10:35 PM, sfeam wrote:
> On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
>> On 02/23/2014 09:43 PM, sfeam wrote:
>>> On Sunday, 23 February 2014 09:26:26 PM Daniel J Sebald wrote:
>>>> In attempting to modify the qt_term.cpp Qt terminal, I ran into some
>>>> annoying problems with recursive calls, namely enhanced_recursion() and
>>>> do_event().
>>>>
>>>> First, let me summarize a few things:
>>>>
>>>> 1) There are a few uses of the global pointer term->. My preference
>>>> would be to remove those, but I understand there needs to be a way to
>>>> get some information back to gnuplot core. It seems to me that putting
>>>> non-const pointers in the API is the best way to do that. But yes, it
>>>> is sort of the same difference.
>>>
>>> I'm afraid I'm not following you.
>>> A few uses for term-> in what piece of code, exactly?
>>
>> There aren't many cases, but here is one example in qt_term.cpp:
>>
>> // Called just before a plot is going to be displayed.
>> void QtTerminalInterface::qt_graphics(unsigned int v_char)
>> {
>> ensureOptionsCreated();
>> out<< GEDesactivate;
>> qt_flushOutBuffer();
>> connectToServer();
>>
>> // Set text encoding
>> if (!(codec = qt_encodingToCodec(encoding)))
>> codec = QTextCodec::codecForLocale();
>>
>> // Set font
>> currentFontSize = qt_optionFontSize;
>> currentFontName = qt_option->FontName;
>>
>> // Set plot size
>> if (qt_setSize)
>> {
>> term->xmax = qt_oversampling*qt_setWidth;
>> term->ymax = qt_oversampling*qt_setHeight;
>> qt_setSize = false;
>> }
>>
>> In a separate thread, the "term->xmax =" will be done asynchronously.
>> Hence a mutex/wait is needed to make sure the code in the separate
>> thread has updated term->xmax and term->max before the core thread can
>> continue onward.
>
> Ah. Now I'm with you.
> Yeah, this is the piece of code that has changed the most in qt
> because nothing seems to work properly on both linux and OSX.
> You are quite correct that term->foo should not be referenced in
> this part of the terminal driver. It is supposed to return the revised
> font information via an event GP_fontprops. And it _was_ doing that
> at one point. I've now lost track of all the work-arounds and what
> exactly they fixed, but certainly it would be good if you can get
> back to the original intent. You can look at other terminal
> drivers as a model if needed.
OK, thanks. I'll look that over. I think that may be the one thing
that isn't working yet for what I've done, i.e., some text isn't showing
up probably owing to the core is told the font height is zero.
>>>> So, with that, I'll ask if there is some way of redesigning
>>>> enhanced_recursion() and do_event().
>>>
>>> These have nothing to do with each other, so I don't understand the
>>> question. Both are part of the core, and are shared by all terminals.
>>
>> What they share is the fact that they go back to the core code and can
>> recursively issue further API calls before the active API returns.
>>
>>
>>>> What is the role of enhanced_recursion()?
>>>
>>> This is the routine that interprets enhanced text markup strings.
>>> It's part of the core text processing. It is called whenever the
>>> core routines want to output a string in enhanced text mode.
>>
>> What you described sounds more like enhanced_writec(), whereas
>> enhanced_recursion() is issued by the terminal, at least it is for the
>> Qt terminal.
>
> All of those enhanced_foo() routines are part of the text output layer.
> enhanced_recursion() in particular is shared by all terminals and
> lives in term.c. It is called whenever an enhanced text string is
> output, and then calls itself recursively as the name suggests to
> handle embedded fragments of the text markup. E.g.
> "{top_{sub1_{sub2_{sub3}}}}"
> Each left curly bracket triggers a new level of recursion.
> Please leave it alone.
Just wondering if that recursion couldn't be kept to the core. It looks
like the things needed (for Qt) to do that recursion are fontName,
fontSize and enhanced_flush(). I think those are all currently
accessible through the API. In any case, I already have that worked out
so no need to modify anything there.
> Anyhow, what's wrong with recursion?
> I posted an example before of a hot key definition that
> necessarily triggers recursion and works correctly on
> all the terminals I tried. If there is a piece of the qt
> event code that is not re-entry safe, let's fix that first
> and then worry about whether something else is also problematic.
I can work this out, now that I've thought about it a bit.
Dan
|
|
From: sfeam <sf...@us...> - 2014-02-24 06:16:21
|
On Sunday, 23 February 2014 11:22:42 PM Daniel J Sebald wrote:
> On 02/23/2014 10:35 PM, sfeam wrote:
> > On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
> >>
> >> // Set plot size
> >> if (qt_setSize)
> >> {
> >> term->xmax = qt_oversampling*qt_setWidth;
> >> term->ymax = qt_oversampling*qt_setHeight;
> >> qt_setSize = false;
> >> }
> >>
> >> In a separate thread, the "term->xmax =" will be done asynchronously.
> >> Hence a mutex/wait is needed to make sure the code in the separate
> >> thread has updated term->xmax and term->max before the core thread can
> >> continue onward.
> >
> > Ah. Now I'm with you.
> > Yeah, this is the piece of code that has changed the most in qt
> > because nothing seems to work properly on both linux and OSX.
> > You are quite correct that term->foo should not be referenced in
> > this part of the terminal driver. It is supposed to return the revised
> > font information via an event GP_fontprops. And it _was_ doing that
> > at one point. I've now lost track of all the work-arounds and what
> > exactly they fixed, but certainly it would be good if you can get
> > back to the original intent. You can look at other terminal
> > drivers as a model if needed.
>
> OK, thanks. I'll look that over. I think that may be the one thing
> that isn't working yet for what I've done, i.e., some text isn't showing
> up probably owing to the core is told the font height is zero.
Let me summarize a bit of the history.
- The core code and the qt_term bits of Qt are in the same process.
Call this "inboard"
The screen display is being managed by a separate Qt process.
Call this "outboard"
The inboard and outboard processes operate asynchronously.
- To reserve space for some text element on the next plot, the core
code needs to know how big the current font is so it sends a query
to the inboard terminal driver. This request can come either via
term->set_font() or in enhanced text mode via term->put_text.
- It would be simplest if the inboard terminal driver could just reply
immediately with the requested font metrics. No communication back
and forth with the outboard terminal driver is required. This is what
the Qt terminal used to do, and still does in version 4.6.
The font size information is obtained by calling
QFontMetrics metrics(QFont(qt_currentFontName, qt_currentFontSize));
Since the inboard driver and the core code are in the same process,
the inboard driver can just set term->h_char and term->v_char
directly and that's the end of it.
- Now here comes the problem. Apparently calling QFontMetrics without
there being a full QApplication and maybe [not sure] an event
doesn't work properly. In particular is was causing problems
on OSX, and it was ugly even on linux.
See the comments in version 4.6 qt_term.cpp
// Create a QApplication without event loop for QObject's that need it,
// namely font handling
// A better strategy would be to transfer the font handling to the
// QtGnuplotWidget, but it would require
// some synchronization between the widget and the gnuplot process.
- So in January Jérôme followed through on that comment and moved
the QFontMetrics call over to the outboard driver for version 4.7.
That simplified things in one way, but introduced a new complication.
Now the inboard driver has to send a change font request to the
outboard driver (a different process) and then wait for the resulting
size information to be sent back via a QEvent. That's where
waitforinput() and do_event() suddenly become involved where
they hadn't been previously. It was also slower, so he later
introduced a cache of font metrics on the inboard side but let's
disregard that for now.
- So now I gather you are working to have an event loop in the
inboard driver again, although I've lost track of exactly why.
But in that case I think it makes sense to move the
font metrics query back into the inboard driver as well.
At which point we return to a setup in which waitforinput()
and do_event() are not involved in font processing.
So much for font handling and the involvement of enhanced text
processing.
There is a separate issue, however, that has been at the heart of
the recent attempts to get OSX working. Information about the
size of the display window necessarily comes from the outboard
driver, which is a separate process. Right now this window size
information is also passed using a GE_fontprops event, but there's no
good reason for that. It really should have a separate event type
to reduce confusion. But changing the name of the event wouldn't
change the information flow in any way.
Ethan
|
|
From: Daniel J S. <dan...@ie...> - 2014-02-24 07:10:40
|
On 02/24/2014 12:16 AM, sfeam wrote:
> On Sunday, 23 February 2014 11:22:42 PM Daniel J Sebald wrote:
>> On 02/23/2014 10:35 PM, sfeam wrote:
>>> On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
>>>>
>>>> // Set plot size
>>>> if (qt_setSize)
>>>> {
>>>> term->xmax = qt_oversampling*qt_setWidth;
>>>> term->ymax = qt_oversampling*qt_setHeight;
>>>> qt_setSize = false;
>>>> }
>>>>
>>>> In a separate thread, the "term->xmax =" will be done asynchronously.
>>>> Hence a mutex/wait is needed to make sure the code in the separate
>>>> thread has updated term->xmax and term->max before the core thread can
>>>> continue onward.
>>>
>>> Ah. Now I'm with you.
>>> Yeah, this is the piece of code that has changed the most in qt
>>> because nothing seems to work properly on both linux and OSX.
>>> You are quite correct that term->foo should not be referenced in
>>> this part of the terminal driver. It is supposed to return the revised
>>> font information via an event GP_fontprops. And it _was_ doing that
>>> at one point. I've now lost track of all the work-arounds and what
>>> exactly they fixed, but certainly it would be good if you can get
>>> back to the original intent. You can look at other terminal
>>> drivers as a model if needed.
>>
>> OK, thanks. I'll look that over. I think that may be the one thing
>> that isn't working yet for what I've done, i.e., some text isn't showing
>> up probably owing to the core is told the font height is zero.
>
> Let me summarize a bit of the history.
>
> - The core code and the qt_term bits of Qt are in the same process.
> Call this "inboard"
> The screen display is being managed by a separate Qt process.
> Call this "outboard"
> The inboard and outboard processes operate asynchronously.
>
> - To reserve space for some text element on the next plot, the core
> code needs to know how big the current font is so it sends a query
> to the inboard terminal driver. This request can come either via
> term->set_font() or in enhanced text mode via term->put_text.
>
> - It would be simplest if the inboard terminal driver could just reply
> immediately with the requested font metrics. No communication back
> and forth with the outboard terminal driver is required. This is what
> the Qt terminal used to do, and still does in version 4.6.
> The font size information is obtained by calling
> QFontMetrics metrics(QFont(qt_currentFontName, qt_currentFontSize));
> Since the inboard driver and the core code are in the same process,
> the inboard driver can just set term->h_char and term->v_char
> directly and that's the end of it.
>
> - Now here comes the problem. Apparently calling QFontMetrics without
> there being a full QApplication and maybe [not sure] an event
> doesn't work properly. In particular is was causing problems
> on OSX, and it was ugly even on linux.
> See the comments in version 4.6 qt_term.cpp
> // Create a QApplication without event loop for QObject's that need it,
> // namely font handling
> // A better strategy would be to transfer the font handling to the
> // QtGnuplotWidget, but it would require
> // some synchronization between the widget and the gnuplot process.
>
> - So in January Jérôme followed through on that comment and moved
> the QFontMetrics call over to the outboard driver for version 4.7.
> That simplified things in one way, but introduced a new complication.
> Now the inboard driver has to send a change font request to the
> outboard driver (a different process) and then wait for the resulting
> size information to be sent back via a QEvent. That's where
> waitforinput() and do_event() suddenly become involved where
> they hadn't been previously. It was also slower, so he later
> introduced a cache of font metrics on the inboard side but let's
> disregard that for now.
>
> - So now I gather you are working to have an event loop in the
> inboard driver again, although I've lost track of exactly why.
Hopefully to make OSX and Windows behave the same as Unix. I don't know
for sure because Qt is a big creature, but I suspect that if there is no
event loop that timers aren't guaranteed to work and who knows what
else. So by providing an event, I'm hoping Qt performs the same on all
platforms. It's just following the examples that Qt documentation gives.
> But in that case I think it makes sense to move the
> font metrics query back into the inboard driver as well.
> At which point we return to a setup in which waitforinput()
> and do_event() are not involved in font processing.
>
> So much for font handling and the involvement of enhanced text
> processing.
>
> There is a separate issue, however, that has been at the heart of
> the recent attempts to get OSX working. Information about the
> size of the display window necessarily comes from the outboard
> driver, which is a separate process. Right now this window size
> information is also passed using a GE_fontprops event, but there's no
> good reason for that. It really should have a separate event type
> to reduce confusion. But changing the name of the event wouldn't
> change the information flow in any way.
Oh, OK. Thanks. I've got the big picture now and I think we will
probably converge on something here that is about right.
I'll correct one thing in that list which is I believe the issue is not
the presence of the QApplication in the main process, but that the
QApplication cannot have an event loop. That is, one can't issue
application.exec(), because doing so will hang gnuplot while
application.exec() blocks to handle all of its realtime traffic
(signals, slots, system calls, etc). Graphics in Qt must be done in a
main thread; it can't be done in a separate thread. That leaves the Qt
graphics code to be run in a separate process. (Note, I've fixed the
process communications a bit so that there aren't these while loops that
keep trying to establish a connection to some external service.)
The QApplication issue is why I've set out to put the bulk of the
terminal interface code in a separate thread because in the separate
thread and event loop, i.e., thread.exec(), is perfectly fine. The
thread acts on its own. My preliminary code indicates there is no
problem sending signals back and forth across that thread. So, with an
event loop running in the second thread, all the timers and whatever
else should be fine--the only restriction being that graphics cannot be
done there. If it turns out OSX doesn't behave the same in the thread
that has an event loop, then something isn't right with Qt. (We'll know
soon.)
So here is what I'm aiming for right now, sort of combining everything:
MAIN PROGRAM | QTHREAD || GNUPLOT_QT
| || (outboard)
| ||
gnuplot core | QtTerminalInterface || Currently has a
QtTerminalEmitter | (active event loop) || few things we hope
(inactive event loop) | || to move back into
| || QtTerminalInterface
where the single line means separate thread and the double line means a
separate process.
OK, for the most part I'll be offline for the week.
Dan
|
|
From: Jérôme L. <jer...@no...> - 2014-02-24 16:03:16
|
Hi,
I got a bit confused about your motivations for introducing an event
loop in the "inboard" qt terminal. In qt_term.cpp, we use, by design, a
very limited set of the Qt library, and rather push as much
functionality as possible the in the gnuplot_qt "outboard" program. And
I think that this set of functions doesn't require an event loop to work
properly (As a matter of fact, I can observe by commenting out the
QCoreApplication declaration in qt_term.cpp that it doesn't even require
an instantiation of QCoreApplication to work -- except from the Windows
specific QCoreApplication::applicationDirPath() call). More
specifically, we use
- Storage classes (QString, QImage, QColor...) that just gather
information but use no event mechanism.
- QLocalSocket that, according to the Qt documentation, can work without
an event loop:
"Although QLocalSocket is designed for use with an event loop, it's
possible to use it without one. In that case, you must use
waitForConnected(), waitForReadyRead(), waitForBytesWritten(), and
waitForDisconnected() which blocks until the operation is complete or
the timeout expires.". In fact, I can imagine that the gnuplot inboard
driver is perhaps precisely the kind of programs that the Qt developers
had in mind when they wrote this sentence: a program that cannot run a
Qt event loop because it implements its own event loop, but that still
wants to send messages to another auxiliary program that runs a Qt event
loop to manage a GUI.
So could you point out a specific past or present problem in the Qt
terminal that is due to the fact that no event loop is running in the
inboard driver ?
To my knowledge, here are the issues that arose on non Linux platform:
- OSX: in the past, there was no gnuplot_qt independent process, and the
GUI was rather managed in a thread. As you point out, this is not
supported in Qt, and while it kind of happened to work with Linux, it
failed on OSX.
- OSX: currently, the QtGnuplotWidget has a hard time to resize itself
to the correct size, but it is specific to the gnuplot_qt outboard driver.
- Windows: Watching at the same time the standard input and a
QLocalSocket file descriptor is not supported on Windows using select()
or the Qt API (which might use select() under the hood). This issue is
not related to the EventLoop or not EventLoop question and has been
recently fixed by specific calls to the win32 API.
- All platforms: waitForConnected() immediatly returns if not socket is
found (in particular, this happens when the gnuplot_qt program is still
being initialized), even when a timeout is set. This was solved by
introducing a custom timeout mechanism, but could more elegantly be
solved by some communication mechanism between gnuplot ("inboard") and
gnuplot_qt ("outboard"). I can see that part of your code actually
implements this.
Concerning QApplication vs. QCoreApplication & font metrics:
Until recently, the inboard driver (qt_term.cpp) used to instantiate a
QApplication to determine the font metrics (no event loop was required
for this though, and more generally, no Qt event loop, nor Qt event
based mechanism has ever been running in the main gnuplot thread). As
Ethan said, this mechanism has been moved to the "outboard" gnuplot_qt
program, and now the inboard driver only instantiate a QCoreApplication
(which, as I said above, even turns out not to be necessary in practice,
although the Qt documentation does not certify this point). The
motivation for this was a significant performance increase: the
initialization of a QApplication requires more than 0.5 s against a few
10ms for a QCoreApplication. Moving back the font metric business in the
main gnuplot program will require to instantiate again a QApplication
and thus cancel this performance improvement.
Concerning the code you propose: apart from the synchronization problems
between the Qt event loop and the main thread that you mention, I am
worried about the performances. As far as I can see, the communication
between the main thread and the thread running the Qt event loop relies
on the signal/slot mechanism, which I think can be significantly slower
that direct function calls. From rough testing, plotting a graph with a
million points takes about 3x more time with the code you posted.
Jérôme
Le 24/02/2014 08:10, Daniel J Sebald a écrit :
> On 02/24/2014 12:16 AM, sfeam wrote:
>> On Sunday, 23 February 2014 11:22:42 PM Daniel J Sebald wrote:
>>> On 02/23/2014 10:35 PM, sfeam wrote:
>>>> On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
>>>>>
>>>>> // Set plot size
>>>>> if (qt_setSize)
>>>>> {
>>>>> term->xmax = qt_oversampling*qt_setWidth;
>>>>> term->ymax = qt_oversampling*qt_setHeight;
>>>>> qt_setSize = false;
>>>>> }
>>>>>
>>>>> In a separate thread, the "term->xmax =" will be done asynchronously.
>>>>> Hence a mutex/wait is needed to make sure the code in the separate
>>>>> thread has updated term->xmax and term->max before the core thread can
>>>>> continue onward.
>>>>
>>>> Ah. Now I'm with you.
>>>> Yeah, this is the piece of code that has changed the most in qt
>>>> because nothing seems to work properly on both linux and OSX.
>>>> You are quite correct that term->foo should not be referenced in
>>>> this part of the terminal driver. It is supposed to return the revised
>>>> font information via an event GP_fontprops. And it _was_ doing that
>>>> at one point. I've now lost track of all the work-arounds and what
>>>> exactly they fixed, but certainly it would be good if you can get
>>>> back to the original intent. You can look at other terminal
>>>> drivers as a model if needed.
>>>
>>> OK, thanks. I'll look that over. I think that may be the one thing
>>> that isn't working yet for what I've done, i.e., some text isn't showing
>>> up probably owing to the core is told the font height is zero.
>>
>> Let me summarize a bit of the history.
>>
>> - The core code and the qt_term bits of Qt are in the same process.
>> Call this "inboard"
>> The screen display is being managed by a separate Qt process.
>> Call this "outboard"
>> The inboard and outboard processes operate asynchronously.
>>
>> - To reserve space for some text element on the next plot, the core
>> code needs to know how big the current font is so it sends a query
>> to the inboard terminal driver. This request can come either via
>> term->set_font() or in enhanced text mode via term->put_text.
>>
>> - It would be simplest if the inboard terminal driver could just reply
>> immediately with the requested font metrics. No communication back
>> and forth with the outboard terminal driver is required. This is what
>> the Qt terminal used to do, and still does in version 4.6.
>> The font size information is obtained by calling
>> QFontMetrics metrics(QFont(qt_currentFontName, qt_currentFontSize));
>> Since the inboard driver and the core code are in the same process,
>> the inboard driver can just set term->h_char and term->v_char
>> directly and that's the end of it.
>>
>> - Now here comes the problem. Apparently calling QFontMetrics without
>> there being a full QApplication and maybe [not sure] an event
>> doesn't work properly. In particular is was causing problems
>> on OSX, and it was ugly even on linux.
>> See the comments in version 4.6 qt_term.cpp
>> // Create a QApplication without event loop for QObject's that need it,
>> // namely font handling
>> // A better strategy would be to transfer the font handling to the
>> // QtGnuplotWidget, but it would require
>> // some synchronization between the widget and the gnuplot process.
>>
>> - So in January Jérôme followed through on that comment and moved
>> the QFontMetrics call over to the outboard driver for version 4.7.
>> That simplified things in one way, but introduced a new complication.
>> Now the inboard driver has to send a change font request to the
>> outboard driver (a different process) and then wait for the resulting
>> size information to be sent back via a QEvent. That's where
>> waitforinput() and do_event() suddenly become involved where
>> they hadn't been previously. It was also slower, so he later
>> introduced a cache of font metrics on the inboard side but let's
>> disregard that for now.
>>
>> - So now I gather you are working to have an event loop in the
>> inboard driver again, although I've lost track of exactly why.
>
> Hopefully to make OSX and Windows behave the same as Unix. I don't know
> for sure because Qt is a big creature, but I suspect that if there is no
> event loop that timers aren't guaranteed to work and who knows what
> else. So by providing an event, I'm hoping Qt performs the same on all
> platforms. It's just following the examples that Qt documentation gives.
>
>
>> But in that case I think it makes sense to move the
>> font metrics query back into the inboard driver as well.
>> At which point we return to a setup in which waitforinput()
>> and do_event() are not involved in font processing.
>>
>> So much for font handling and the involvement of enhanced text
>> processing.
>>
>> There is a separate issue, however, that has been at the heart of
>> the recent attempts to get OSX working. Information about the
>> size of the display window necessarily comes from the outboard
>> driver, which is a separate process. Right now this window size
>> information is also passed using a GE_fontprops event, but there's no
>> good reason for that. It really should have a separate event type
>> to reduce confusion. But changing the name of the event wouldn't
>> change the information flow in any way.
>
> Oh, OK. Thanks. I've got the big picture now and I think we will
> probably converge on something here that is about right.
>
> I'll correct one thing in that list which is I believe the issue is not
> the presence of the QApplication in the main process, but that the
> QApplication cannot have an event loop. That is, one can't issue
> application.exec(), because doing so will hang gnuplot while
> application.exec() blocks to handle all of its realtime traffic
> (signals, slots, system calls, etc). Graphics in Qt must be done in a
> main thread; it can't be done in a separate thread. That leaves the Qt
> graphics code to be run in a separate process. (Note, I've fixed the
> process communications a bit so that there aren't these while loops that
> keep trying to establish a connection to some external service.)
>
> The QApplication issue is why I've set out to put the bulk of the
> terminal interface code in a separate thread because in the separate
> thread and event loop, i.e., thread.exec(), is perfectly fine. The
> thread acts on its own. My preliminary code indicates there is no
> problem sending signals back and forth across that thread. So, with an
> event loop running in the second thread, all the timers and whatever
> else should be fine--the only restriction being that graphics cannot be
> done there. If it turns out OSX doesn't behave the same in the thread
> that has an event loop, then something isn't right with Qt. (We'll know
> soon.)
>
> So here is what I'm aiming for right now, sort of combining everything:
>
> MAIN PROGRAM | QTHREAD || GNUPLOT_QT
> | || (outboard)
> | ||
> gnuplot core | QtTerminalInterface || Currently has a
> QtTerminalEmitter | (active event loop) || few things we hope
> (inactive event loop) | || to move back into
> | || QtTerminalInterface
>
> where the single line means separate thread and the double line means a
> separate process.
>
> OK, for the most part I'll be offline for the week.
>
> Dan
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>
|
|
From: Jérôme L. <lod...@us...> - 2014-02-24 15:59:56
|
Hi,
I got a bit confused about your motivations for introducing an event
loop in the "inboard" qt terminal. In qt_term.cpp, we use, by design, a
very limited set of the Qt library, and rather push as much
functionality as possible the in the gnuplot_qt "outboard" program. And
I think that this set of functions doesn't require an event loop to work
properly (As a matter of fact, I can observe by commenting out the
QCoreApplication declaration in qt_term.cpp that it doesn't even require
an instantiation of QCoreApplication to work -- except from the Windows
specific QCoreApplication::applicationDirPath() call). More
specifically, we use
- Storage classes (QString, QImage, QColor...) that just gather
information but use no event mechanism.
- QLocalSocket that, according to the Qt documentation, can work without
an event loop:
"Although QLocalSocket is designed for use with an event loop, it's
possible to use it without one. In that case, you must use
waitForConnected(), waitForReadyRead(), waitForBytesWritten(), and
waitForDisconnected() which blocks until the operation is complete or
the timeout expires.". In fact, I can imagine that the gnuplot inboard
driver is perhaps precisely the kind of programs that the Qt developers
had in mind when they wrote this sentence: a program that cannot run a
Qt event loop because it implements its own event loop, but that still
wants to send messages to another auxiliary program that runs a Qt event
loop to manage a GUI.
So could you point out a specific past or present problem in the Qt
terminal that is due to the fact that no event loop is running in the
inboard driver ?
To my knowledge, here are the issues that arose on non Linux platform:
- OSX: in the past, there was no gnuplot_qt independent process, and the
GUI was rather managed in a thread. As you point out, this is not
supported in Qt, and while it kind of happened to work with Linux, it
failed on OSX.
- OSX: currently, the QtGnuplotWidget has a hard time to resize itself
to the correct size, but it is specific to the gnuplot_qt outboard driver.
- Windows: Watching at the same time the standard input and a
QLocalSocket file descriptor is not supported on Windows using select()
or the Qt API (which might use select() under the hood). This issue is
not related to the EventLoop or not EventLoop question and has been
recently fixed by specific calls to the win32 API.
- All platforms: waitForConnected() immediatly returns if not socket is
found (in particular, this happens when the gnuplot_qt program is still
being initialized), even when a timeout is set. This was solved by
introducing a custom timeout mechanism, but could more elegantly be
solved by some communication mechanism between gnuplot ("inboard") and
gnuplot_qt ("outboard"). I can see that part of your code actually
implements this.
Concerning QApplication vs. QCoreApplication & font metrics:
Until recently, the inboard driver (qt_term.cpp) used to instantiate a
QApplication to determine the font metrics (no event loop was required
for this though, and more generally, no Qt event loop, nor Qt event
based mechanism has ever been running in the main gnuplot thread). As
Ethan said, this mechanism has been moved to the "outboard" gnuplot_qt
program, and now the inboard driver only instantiate a QCoreApplication
(which, as I said above, even turns out not to be necessary in practice,
although the Qt documentation does not certify this point). The
motivation for this was a significant performance increase: the
initialization of a QApplication requires more than 0.5 s against a few
10ms for a QCoreApplication. Moving back the font metric business in the
main gnuplot program will require to instantiate again a QApplication
and thus cancel this performance improvement.
Concerning the code you propose: apart from the synchronization problems
between the Qt event loop and the main thread that you mention, I am
worried about the performances. As far as I can see, the communication
between the main thread and the thread running the Qt event loop relies
on the signal/slot mechanism, which I think can be significantly slower
that direct function calls. From rough testing, plotting a graph with a
million points takes about 3x more time with the code you posted.
Jérôme
Le 24/02/2014 08:10, Daniel J Sebald a écrit :
> On 02/24/2014 12:16 AM, sfeam wrote:
>> On Sunday, 23 February 2014 11:22:42 PM Daniel J Sebald wrote:
>>> On 02/23/2014 10:35 PM, sfeam wrote:
>>>> On Sunday, 23 February 2014 10:04:42 PM Daniel J Sebald wrote:
>>>>>
>>>>> // Set plot size
>>>>> if (qt_setSize)
>>>>> {
>>>>> term->xmax = qt_oversampling*qt_setWidth;
>>>>> term->ymax = qt_oversampling*qt_setHeight;
>>>>> qt_setSize = false;
>>>>> }
>>>>>
>>>>> In a separate thread, the "term->xmax =" will be done asynchronously.
>>>>> Hence a mutex/wait is needed to make sure the code in the separate
>>>>> thread has updated term->xmax and term->max before the core thread can
>>>>> continue onward.
>>>>
>>>> Ah. Now I'm with you.
>>>> Yeah, this is the piece of code that has changed the most in qt
>>>> because nothing seems to work properly on both linux and OSX.
>>>> You are quite correct that term->foo should not be referenced in
>>>> this part of the terminal driver. It is supposed to return the revised
>>>> font information via an event GP_fontprops. And it _was_ doing that
>>>> at one point. I've now lost track of all the work-arounds and what
>>>> exactly they fixed, but certainly it would be good if you can get
>>>> back to the original intent. You can look at other terminal
>>>> drivers as a model if needed.
>>>
>>> OK, thanks. I'll look that over. I think that may be the one thing
>>> that isn't working yet for what I've done, i.e., some text isn't showing
>>> up probably owing to the core is told the font height is zero.
>>
>> Let me summarize a bit of the history.
>>
>> - The core code and the qt_term bits of Qt are in the same process.
>> Call this "inboard"
>> The screen display is being managed by a separate Qt process.
>> Call this "outboard"
>> The inboard and outboard processes operate asynchronously.
>>
>> - To reserve space for some text element on the next plot, the core
>> code needs to know how big the current font is so it sends a query
>> to the inboard terminal driver. This request can come either via
>> term->set_font() or in enhanced text mode via term->put_text.
>>
>> - It would be simplest if the inboard terminal driver could just reply
>> immediately with the requested font metrics. No communication back
>> and forth with the outboard terminal driver is required. This is what
>> the Qt terminal used to do, and still does in version 4.6.
>> The font size information is obtained by calling
>> QFontMetrics metrics(QFont(qt_currentFontName, qt_currentFontSize));
>> Since the inboard driver and the core code are in the same process,
>> the inboard driver can just set term->h_char and term->v_char
>> directly and that's the end of it.
>>
>> - Now here comes the problem. Apparently calling QFontMetrics without
>> there being a full QApplication and maybe [not sure] an event
>> doesn't work properly. In particular is was causing problems
>> on OSX, and it was ugly even on linux.
>> See the comments in version 4.6 qt_term.cpp
>> // Create a QApplication without event loop for QObject's that need it,
>> // namely font handling
>> // A better strategy would be to transfer the font handling to the
>> // QtGnuplotWidget, but it would require
>> // some synchronization between the widget and the gnuplot process.
>>
>> - So in January Jérôme followed through on that comment and moved
>> the QFontMetrics call over to the outboard driver for version 4.7.
>> That simplified things in one way, but introduced a new complication.
>> Now the inboard driver has to send a change font request to the
>> outboard driver (a different process) and then wait for the resulting
>> size information to be sent back via a QEvent. That's where
>> waitforinput() and do_event() suddenly become involved where
>> they hadn't been previously. It was also slower, so he later
>> introduced a cache of font metrics on the inboard side but let's
>> disregard that for now.
>>
>> - So now I gather you are working to have an event loop in the
>> inboard driver again, although I've lost track of exactly why.
>
> Hopefully to make OSX and Windows behave the same as Unix. I don't know
> for sure because Qt is a big creature, but I suspect that if there is no
> event loop that timers aren't guaranteed to work and who knows what
> else. So by providing an event, I'm hoping Qt performs the same on all
> platforms. It's just following the examples that Qt documentation gives.
>
>
>> But in that case I think it makes sense to move the
>> font metrics query back into the inboard driver as well.
>> At which point we return to a setup in which waitforinput()
>> and do_event() are not involved in font processing.
>>
>> So much for font handling and the involvement of enhanced text
>> processing.
>>
>> There is a separate issue, however, that has been at the heart of
>> the recent attempts to get OSX working. Information about the
>> size of the display window necessarily comes from the outboard
>> driver, which is a separate process. Right now this window size
>> information is also passed using a GE_fontprops event, but there's no
>> good reason for that. It really should have a separate event type
>> to reduce confusion. But changing the name of the event wouldn't
>> change the information flow in any way.
>
> Oh, OK. Thanks. I've got the big picture now and I think we will
> probably converge on something here that is about right.
>
> I'll correct one thing in that list which is I believe the issue is not
> the presence of the QApplication in the main process, but that the
> QApplication cannot have an event loop. That is, one can't issue
> application.exec(), because doing so will hang gnuplot while
> application.exec() blocks to handle all of its realtime traffic
> (signals, slots, system calls, etc). Graphics in Qt must be done in a
> main thread; it can't be done in a separate thread. That leaves the Qt
> graphics code to be run in a separate process. (Note, I've fixed the
> process communications a bit so that there aren't these while loops that
> keep trying to establish a connection to some external service.)
>
> The QApplication issue is why I've set out to put the bulk of the
> terminal interface code in a separate thread because in the separate
> thread and event loop, i.e., thread.exec(), is perfectly fine. The
> thread acts on its own. My preliminary code indicates there is no
> problem sending signals back and forth across that thread. So, with an
> event loop running in the second thread, all the timers and whatever
> else should be fine--the only restriction being that graphics cannot be
> done there. If it turns out OSX doesn't behave the same in the thread
> that has an event loop, then something isn't right with Qt. (We'll know
> soon.)
>
> So here is what I'm aiming for right now, sort of combining everything:
>
> MAIN PROGRAM | QTHREAD || GNUPLOT_QT
> | || (outboard)
> | ||
> gnuplot core | QtTerminalInterface || Currently has a
> QtTerminalEmitter | (active event loop) || few things we hope
> (inactive event loop) | || to move back into
> | || QtTerminalInterface
>
> where the single line means separate thread and the double line means a
> separate process.
>
> OK, for the most part I'll be offline for the week.
>
> Dan
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>
|
|
From: Daniel J S. <dan...@ie...> - 2014-02-26 05:39:47
|
On 02/24/2014 09:59 AM, Jérôme Lodewyck wrote:
> Hi,
>
> I got a bit confused about your motivations for introducing an event
> loop in the "inboard" qt terminal. In qt_term.cpp, we use, by design, a
> very limited set of the Qt library, and rather push as much
> functionality as possible the in the gnuplot_qt "outboard" program.
That's all fine. I've no intention of moving anything from gnuplot_qt
over to the qt_term.cpp.
I've written several times that the reason for supplying an event loop
to the code is because I'm suspicious of the overall functionality of Qt
code on all platforms without the event loop. But read on below...
> And
> I think that this set of functions doesn't require an event loop to work
> properly (As a matter of fact, I can observe by commenting out the
> QCoreApplication declaration in qt_term.cpp that it doesn't even require
> an instantiation of QCoreApplication to work -- except from the Windows
> specific QCoreApplication::applicationDirPath() call). More
> specifically, we use
> - Storage classes (QString, QImage, QColor...) that just gather
> information but use no event mechanism.
> - QLocalSocket that, according to the Qt documentation, can work without
> an event loop:
> "Although QLocalSocket is designed for use with an event loop, it's
> possible to use it without one. In that case, you must use
> waitForConnected(), waitForReadyRead(), waitForBytesWritten(), and
> waitForDisconnected() which blocks until the operation is complete or
> the timeout expires.". In fact, I can imagine that the gnuplot inboard
> driver is perhaps precisely the kind of programs that the Qt developers
> had in mind when they wrote this sentence: a program that cannot run a
> Qt event loop because it implements its own event loop, but that still
> wants to send messages to another auxiliary program that runs a Qt event
> loop to manage a GUI.
Yes, probably so. I read the documentation about the event loop in some
of these socket functions.
> So could you point out a specific past or present problem in the Qt
> terminal that is due to the fact that no event loop is running in the
> inboard driver ?
The answer is "no". But the follow up to that is I really didn't search
for that. I'm trying to solve the problem of the Qt terminal apparently
not working on Mac OSX or Windows the way it should in combination with
the fact that I don't have Mac OSX or Windows. So I'm striving for an
event loop in hopes that makes the setup work better on all platforms. I
can't answer your question...that is, without looking into greater
detail of the Qt source code.
> To my knowledge, here are the issues that arose on non Linux platform:
> - OSX: in the past, there was no gnuplot_qt independent process, and the
> GUI was rather managed in a thread. As you point out, this is not
> supported in Qt, and while it kind of happened to work with Linux, it
> failed on OSX.
Yes, that's not right.
> - OSX: currently, the QtGnuplotWidget has a hard time to resize itself
> to the correct size, but it is specific to the gnuplot_qt outboard driver.
I thought Mojca said gnuplot Qt term on OSX is failing to run.
> - All platforms: waitForConnected() immediatly returns if not socket is
> found (in particular, this happens when the gnuplot_qt program is still
> being initialized), even when a timeout is set. This was solved by
> introducing a custom timeout mechanism, but could more elegantly be
> solved by some communication mechanism between gnuplot ("inboard") and
> gnuplot_qt ("outboard"). I can see that part of your code actually
> implements this.
Correct. That custom timeout is bad.
> Concerning QApplication vs. QCoreApplication & font metrics:
> Until recently, the inboard driver (qt_term.cpp) used to instantiate a
> QApplication to determine the font metrics (no event loop was required
> for this though, and more generally, no Qt event loop, nor Qt event
> based mechanism has ever been running in the main gnuplot thread). As
> Ethan said, this mechanism has been moved to the "outboard" gnuplot_qt
> program, and now the inboard driver only instantiate a QCoreApplication
> (which, as I said above, even turns out not to be necessary in practice,
> although the Qt documentation does not certify this point). The
> motivation for this was a significant performance increase: the
> initialization of a QApplication requires more than 0.5 s against a few
> 10ms for a QCoreApplication. Moving back the font metric business in the
> main gnuplot program will require to instantiate again a QApplication
> and thus cancel this performance improvement.
The 0.5 s happened every time a new Qt window is opened?
> Concerning the code you propose: apart from the synchronization problems
> between the Qt event loop and the main thread that you mention, I am
> worried about the performances. As far as I can see, the communication
> between the main thread and the thread running the Qt event loop relies
> on the signal/slot mechanism, which I think can be significantly slower
> that direct function calls. From rough testing, plotting a graph with a
> million points takes about 3x more time with the code you posted.
It will be a performance loss, yes. I can't see it being 3x, though, if
I were to iron out some things. OK, so just a couple points:
1) Mojca reports the trial-and-error code I sent runs similar to what
I'm seeing. So we know we should be able to achieve platform independence.
2) I think the thing we agree on is that the way the QProcess is created
and the way the link is established to gnuplot_qt isn't very good. That
while-loop approach shouldn't be needed.
So, how about as a first step this weekend I take just the QProcess
startup code and make a nice diff/patch for you to try out and tweak how
you see fit. We'll see if that much gets us to a functioning OSX Qt
terminal. Simplifying that should clear things up so that if OSX Qt
term still doesn't function we might find the source of the problem.
If it turns out that an event loop issue is a source of problem, we can
probably split the API into those that do and don't need an event loop.
For example, I doubt the critical slowdown for the 1 M point example
you cite needs to use signal/slots and can go directly to the socket.
It's probably one move after another. But just speculation. First
let's tackle the comm-link code.
Dan
|
|
From: Jérôme L. <lod...@us...> - 2014-02-27 06:46:50
|
Le mardi 25 février 2014 23:39:38 Daniel J Sebald a écrit : > > So could you point out a specific past or present problem in the Qt > > terminal that is due to the fact that no event loop is running in the > > inboard driver ? > > The answer is "no". But the follow up to that is I really didn't search > for that. I'm trying to solve the problem of the Qt terminal apparently > not working on Mac OSX or Windows the way it should in combination with > the fact that I don't have Mac OSX or Windows. So I'm striving for an > event loop in hopes that makes the setup work better on all platforms. I > can't answer your question...that is, without looking into greater > detail of the Qt source code. Maybe I am wrong, but I think that adding an event loop will not make things work better. All the difficulties we had to make the Qt terminal work on all platforms are related to correctly setting the size of the plot window and correctly integrating the initialization of the QLocalSocket into the workflow of the main gnuplot program; which are problems that you will also face with an event loop. If you are motivated to pursue your idea, we will be able to balance whether things are less quirky or not with an event loop, but as Ethan said, things are already working almost correctly. > > motivation for this was a significant performance increase: the > > initialization of a QApplication requires more than 0.5 s against a few > > 10ms for a QCoreApplication. Moving back the font metric business in the > > main gnuplot program will require to instantiate again a QApplication > > and thus cancel this performance improvement. > > The 0.5 s happened every time a new Qt window is opened? No, only for the first one. But my motivation for this patch was that I have developed a monitoring application that fires about 20 gnuplot instances that forward their plots to a single Qt application (much like the embed_example, but with a gnuplot process dedicated tor each plot). With qt_term.cpp using a QApplication, my application took more than 10s to start up. > So, how about as a first step this weekend I take just the QProcess > startup code and make a nice diff/patch for you to try out and tweak how > you see fit. Let's see ! Jérôme |
|
From: Daniel J S. <dan...@ie...> - 2014-02-27 07:20:56
|
On 02/27/2014 12:46 AM, Jérôme Lodewyck wrote: > Le mardi 25 février 2014 23:39:38 Daniel J Sebald a écrit : >>> So could you point out a specific past or present problem in the Qt >>> terminal that is due to the fact that no event loop is running in the >>> inboard driver ? >> >> The answer is "no". But the follow up to that is I really didn't search >> for that. I'm trying to solve the problem of the Qt terminal apparently >> not working on Mac OSX or Windows the way it should in combination with >> the fact that I don't have Mac OSX or Windows. So I'm striving for an >> event loop in hopes that makes the setup work better on all platforms. I >> can't answer your question...that is, without looking into greater >> detail of the Qt source code. > > Maybe I am wrong, but I think that adding an event loop will not make things > work better. All the difficulties we had to make the Qt terminal work on all > platforms are related to correctly setting the size of the plot window I just sent a post identifying where the problem is with sizing, but it will take a little more to fix properly. > and > correctly integrating the initialization of the QLocalSocket into the workflow > of the main gnuplot program; which are problems that you will also face with > an event loop. > If you are motivated to pursue your idea, we will be able to balance whether > things are less quirky or not with an event loop, but as Ethan said, things > are already working almost correctly. The event loop is on hold. Let's clean up QProcess, QLocalSocket workflow first. Dan |
|
From: Bastian M. <bma...@we...> - 2014-02-26 06:46:42
|
Am 26.02.2014 06:39, schrieb Daniel J Sebald: > On 02/24/2014 09:59 AM, Jérôme Lodewyck wrote: >> So could you point out a specific past or present problem in the Qt >> terminal that is due to the fact that no event loop is running in the >> inboard driver ? > > The answer is "no". But the follow up to that is I really didn't search > for that. I'm trying to solve the problem of the Qt terminal apparently > not working on Mac OSX or Windows the way it should in combination with > the fact that I don't have Mac OSX or Windows. So I'm striving for an > event loop in hopes that makes the setup work better on all platforms. I > can't answer your question...that is, without looking into greater > detail of the Qt source code. > FWIW, Qt runs nicely on Windows. The only real issue there was to implement a way to handle all possible types of events (console, stdin, Qt pipe, Windows messages) at the same time. That has been solved in a non-portable way, but so is the code using select() on Unix-like systems. See the ChangeLog entry on 2014-02-14 and the SF patch tracker #645. Bastian |
|
From: Daniel J S. <dan...@ie...> - 2014-02-27 05:30:02
|
On 02/26/2014 12:46 AM, Bastian Märkisch wrote: > Am 26.02.2014 06:39, schrieb Daniel J Sebald: >> On 02/24/2014 09:59 AM, Jérôme Lodewyck wrote: >>> So could you point out a specific past or present problem in the Qt >>> terminal that is due to the fact that no event loop is running in the >>> inboard driver ? >> >> The answer is "no". But the follow up to that is I really didn't search >> for that. I'm trying to solve the problem of the Qt terminal apparently >> not working on Mac OSX or Windows the way it should in combination with >> the fact that I don't have Mac OSX or Windows. So I'm striving for an >> event loop in hopes that makes the setup work better on all platforms. I >> can't answer your question...that is, without looking into greater >> detail of the Qt source code. >> > > FWIW, Qt runs nicely on Windows. The only real issue there was to > implement a way to handle all possible types of events (console, stdin, > Qt pipe, Windows messages) at the same time. That has been solved in a > non-portable way, but so is the code using select() on Unix-like > systems. See the ChangeLog entry on 2014-02-14 and the SF patch tracker > #645. There is a lot to read there, but I get the overall idea. That select() (and analogous Windows code) looks like another case of something that might be better in the core code. Looking at qt_waitforinput() it seems it cycles through all the possible input modes allotting 1 millisecond to input. Why couldn't the terminal API have a timeout specifier, i.e., int qt_waitforinput(int ms) and then only have the Qt terminal pertinent code while gnuplot core watches for the other input routes? In any case, Qt does have a fairly robust key sequence mechanism, so CNTRL-C etc via gnuplot_qt shouldn't be too difficult. I placed a short stream output for the socket name when gnuplot_qt starts up and had no problem reading that standard output after waitForReadyRead(), etc. If I understand correctly, Qt queues the standard output until something reads it (which would normally be a console or something). Can't that be utilized? I switched the QProcess to the normal one, not detached, which gives better control. BTW, someone raised the question about persistent gnuplot_qt. Switching this QProcess at least doesn't hurt that prospect, as right now I have the gnuplot_qt window hanging around always after gnuplot exits (probably a bug). This weekend I'll attempt a good patch of this QProcess change for the patch tracker. Dan |
|
From: sfeam <sf...@us...> - 2014-02-27 06:04:15
|
On Wednesday, 26 February 2014 11:29:52 PM Daniel J Sebald wrote: > Why couldn't the terminal API have a timeout specifier, i.e., > > int qt_waitforinput(int ms) > > and then only have the Qt terminal pertinent code while gnuplot core > watches for the other input routes? Why would there be a timeout? If you are waiting for the user to finish inspecting their plot before continuing, that could take a long time. They might even go off for lunch before deciding to hit <cr> or click the mouse. They'd be rather annoyed if they came back from lunch to find that the plot had disappeared or was no longer selectable. Anyhow, it seems like as of today we have Qt working on all three platforms with maybe some quirks about slow font handling on OSX that we probably can't do a whole lot about. The one bit that I would say is still unsatisfactory in the Qt terminal is the fact that you can't set a size preference, the size you manually set via "set term qt size XX,YY" is ignored, and any resizing you do using the mouse is not persistent. If you can figure out how to fix that then Mojca and I at least would be happier, and probably future Qt users as well. Ethan |
|
From: Daniel J S. <dan...@ie...> - 2014-02-27 07:16:50
|
On 02/27/2014 12:02 AM, sfeam wrote:
> The one bit that I would say is still unsatisfactory in the Qt
> terminal is the fact that you can't set a size preference, the
> size you manually set via "set term qt size XX,YY" is ignored,
> and any resizing you do using the mouse is not persistent.
> If you can figure out how to fix that then Mojca and I at least
> would be happier, and probably future Qt users as well.
I'm not seeing that on Linux. The window size seems to work fine, as
far as I can tell. The information is being sent. Is some kind of
strange combination of switching between windows and sizes? The
following does seem some somewhat suspect:
// Set plot size
if (qt_setSize)
{
term->xmax = qt_oversampling*qt_setWidth;
term->ymax = qt_oversampling*qt_setHeight;
qt_setSize = false;
}
If 'size' is set in the options then the term structure is updated and
setSize cleared. But what if one then just switches the term number?
set term qt 4
plot x
set term qt 5 size 300,300
plot x**2
set term qt 4
plot x
I just confirmed that doesn't work properly. Is that what you are
referring to? Intermixing that qt_setSize test with the streams sent to
the outboard program might fix that size-persistence issue, i.e., the
size command doesn't get sent unless the user actually specified the size.
qt->out << GESetCtrl << qt_optionCtrl;
if (qt_setSize)
{
term->xmax = qt_oversampling*qt_setWidth;
term->ymax = qt_oversampling*qt_setHeight;
// Set plot size
qt->out << GESetWidgetSize << QSize(term->xmax,
term->ymax)/qt_oversampling;
// Initialize the scene
qt->out << GESetSceneSize << QSize(term->xmax,
term->ymax)/qt_oversampling;
qt_setSize = false;
}
qt->out << GEClear;
I tried something like that and it doesn't work exactly right either.
I'll have to look at that later.
BTW, I also notice that the maximum size is kind of small. Try
set term qt 6 size 800,800
plot x
and it doesn't look any larger than the default plot size. Going
smaller works though:
set term qt 5 size 300,300
plot x**2
Another BTW, the mouse scroll wheel button (center button) seems a
little odd. Using that in one Qt window causes plot activity, but it
also cause plot activity in another Qt plot window that is open.
Dan
|
|
From: Jérôme L. <lod...@us...> - 2014-02-27 17:32:01
|
Le 27/02/2014 18:17, Ethan A Merritt a écrit : > I must amend my previous statement. > "set term qt size" seems to have started working at some point (recently?). > It used to be ignored entirely, and I did not recheck. Actually, I think I broke it accidentally, but restored the correct behaviour shortly afterwards. But it still has some rough edges as reported by Daniel and Mojca > By the way, is there a some way to toggle the toolbar itself on and off? Adding such an option is possible, but the question is how to restore it if no other widget is visible and the mouse & key events are captures by gnuplot. > I'd like to conserve vertical screen space. It is still possible to have it clutter your horizontal screen space instead by moving it to the left edge of the window. > Alternatively, would it be possible to write the mouse coordinates into a widget > contained in the toolbar rather than into the status bar? That would also > conserve vertical space. And it might have the side-effect of working around > the current OSX glitch of not properly allocating space for the status bar. I have precisely tested this idea recently but didn't find the result visually appealing. I can submit a patch tough. Jérôme |
|
From: Mojca M. <moj...@gm...> - 2014-02-27 17:41:43
|
On Thu, Feb 27, 2014 at 6:31 PM, Jérôme Lodewyck wrote: > >> Alternatively, would it be possible to write the mouse coordinates into a widget >> contained in the toolbar rather than into the status bar? That would also >> conserve vertical space. And it might have the side-effect of working around >> the current OSX glitch of not properly allocating space for the status bar. > > I have precisely tested this idea recently but didn't find the result > visually appealing. I can submit a patch tough. Another option would be to write the coordinates on a semi-transparent frame somewhere in the bottom left corner when mouse is in the plotting area. And make it disappear when the mouse goes away or is inactive long enough (to allow to clearly see the whole plot). Something similar to the way that play/pause/stop buttons work in media players or in YouTube. When mouse is active, they are shown, after certain actions they are removed to allow to clearly see the video. Mojca |
|
From: Jérôme L. <lod...@us...> - 2014-02-27 06:55:42
|
Le mercredi 26 février 2014 22:02:13 sfeam a écrit : > The one bit that I would say is still unsatisfactory in the Qt > terminal is the fact that you can't set a size preference, the > size you manually set via "set term qt size XX,YY" is ignored, Is that issue specific to OS X ? On my system, it works as expected. > and any resizing you do using the mouse is not persistent. Does that mean that the window resizes to default after each plot command ? Jérôme |
|
From: Mojca M. <moj...@gm...> - 2014-02-27 07:58:02
Attachments:
qt_size_200_200.png
|
On Thu, Feb 27, 2014 at 7:55 AM, Jérôme Lodewyck wrote: > Le mercredi 26 février 2014 22:02:13 sfeam a écrit : >> The one bit that I would say is still unsatisfactory in the Qt >> terminal is the fact that you can't set a size preference, the >> size you manually set via "set term qt size XX,YY" is ignored, > > Is that issue specific to OS X ? On my system, it works as expected. On OS X it works *almost* as expected. - the first window is too small (x axis is covered with status bar), but the second plot fixes that - if I use "set term qt size 1000,800", the size of the first plot is right, but the window holding the plot is too small (it looks like it had the default size), so bottom right part of the plot is hidden (if I disable replotting on resize, resizing the window leads to the whole plot being visible) - if I use "set term qt size 200,200", I get what you see in attachment - it's hard to tell exactly, but very often the first plot after changing the size with "set term qt size" has wrong dimensions and the second one works just fine In principle it works most of the time (except when it doesn't). >> and any resizing you do using the mouse is not persistent. > > Does that mean that the window resizes to default after each plot command ? It doesn't resize for me. If I resize the window with mouse, the next plot properly adapts to that size. Mojca |
|
From: Ethan A M. <sf...@us...> - 2014-02-27 17:20:27
|
On Thursday, 27 February, 2014 07:55:33 Jérôme Lodewyck wrote: > Le mercredi 26 février 2014 22:02:13 sfeam a écrit : > > The one bit that I would say is still unsatisfactory in the Qt > > terminal is the fact that you can't set a size preference, the > > size you manually set via "set term qt size XX,YY" is ignored, > > Is that issue specific to OS X ? On my system, it works as expected. I must amend my previous statement. "set term qt size" seems to have started working at some point (recently?). It used to be ignored entirely, and I did not recheck. > > and any resizing you do using the mouse is not persistent. > Does that mean that the window resizes to default after each plot command ? I mean that if I resize with the mouse in one gnuplot session, the size is not remembered for the next gnuplot session. It's debatable whether this should be the default, but I would at least like to have an option in the tools widget to save the current size as the default. By the way, is there a some way to toggle the toolbar itself on and off? I'd like to conserve vertical screen space. Alternatively, would it be possible to write the mouse coordinates into a widget contained in the toolbar rather than into the status bar? That would also conserve vertical space. And it might have the side-effect of working around the current OSX glitch of not properly allocating space for the status bar. Ethan |
|
From: Ethan A M. <sf...@us...> - 2014-02-27 17:40:42
|
On Thursday, 27 February, 2014 09:17:33 Ethan A Merritt wrote: > By the way, is there a some way to toggle the toolbar itself on and off? > I'd like to conserve vertical screen space. Answering my own question... I find that if you right-click in the blank area of the toolbar there is an unlabeled check-box which, if clicked, makes the toolbar vanish. Ethan |
