gnuplot-beta

Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Mojca M. <moj...@gm...>]

On Wed, Jan 4, 2012 at 23:09, Ethan A Merritt wrote:
> Mojca Miklavec wrote >
>
>> There are at least the following issues:
>> - configure script is unable to find Qt
>
> It uses pkg-config.  If your Qt installation does not come with a set of
> *.pc files then the configuration script will not work.

Even if it would come with *.pc scripts, Mac OS X doesn't ship
pkg-config by default, so it would be useless.

But the good news is that it is installed at a default location (with
default binary installer) and many programs, like Geant4 for example,
take this into account.

Again, I don't know how to use autotools, but the algorithm is simple enough:
- check if pkg-config finds Qt
- if not, check if flags that I mentioned work fine

>> - MOC and UIC are set to an empty string; here "moc" and "uic"
>> commands work fine, but I don't know why configure script doesn't find
>> them
>
> Because it queries pkg-config to find out where they are.
>   MOC=`pkg-config --variable=moc_location QtCore`
>
> By the way, what version of Qt are you trying to use?

I just realized that I had no Qt installed at all (I only had
developer tools which are installed at some weird location), so I
installed the latest version, 4.8.0, but I don't think that it really
matters. I was testing it long ago (around July when 4.8.0 wasn't
available yet, see "Any chance to fix gnuplot on mac?" thread, but
there was some additional private conversation with Jérôme Lodewyck
who helped me solve some issues) with exactly the same outcome.

If needed, I can probably figure out how to uninstall 4.8.0 and
install 4.7.*, but that won't help solve the two problems with
compiling (and last time when I tried it there were exactly the same
problems with running it).

> I've just been debugging configuration problems with Qt 4.7.4, where it turns
> out that I need to do the same thing in order to pick up locations for
> the rcc and lrelease utilities.
>
>> - if I fix the two above, qt terminal compiles fine, but I cannot plot anything:
>>
>> gnuplot> plot sin(x)
>> The process has forked and you cannot use this CoreFoundation
>> functionality safely. You MUST exec().
>> Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()
>> to debug.
>
> No idea what that's about. It is clearly some OSX weirdness.
> Googling that error message turns up this totally bizarre advice:

Last time I didn't find anything useful either. However google now
pointed me to the following:

http://gitorious.org/~friesoft/dinjam/friesoft-development/blobs/master/src/sources/data/standarddirs_mac.cpp

/**
 Calling CoreFoundation APIs (which is unavoidable in Qt/Mac) has
always had issues
 on Mac OS X, but as of 10.5 is explicitly disallowed with an exception.  As a
 result, in the case where we would normally fork and then dlopen
code, or continue
 to run other code, we must now fork-and-exec.

 See "CoreFoundation and fork()" at
http://developer.apple.com/releasenotes/CoreFoundation/CoreFoundation.html
*/

(The link is not particularly useful since it now points to Release
Notes of 10.7 instead of 10.5.)

or

http://stackoverflow.com/questions/3835247/do-work-out-of-process-in-a-cocoa-app

I would be willing to debug, but I have no idea how to do so. I
created a ticket:
    https://sourceforge.net/tracker/?func=detail&aid=3469233&group_id=2055&atid=102055
where I attached crash report (I just realized where systems stores
crash reports) if that is of any help, but I'm not sure neither how to
debug nor how to fix anything.

If you tell me how exactly to "exec()" after fork in qt_term.cpp, I
can test it, but I don't fully understand the code, so I'm not sure
what exactly to change and how.

It is pretty clear that problems start at this line in qt_term.cpp:
    QtGnuplotApplication application(argc, (char**)( NULL));
but after this line gets executed, I cannot even printf (whatever I
put into print, it doesn't get displayed until something semi-crashes
and printf sometimes starts working afer QApplication* application =
new QApplication(argc, (char**)( NULL)); again).

Mojca

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/04/2012 04:46 PM, Mojca Miklavec wrote:
> On Wed, Jan 4, 2012 at 23:09, Ethan A Merritt wrote:
>> Mojca Miklavec wrote>
>>
>>> There are at least the following issues:
>>> - configure script is unable to find Qt
>>
>> It uses pkg-config.  If your Qt installation does not come with a set of
>> *.pc files then the configuration script will not work.
>
> Even if it would come with *.pc scripts, Mac OS X doesn't ship
> pkg-config by default, so it would be useless.
>
> But the good news is that it is installed at a default location (with
> default binary installer) and many programs, like Geant4 for example,
> take this into account.
>
> Again, I don't know how to use autotools, but the algorithm is simple enough:
> - check if pkg-config finds Qt
> - if not, check if flags that I mentioned work fine
>
>>> - MOC and UIC are set to an empty string; here "moc" and "uic"
>>> commands work fine, but I don't know why configure script doesn't find
>>> them
>>
>> Because it queries pkg-config to find out where they are.
>>    MOC=`pkg-config --variable=moc_location QtCore`
>>
>> By the way, what version of Qt are you trying to use?
>
> I just realized that I had no Qt installed at all (I only had
> developer tools which are installed at some weird location), so I
> installed the latest version, 4.8.0, but I don't think that it really
> matters. I was testing it long ago (around July when 4.8.0 wasn't
> available yet, see "Any chance to fix gnuplot on mac?" thread, but
> there was some additional private conversation with Jérôme Lodewyck
> who helped me solve some issues) with exactly the same outcome.
>
> If needed, I can probably figure out how to uninstall 4.8.0 and
> install 4.7.*, but that won't help solve the two problems with
> compiling (and last time when I tried it there were exactly the same
> problems with running it).
>
>> I've just been debugging configuration problems with Qt 4.7.4, where it turns
>> out that I need to do the same thing in order to pick up locations for
>> the rcc and lrelease utilities.
>>
>>> - if I fix the two above, qt terminal compiles fine, but I cannot plot anything:
>>>
>>> gnuplot>  plot sin(x)
>>> The process has forked and you cannot use this CoreFoundation
>>> functionality safely. You MUST exec().
>>> Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()
>>> to debug.
>>
>> No idea what that's about. It is clearly some OSX weirdness.
>> Googling that error message turns up this totally bizarre advice:

The SMB probably means Samba server, which is a Windows emulation thing, 
I think.  This is probably just a consequence of the change that Mojca 
has found below.


> Last time I didn't find anything useful either. However google now
> pointed me to the following:
>
> http://gitorious.org/~friesoft/dinjam/friesoft-development/blobs/master/src/sources/data/standarddirs_mac.cpp
>
> /**
>   Calling CoreFoundation APIs (which is unavoidable in Qt/Mac) has
> always had issues
>   on Mac OS X, but as of 10.5 is explicitly disallowed with an exception.  As a
>   result, in the case where we would normally fork and then dlopen
> code, or continue
>   to run other code, we must now fork-and-exec.

So what you are seeing is the exception that OS X is now throwing when 
it sees you fork() and then not exec(), apparently.  My guess is this 
might be a security measure in some way.  (How?  I don't know, just 
guessing.)


> I would be willing to debug, but I have no idea how to do so. I
> created a ticket:
>      https://sourceforge.net/tracker/?func=detail&aid=3469233&group_id=2055&atid=102055
> where I attached crash report (I just realized where systems stores
> crash reports) if that is of any help, but I'm not sure neither how to
> debug nor how to fix anything.
>
> If you tell me how exactly to "exec()" after fork in qt_term.cpp, I
> can test it, but I don't fully understand the code, so I'm not sure
> what exactly to change and how.

It appears that the fork() is in the qt_init() routine:

236        pid = fork();
237	        if (pid < 0)
238	                fprintf(stderr, "Forking error\n");
239	        else if (pid == 0) // Child: start the GUI
240	        {
241	                signal(SIGINT, SIG_IGN); // Do not listen to SIGINT 
signals anymore
242	
243	#ifndef HAVE_QT_47
244	                /*
245	                 * FIXME: EAM Nov 2011
246	                 * It is better to use environmental variable
247	                 * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
248	                 * "raster" is ~5x faster than "native" (default).
249	                 * Unfortunately "opengl" isn't recognized on my 
test systems :-(
250	                 */
251	                // This makes a huge difference to the speed of 
polygon rendering.
252	                // Alternatives are "native", "raster", "opengl"
253	                QApplication::setGraphicsSystem("raster");
254	#endif
255	
256	                QtGnuplotApplication application(argc, (char**)( NULL));


> It is pretty clear that problems start at this line in qt_term.cpp:
>      QtGnuplotApplication application(argc, (char**)( NULL));
> but after this line gets executed, I cannot even printf (whatever I
> put into print, it doesn't get displayed until something semi-crashes
> and printf sometimes starts working afer QApplication* application =
> new QApplication(argc, (char**)( NULL)); again).

I would guess that the line Ethan conditioned isn't active when you 
compile.  That leaves "signal(SIGINT, SIG_IGN);" as the first line after 
the fork, and that is probably a CoreFunction routine itself so doesn't 
throw the exception or odd behavior.  Thus "QtGnuplotApplication 
application(argc, (char**)( NULL));" is the first line of code to follow 
the fork and probably throws the exception.  According to what you have 
found exec() should nearly follow fork() and you can't have all that 
code in between.  Thus as a first test, perhaps try moving the fork 
before the exec(), e.g.,

#ifndef HAVE_QT_47
                 /*
                  * FIXME: EAM Nov 2011
                  * It is better to use environmental variable
                  * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
                  * "raster" is ~5x faster than "native" (default).
                  * Unfortunately "opengl" isn't recognized on my test 
systems :-(
                  */
                 // This makes a huge difference to the speed of polygon 
rendering.
                 // Alternatives are "native", "raster", "opengl"
                 QApplication::setGraphicsSystem("raster");
#endif

                 QtGnuplotApplication application(argc, (char**)( NULL));

                 // Make sure the forked copy doesn't trash the history 
file
                 cancel_history();

                 // Load translations for the qt library
                 QTranslator qtTranslator;
                 qtTranslator.load("qt_" + QLocale::system().name(), 
QLibraryInfo::location(QLibraryInfo::TranslationsPath));
                 application.installTranslator(&qtTranslator);

                 // Load translations for the qt terminal
                 QTranslator translator;
                 translator.load("qtgnuplot_" + 
QLocale::system().name(), QTGNUPLOT_DATA_DIR);
                 application.installTranslator(&translator);

	        pid = fork();

	        if (pid < 0)
	                fprintf(stderr, "Forking error\n");
	        else if (pid == 0) // Child: start the GUI
	        {
	                signal(SIGINT, SIG_IGN); // Do not listen to SIGINT 
signals anymore
	                // Start
	                application.exec();
	                exit(0);
	        }


That may fail because of dependence on the fork() being called before 
some of those other routines, but it's a start.  (If it fails, try 
moving the application.exec() up near the fork, but you'll have to leave 
behind the exit(0) otherwise the translations will be missing.)

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[sfeam (E. Merritt) <eam...@gm...>]

On Wednesday, 04 January 2012, Daniel J Sebald wrote:
> On 01/04/2012 04:46 PM, Mojca Miklavec wrote:
> > On Wed, Jan 4, 2012 at 23:09, Ethan A Merritt wrote:
> >> Mojca Miklavec wrote>
> >> I've just been debugging configuration problems with Qt 4.7.4, where it turns
> >> out that I need to do the same thing in order to pick up locations for
> >> the rcc and lrelease utilities.
> >>
> >>> - if I fix the two above, qt terminal compiles fine, but I cannot plot anything:
> >>>
> >>> gnuplot>  plot sin(x)
> >>> The process has forked and you cannot use this CoreFoundation
> >>> functionality safely. You MUST exec().
> >>> Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()
> >>> to debug.> 
> 
> > Last time I didn't find anything useful either. However google now
> > pointed me to the following:
> >
> > http://gitorious.org/~friesoft/dinjam/friesoft-development/blobs/master/src/sources/data/standarddirs_mac.cpp
> >
> > /**
> >   Calling CoreFoundation APIs (which is unavoidable in Qt/Mac) has
> > always had issues
> >   on Mac OS X, but as of 10.5 is explicitly disallowed with an exception.  As a
> >   result, in the case where we would normally fork and then dlopen
> > code, or continue
> >   to run other code, we must now fork-and-exec.
> 
> So what you are seeing is the exception that OS X is now throwing when 
> it sees you fork() and then not exec(), apparently.  My guess is this 
> might be a security measure in some way.  (How?  I don't know, just 
> guessing.)
> 
> 
> > I would be willing to debug, but I have no idea how to do so. I
> > created a ticket:
> >      https://sourceforge.net/tracker/?func=detail&aid=3469233&group_id=2055&atid=102055
> > where I attached crash report (I just realized where systems stores
> > crash reports) if that is of any help, but I'm not sure neither how to
> > debug nor how to fix anything.
> >
> > If you tell me how exactly to "exec()" after fork in qt_term.cpp, I
> > can test it, but I don't fully understand the code, so I'm not sure
> > what exactly to change and how.
> 
> It appears that the fork() is in the qt_init() routine:
> 
> 236        pid = fork();
> 237	        if (pid < 0)
> 238	                fprintf(stderr, "Forking error\n");
> 239	        else if (pid == 0) // Child: start the GUI
> 240	        {
> 241	                signal(SIGINT, SIG_IGN); // Do not listen to SIGINT 
> signals anymore
> 242	
> 243	#ifndef HAVE_QT_47
> 244	                /*
> 245	                 * FIXME: EAM Nov 2011
> 246	                 * It is better to use environmental variable
> 247	                 * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
> 248	                 * "raster" is ~5x faster than "native" (default).
> 249	                 * Unfortunately "opengl" isn't recognized on my 
> test systems :-(
> 250	                 */
> 251	                // This makes a huge difference to the speed of 
> polygon rendering.
> 252	                // Alternatives are "native", "raster", "opengl"
> 253	                QApplication::setGraphicsSystem("raster");
> 254	#endif
> 255	
> 256	                QtGnuplotApplication application(argc, (char**)( NULL));
> 
> 
> > It is pretty clear that problems start at this line in qt_term.cpp:
> >      QtGnuplotApplication application(argc, (char**)( NULL));
> > but after this line gets executed, I cannot even printf (whatever I
> > put into print, it doesn't get displayed until something semi-crashes
> > and printf sometimes starts working afer QApplication* application =
> > new QApplication(argc, (char**)( NULL)); again).
> 
> I would guess that the line Ethan conditioned isn't active when you 
> compile.  That leaves "signal(SIGINT, SIG_IGN);" as the first line after 
> the fork, and that is probably a CoreFunction routine itself so doesn't 
> throw the exception or odd behavior.  Thus "QtGnuplotApplication 
> application(argc, (char**)( NULL));" is the first line of code to follow 
> the fork and probably throws the exception.  According to what you have 
> found exec() should nearly follow fork() and you can't have all that 
> code in between.  Thus as a first test, perhaps try moving the fork 
> before the exec(), e.g.,
> 
> #ifndef HAVE_QT_47
>                  /*
>                   * FIXME: EAM Nov 2011
>                   * It is better to use environmental variable
>                   * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
>                   * "raster" is ~5x faster than "native" (default).
>                   * Unfortunately "opengl" isn't recognized on my test 
> systems :-(
>                   */
>                  // This makes a huge difference to the speed of polygon 
> rendering.
>                  // Alternatives are "native", "raster", "opengl"
>                  QApplication::setGraphicsSystem("raster");
> #endif
> 
>                  QtGnuplotApplication application(argc, (char**)( NULL));
> 
>                  // Make sure the forked copy doesn't trash the history 
> file
>                  cancel_history();
> 
>                  // Load translations for the qt library
>                  QTranslator qtTranslator;
>                  qtTranslator.load("qt_" + QLocale::system().name(), 
> QLibraryInfo::location(QLibraryInfo::TranslationsPath));
>                  application.installTranslator(&qtTranslator);
> 
>                  // Load translations for the qt terminal
>                  QTranslator translator;
>                  translator.load("qtgnuplot_" + 
> QLocale::system().name(), QTGNUPLOT_DATA_DIR);
>                  application.installTranslator(&translator);
> 
> 	        pid = fork();
> 
> 	        if (pid < 0)
> 	                fprintf(stderr, "Forking error\n");
> 	        else if (pid == 0) // Child: start the GUI
> 	        {
> 	                signal(SIGINT, SIG_IGN); // Do not listen to SIGINT 
> signals anymore
> 	                // Start
> 	                application.exec();
> 	                exit(0);
> 	        }
> 
> 
> That may fail because of dependence on the fork() being called before 
> some of those other routines, but it's a start.  (If it fails, try 
> moving the application.exec() up near the fork, but you'll have to leave 
> behind the exit(0) otherwise the translations will be missing.)
> 
> Dan

Dan:

I could be wrong, but I don't think that  "application.exec()" is actually
an exec() call in the normal linux sense.   See for example
  http://stackoverflow.com/questions/3880693/qapplication-exec-creates-new-thread-process

In this case it just means "start looping over the draw commands until I
tell you to stop".   So your idea would be spot on if this were a real exec(),
but I'm afraid it's not going to help in this case.   But hey, I could be wrong.
It's simple enough to give it a try.

		Ethan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/04/2012 11:20 PM, sfeam (Ethan Merritt) wrote:
> On Wednesday, 04 January 2012, Daniel J Sebald wrote:
[snip]
> Dan:
>
> I could be wrong, but I don't think that  "application.exec()" is actually
> an exec() call in the normal linux sense.   See for example
>    http://stackoverflow.com/questions/3880693/qapplication-exec-creates-new-thread-process
>
> In this case it just means "start looping over the draw commands until I
> tell you to stop".   So your idea would be spot on if this were a real exec(),
> but I'm afraid it's not going to help in this case.   But hey, I could be wrong.
> It's simple enough to give it a try.

Oh, that's a problem then.  The the code following the fork probably 
needs to be written as a separate program and run via exec().  That's 
considerable more work.

Sorry, I can't find application.exec(), but if all it does is reads 
commands until done, then after that exit(0), why does this process need 
to be forked?  (Maybe you don't know the reason.)

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/04/2012 11:49 PM, Daniel J Sebald wrote:
> On 01/04/2012 11:20 PM, sfeam (Ethan Merritt) wrote:
>> On Wednesday, 04 January 2012, Daniel J Sebald wrote:
> [snip]
>> Dan:
>>
>> I could be wrong, but I don't think that  "application.exec()" is actually
>> an exec() call in the normal linux sense.   See for example
>>     http://stackoverflow.com/questions/3880693/qapplication-exec-creates-new-thread-process
>>
>> In this case it just means "start looping over the draw commands until I
>> tell you to stop".   So your idea would be spot on if this were a real exec(),
>> but I'm afraid it's not going to help in this case.   But hey, I could be wrong.
>> It's simple enough to give it a try.
>
> Oh, that's a problem then.  The the code following the fork probably
> needs to be written as a separate program and run via exec().  That's
> considerable more work.
>
> Sorry, I can't find application.exec(), but if all it does is reads
> commands until done, then after that exit(0), why does this process need
> to be forked?  (Maybe you don't know the reason.)

Answering my own question as best I can.  If I'm understanding 
correctly, Qt provides a mechanism for creating threads:

http://developer.qt.nokia.com/doc/qt-4.8/qthread.html

This is supposedly platform independent, which would suggest gnuplot's 
qt_init() shouldn't be using fork(), which is a platform dependent 
routine.  Maybe QThread is the way to go, and it takes care of all the 
fork/exec issues at its core.

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[sfeam (E. Merritt) <eam...@gm...>]

On Wednesday, 04 January 2012, Daniel J Sebald wrote:
> On 01/04/2012 11:20 PM, sfeam (Ethan Merritt) wrote:
> > On Wednesday, 04 January 2012, Daniel J Sebald wrote:
> [snip]
> > Dan:
> >
> > I could be wrong, but I don't think that  "application.exec()" is actually
> > an exec() call in the normal linux sense.   See for example
> >    http://stackoverflow.com/questions/3880693/qapplication-exec-creates-new-thread-process
> >
> > In this case it just means "start looping over the draw commands until I
> > tell you to stop".   So your idea would be spot on if this were a real exec(),
> > but I'm afraid it's not going to help in this case.   But hey, I could be wrong.
> > It's simple enough to give it a try.
> 
> Oh, that's a problem then.  The the code following the fork probably 
> needs to be written as a separate program and run via exec().  That's 
> considerable more work.
> 
> Sorry, I can't find application.exec()

It's inherited from the generic QtApplication.exec()

> , but if all it does is reads 
> commands until done, then after that exit(0), why does this process need 
> to be forked?  (Maybe you don't know the reason.)

The parent process is the usual gnuplot core code.
The child process is the Qt display event loop, which manages the plot
window, traps mouse events, and so on.
The parent and child communicate via a shared channel that was set up 
before the fork().  This stuff is all conveniently handled by the Qt
library routines, so you don't see much of it in the gnuplot code proper.

Here's how I understand it.  
Suppose the core code wants to draw a point at (x,y).
The call from the gnuplot side looks like this
   (term->point)(x,y,type)
But the different terminals do very different things.

Non-interactive terminals:   
	Call a library routine 
	(synchronous; part of the same process)
x11: 
	Send the command over a pipe to gnuplot_x11 
	(asynchronous; totally separate process)
qt:
	Send the command via a shared channel to the Qt event loop
	(asynchronous; forked copy of the original process)

I'm a bit fuzzy on how wxt works.  I _think_ it works the same as qt
except that the fork is hidden inside the wxWidgets support code.
If you run gnuplot using the wxt terminal you can see that it starts
up a separate thread.   For a very long time people had trouble with
wxt on OSX for I guess the same reason as we're now having trouble
with qt.  However, somehow the wxt code was tweaked to avoid this
specific OSX error (I am very hazy on that part!)  One might think
a similar tweak would work for Qt, but the web page that Mojca found
says that tripping over this OSX limitation in QtCore is "unavoidable".

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/05/2012 12:58 AM, sfeam (Ethan Merritt) wrote:
> On Wednesday, 04 January 2012, Daniel J Sebald wrote:
>> On 01/04/2012 11:20 PM, sfeam (Ethan Merritt) wrote:
>>> On Wednesday, 04 January 2012, Daniel J Sebald wrote:
>> [snip]
>>> Dan:
>>>
>>> I could be wrong, but I don't think that  "application.exec()" is actually
>>> an exec() call in the normal linux sense.   See for example
>>>     http://stackoverflow.com/questions/3880693/qapplication-exec-creates-new-thread-process
>>>
>>> In this case it just means "start looping over the draw commands until I
>>> tell you to stop".   So your idea would be spot on if this were a real exec(),
>>> but I'm afraid it's not going to help in this case.   But hey, I could be wrong.
>>> It's simple enough to give it a try.
>>
>> Oh, that's a problem then.  The the code following the fork probably
>> needs to be written as a separate program and run via exec().  That's
>> considerable more work.
>>
>> Sorry, I can't find application.exec()
>
> It's inherited from the generic QtApplication.exec()
>
>> , but if all it does is reads
>> commands until done, then after that exit(0), why does this process need
>> to be forked?  (Maybe you don't know the reason.)
>
> The parent process is the usual gnuplot core code.
> The child process is the Qt display event loop, which manages the plot
> window, traps mouse events, and so on.
> The parent and child communicate via a shared channel that was set up
> before the fork().  This stuff is all conveniently handled by the Qt
> library routines, so you don't see much of it in the gnuplot code proper.
>
> Here's how I understand it.
> Suppose the core code wants to draw a point at (x,y).
> The call from the gnuplot side looks like this
>     (term->point)(x,y,type)
> But the different terminals do very different things.
>
> Non-interactive terminals:
> 	Call a library routine
> 	(synchronous; part of the same process)
> x11:
> 	Send the command over a pipe to gnuplot_x11
> 	(asynchronous; totally separate process)
> qt:
> 	Send the command via a shared channel to the Qt event loop
> 	(asynchronous; forked copy of the original process)
>
> I'm a bit fuzzy on how wxt works.  I _think_ it works the same as qt
> except that the fork is hidden inside the wxWidgets support code.
> If you run gnuplot using the wxt terminal you can see that it starts
> up a separate thread.   For a very long time people had trouble with
> wxt on OSX for I guess the same reason as we're now having trouble
> with qt.  However, somehow the wxt code was tweaked to avoid this
> specific OSX error (I am very hazy on that part!)  One might think
> a similar tweak would work for Qt, but the web page that Mojca found
> says that tripping over this OSX limitation in QtCore is "unavoidable".

The statement at that reference is slightly vague on the matter:

/**
  Calling CoreFoundation APIs (which is unavoidable in Qt/Mac) has 
always had issues
  on Mac OS X, but as of 10.5 is explicitly disallowed with an 
exception.  As a
  result, in the case where we would normally fork and then dlopen code, 
or continue
  to run other code, we must now fork-and-exec.

  See "CoreFoundation and fork()" at 
http://developer.apple.com/releasenotes/CoreFoundation/CoreFoundation.html
*/

I think it means that Qt has to call some Apple-written APIs 
(Lion/Cocoa/Carbon/CoreServices).  But the "we" here--does that mean the 
Qt developers writing the Qt code now have to use fork-and-exec?  Or 
does that mean the people using the Qt utility have to use 
fork-and-exec?  If it is the former, then perhaps Qt users should not be 
using fork, but QThread.

Anyway, here is that missing discussion on CoreFoundation and fork():

http://osdir.com/ml/darwin-dev/2009-05/msg00020.html

"CoreFoundation cannot be used on the child-side of fork()"

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Mojca M. <moj...@gm...>]

On Thu, Jan 5, 2012 at 08:35, Daniel J Sebald wrote:
>
> If it is
> the former, then perhaps Qt users should not be using fork, but QThread.

As already said, I don't know anything about forking, but it is also
not clear to me why forking or creating a new binary is needed at all.

Here is an example of communication between GUI and Threads:

http://developer.qt.nokia.com/doc/qt-4.8/thread-basics.html#example-3-clock

Mojca

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/05/2012 05:10 AM, Mojca Miklavec wrote:
> On Thu, Jan 5, 2012 at 08:35, Daniel J Sebald wrote:
>>
>>   If it is
>> the former, then perhaps Qt users should not be using fork, but QThread.
>
> As already said, I don't know anything about forking, but it is also
> not clear to me why forking or creating a new binary is needed at all.

The reason is (as from what I read about Qt and my limited 
understanding) QApplication has to be the only event loop in a process. 
  It cannot run as an event loop along with a process that is also 
running an event loop.  In other words, gnuplot (an event loop) can't 
coexist with QApplication GUI (an event loop because it is processing 
button/keyboard/other events).

A fork is simply a means of creating another process, at least for most 
high level applications.  (Low level, there are other aspects of the 
duplication from fork that can be utilized, but not important here.)


> Here is an example of communication between GUI and Threads:
>
> http://developer.qt.nokia.com/doc/qt-4.8/thread-basics.html#example-3-clock

Those are threads within the Qt application, which isn't exactly the 
same as the OS threads commonly thought of.  The example you show has a 
main() routine for which QApplication is the only event loop.  The other 
elements are QObjects and such.  The issue is:

1) QApplication must be the only event loop, i.e., being able to run a 
QApplication GUI and gnuplot simultaneously.  (Hence the need for 
fork...why Qt doesn't handle this is what I asked before.)

2) Dealing with the Mac OS X limitation that CoreFoundation APIs (which 
Qt uses) cannot appear in a child process any longer.  (Hence the need 
for fork + exec and a separate binary.)

Dan

>
> Mojca
>
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual
> desktops for less than the cost of PCs and save 60% on VDI infrastructure
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

-- 

Dan Sebald
email: daniel(DOT)sebald(AT)ieee(DOT)org
URL: http://www(DOT)dansebald(DOT)com

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/05/2012 01:35 AM, Daniel J Sebald wrote:

> I think it means that Qt has to call some Apple-written APIs
> (Lion/Cocoa/Carbon/CoreServices).  But the "we" here--does that mean the
> Qt developers writing the Qt code now have to use fork-and-exec?  Or
> does that mean the people using the Qt utility have to use
> fork-and-exec?  If it is the former, then perhaps Qt users should not be
> using fork, but QThread.

Eh, the QThread isn't exactly analogous to process.  A QThread has its 
own stack, but memory is shared across threads.  It's sort of like 
threads inside the QApplication.  No use.

I see a statement that QApplication needs to be the one and only event 
loop in a process to work correctly.  Hence the need for a fork I would 
assume.  (I'm guessing the exit(0) in qt_init() applies to the child 
process.  Exiting gnuplot would make no sense.)  Why Qt doesn't handle 
this, I don't know.

Be that as it may, it might pay to step back a bit and consider just 
what the QApplication is doing.

But first, let me ask whether

		// Start
		application.exec();
		exit(0);

should be _exit(0) instead of exit(0).  A child shouldn't flush parent 
files:

http://www.cs.uleth.ca/~holzmann/C/system/pipeforkexec.html

The bigger question is whether this approach to creating a GUI 
QApplication, using fork() alone, is the best.  It is sort of a hybrid 
of two worlds, the QApplication being another process ID while at the 
same time still being a object instance in the parent space.  I suppose 
it works, but it's also somewhat of an odd feeling.  Also, these lines 
of code:

		// Make sure the forked copy doesn't trash the history file
		cancel_history();

and

	// The creation of a QApplication mangled our locale settings
#ifdef HAVE_LOCALE_H
	setlocale(LC_NUMERIC, "C");
	setlocale(LC_TIME, current_locale);
#endif

are somewhat kluge-like in the sense they shouldn't be necessary.  (I 
wonder if the mangled locale settings is because the child code used 
exit() instead of _exit().)

Jérôme, do you think it would make sense to set up initialization of the 
Qt terminal similar to the gplt_x11 terminal for better organization and 
enable function under OS X at the same time?  That is, put these lines 
into a small separate C executable:

main(appropriate vectors)
{
		signal(SIGINT, SIG_IGN); // Do not listen to SIGINT signals anymore

#ifndef HAVE_QT_47
		/*
		 * FIXME: EAM Nov 2011
		 * It is better to use environmental variable
		 * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
		 * "raster" is ~5x faster than "native" (default).
		 * Unfortunately "opengl" isn't recognized on my test systems :-(
		 */
		// This makes a huge difference to the speed of polygon rendering.
		// Alternatives are "native", "raster", "opengl"
		QApplication::setGraphicsSystem("raster");
#endif

		QtGnuplotApplication application(argc, (char**)( NULL));

		// Make sure the forked copy doesn't trash the history file
		cancel_history();

		// Load translations for the qt library
		QTranslator qtTranslator;
		qtTranslator.load("qt_" + QLocale::system().name(), 
QLibraryInfo::location(QLibraryInfo::TranslationsPath));
		application.installTranslator(&qtTranslator);

		// Load translations for the qt terminal
		QTranslator translator;
		translator.load("qtgnuplot_" + QLocale::system().name(), 
QTGNUPLOT_DATA_DIR);
		application.installTranslator(&translator);

		// Start
		application.exec();
		exit(0);

}

And in the qt_init() have something like the following (I left out 
details, but they are available in x11.trm file):

	if (pid < 0)
		fprintf(stderr, "Forking error\n");
	else if (pid == 0) // Child: start the GUI
	{
		execvp(qtterm_full_command_path, optvec);
		_exit(0);
	}

The advantage of using execvp is that it sort of wipes the slate clean 
for creating the QApplication.

Please give us your thoughts.

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/05/2012 05:17 AM, Daniel J Sebald wrote:

>  It is sort of a hybrid
> of two worlds, the QApplication being another process ID while at the
> same time still being a object instance in the parent space.  I suppose
> it works, but it's also somewhat of an odd feeling.

That's a wrong statement that I meant to delete before sending the last 
email, but for some reason it slipped through.

Dan

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Ethan A M. <sf...@us...>]

> On 01/05/2012 01:35 AM, Daniel J Sebald wrote:
> 
> Be that as it may, it might pay to step back a bit and consider just 
> what the QApplication is doing.
> 
> But first, let me ask whether
> 
>                 // Start
>                 application.exec();
>                 exit(0);
> 
> should be _exit(0) instead of exit(0).

Good point.  But that wouldn't help if there is an exit() call
inside application.exec() somewhere, e.g. from an error.

> Also, these lines of code:
> 
>                 // Make sure the forked copy doesn't trash the history file
>                 cancel_history();
> 
> and
> 
>         // The creation of a QApplication mangled our locale settings
> #ifdef HAVE_LOCALE_H
>         setlocale(LC_NUMERIC, "C");
>         setlocale(LC_TIME, current_locale);
> #endif
> 
> are somewhat kluge-like in the sense they shouldn't be necessary.  (I 
> wonder if the mangled locale settings is because the child code used 
> exit() instead of _exit().)

You are absolutely right that both of these are empirical fixes
(kluges if you like), and it would seem that there should be a cleaner solution.  
I would be happy to apply any better solution you can suggest!  
In the case of normal program execution, your suggestion to
use _exit(0) would make the call to cancel_history() unnecessary.
If there's an error exit, I'm not so sure.


> J�r�me, do you think it would make sense to set up initialization of the 
> Qt terminal similar to the gplt_x11 terminal for better organization and 
> enable function under OS X at the same time?  

Wouldn't the IPC channel have to be revised as well?
See the comments at the top of qt_flushOutBuffer() in qt_term.cpp

	Ethan



> That is, put these lines 
> into a small separate C executable:
> 
> main(appropriate vectors)
> {
>                 signal(SIGINT, SIG_IGN); // Do not listen to SIGINT signals anymore
> 
> #ifndef HAVE_QT_47
>                 /*
>                  * FIXME: EAM Nov 2011
>                  * It is better to use environmental variable
>                  * QT_GRAPHICSSYSTEM but this requires qt >= 4.7
>                  * "raster" is ~5x faster than "native" (default).
>                  * Unfortunately "opengl" isn't recognized on my test systems :-(
>                  */
>                 // This makes a huge difference to the speed of polygon rendering.
>                 // Alternatives are "native", "raster", "opengl"
>                 QApplication::setGraphicsSystem("raster");
> #endif
> 
>                 QtGnuplotApplication application(argc, (char**)( NULL));
> 
>                 // Make sure the forked copy doesn't trash the history file
>                 cancel_history();
> 
>                 // Load translations for the qt library
>                 QTranslator qtTranslator;
>                 qtTranslator.load("qt_" + QLocale::system().name(), 
> QLibraryInfo::location(QLibraryInfo::TranslationsPath));
>                 application.installTranslator(&qtTranslator);
> 
>                 // Load translations for the qt terminal
>                 QTranslator translator;
>                 translator.load("qtgnuplot_" + QLocale::system().name(), 
> QTGNUPLOT_DATA_DIR);
>                 application.installTranslator(&translator);
> 
>                 // Start
>                 application.exec();
>                 exit(0);
> 
> }
> 
> And in the qt_init() have something like the following (I left out 
> details, but they are available in x11.trm file):
> 
>         if (pid < 0)
>                 fprintf(stderr, "Forking error\n");
>         else if (pid == 0) // Child: start the GUI
>         {
>                 execvp(qtterm_full_command_path, optvec);
>                 _exit(0);
>         }
> 
> The advantage of using execvp is that it sort of wipes the slate clean 
> for creating the QApplication.
> 
> Please give us your thoughts.
> 
> Dan
> 
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual 
> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

Re: Qt on Mac (was: Getting ready to put out a 4.6 release candidate)

[Daniel J S. <dan...@ie...>]

On 01/05/2012 11:01 AM, Ethan A Merritt wrote:
>> On 01/05/2012 01:35 AM, Daniel J Sebald wrote:
>>
>> Be that as it may, it might pay to step back a bit and consider just
>> what the QApplication is doing.
>>
>> But first, let me ask whether
>>
>>                  // Start
>>                  application.exec();
>>                  exit(0);
>>
>> should be _exit(0) instead of exit(0).
>
> Good point.  But that wouldn't help if there is an exit() call
> inside application.exec() somewhere, e.g. from an error.

If there's an exit() inside QtCore I can't tell (e.g., user closes all 
terminals using the mouse), but I would hope that is done gracefully in 
Qt.  There is a qt_atexit() routine, so I would guess that is the last 
thing called when QApplication is complete.  But that still doesn't 
answer if Qt exits directly or always comes back to the application.exec().

There is an exit() inside QtGnuplotEventHandler::readEvent() that you 
added to deal with any zombie processes.  The other way to get rid of 
zombies is to use the "wait()" inside the parent process after sending a 
drawing command to the Qt terminal.  (In that case, maybe exit should be 
replaced by a change in signal to indicate to the parent that something 
went wrong.)  If zombie processes are destroyed along with the parent, 
having a zombie process around isn't necessarily bad, unless it is a CPU 
hogging infinite loop.


>> J�r�me, do you think it would make sense to set up initialization of the
>> Qt terminal similar to the gplt_x11 terminal for better organization and
>> enable function under OS X at the same time?
>
> Wouldn't the IPC channel have to be revised as well?
> See the comments at the top of qt_flushOutBuffer() in qt_term.cpp

I see the comment, but I'm not completely understanding.  I did read in 
the Qt documentation somewhere that this flush was necessary so that 
there aren't duplicate input data of some sort.

Dan