Hi,
I'm working with the C Core api and I've noticed that a thread is spawned by a ResizeImage() call. This thread remains after the call and is sleeping. Even after calling DestroyMagick() this thread remains.
Normally this wouldn't be a problem, obviously the thread goes away when the program exists, however in my case, this is causing problems. My main process spawns a thread as a child process, this process then makes the ResizeImage() call. When I go to do a waitpid() on the process I spawned, it never returns since there is still a running thread in the thread group it is the leader of.
I call InitializeMagick() and DestroyMagick() in the main process, however, DestroyMagick() doesn't get called since it comes after the blocking waitpid(). Changing things up so it does get called, I still see the thread sleeping.
For a simple example, where you can see the thread exists and is sleeping, just take the example code at the bottom of the C Core api webpage, and add:
#include<stdlib.h>
and, one copy before and one copy after the ResizeImage() call of
system("ps -eLf | grep <program name>");
This will print out the information about the threads of the program running. There will probably be 4 lines, 2 for the original program and the thread it spawned, and one for the system call and one for the grep.
I can also just provide source to demonstrate this or where waitpid() never returns.
Is this the correct behavior from GraphicsMagick? and, is there anything I can do to stop the sleeping thread?
Thanks,
Mackenzie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
GraphicsMagick does not specifically start or stop any threads. Management of the worker threads is a function of the OpenMP implementation. For GCC this would be the GOMP library. Based on your user ID, perhaps you are using Apple's OS X so you might be using a different OpenMP library associated with LLVM. I have heard of similar issues on OS X before and Apple is/was aware of a problem with their implementation.
POSIX fork() has complexities when dealing with threaded programs. The threading implementation may encounter difficulties if it is not designed to handle fork (e.g. unlock locks held when the program forks). If the OS supports posix_spawn() then this may be a better choice to run a child process since it provides more control over how threads are handled.
Lastly, there are two exit calls. There is exit() to exit with all formal exit mechanisms (e.g. atexit() handlers) and there is _exit() to exit without invoking any exit handlers.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I apologize for not stating what operating system, I'm using Ubuntu 12.04 LTS, gcc 4.6.3, and libgomp1 4.6.3-1ubuntu5. I had never considered my user ID choice in that context, maybe I'll be rethinking my choice.
As far as thread spawning, I'm using clone(). However, even in a program which doesn't invlove any threads on my end, I still see the sleeping thread remaining. Attaching to the thread with gdb gives me a backtrace starting with clone(), then pthread_start(), then off to ??. Would need a full debug build to know better where it goes.
As you said, if you don't do any thread management, then this one remaning thread is going to be somehow related GOMP, so this probably isn't your problem.
thanks,
Mackenzie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The extra thread is likely something that OpenMP spawns to manage the worker threads. It would not go away since it wants to be ready to handle the next request in the least amount of time. I don't see any mention of implementation in the GOMP manual (http://http://gcc.gnu.org/onlinedocs/libgomp/).
I see that clone() supports many different options. I have no experience with it.
GM should be compatible with being used by different threads of a pthreads-based application because it uses pthreads-based mutex locking where needed.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I grabbed the openmp helloworld program off of wikipedia, it is definitly openmp that is creating and leaving the extra thread. I'm a little surprised there isn't a way to tell that thread to exit, but I couldn't find anything.
Pthreads use clone() (when it exists) as the fundamental threading mechanism. I haven't had any problems with GM, and the pthread-based locking works fine with our clone() threads, so I'm sure there are no issues the GM related to locking.
Thanks for you assistance, it looks like I'll just have to work around this last thread existing when I go to shutdown.
Mackenzie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I'm working with the C Core api and I've noticed that a thread is spawned by a ResizeImage() call. This thread remains after the call and is sleeping. Even after calling DestroyMagick() this thread remains.
Normally this wouldn't be a problem, obviously the thread goes away when the program exists, however in my case, this is causing problems. My main process spawns a thread as a child process, this process then makes the ResizeImage() call. When I go to do a waitpid() on the process I spawned, it never returns since there is still a running thread in the thread group it is the leader of.
I call InitializeMagick() and DestroyMagick() in the main process, however, DestroyMagick() doesn't get called since it comes after the blocking waitpid(). Changing things up so it does get called, I still see the thread sleeping.
For a simple example, where you can see the thread exists and is sleeping, just take the example code at the bottom of the C Core api webpage, and add:
and, one copy before and one copy after the ResizeImage() call of
system("ps -eLf | grep <program name>");This will print out the information about the threads of the program running. There will probably be 4 lines, 2 for the original program and the thread it spawned, and one for the system call and one for the grep.
I can also just provide source to demonstrate this or where waitpid() never returns.
Is this the correct behavior from GraphicsMagick? and, is there anything I can do to stop the sleeping thread?
Thanks,
Mackenzie
GraphicsMagick does not specifically start or stop any threads. Management of the worker threads is a function of the OpenMP implementation. For GCC this would be the GOMP library. Based on your user ID, perhaps you are using Apple's OS X so you might be using a different OpenMP library associated with LLVM. I have heard of similar issues on OS X before and Apple is/was aware of a problem with their implementation.
POSIX fork() has complexities when dealing with threaded programs. The threading implementation may encounter difficulties if it is not designed to handle fork (e.g. unlock locks held when the program forks). If the OS supports posix_spawn() then this may be a better choice to run a child process since it provides more control over how threads are handled.
Lastly, there are two exit calls. There is exit() to exit with all formal exit mechanisms (e.g. atexit() handlers) and there is _exit() to exit without invoking any exit handlers.
Thank you for the blazing fast response.
I apologize for not stating what operating system, I'm using Ubuntu 12.04 LTS, gcc 4.6.3, and libgomp1 4.6.3-1ubuntu5. I had never considered my user ID choice in that context, maybe I'll be rethinking my choice.
As far as thread spawning, I'm using clone(). However, even in a program which doesn't invlove any threads on my end, I still see the sleeping thread remaining. Attaching to the thread with gdb gives me a backtrace starting with clone(), then pthread_start(), then off to ??. Would need a full debug build to know better where it goes.
As you said, if you don't do any thread management, then this one remaning thread is going to be somehow related GOMP, so this probably isn't your problem.
thanks,
Mackenzie
The extra thread is likely something that OpenMP spawns to manage the worker threads. It would not go away since it wants to be ready to handle the next request in the least amount of time. I don't see any mention of implementation in the GOMP manual (http://http://gcc.gnu.org/onlinedocs/libgomp/).
I see that clone() supports many different options. I have no experience with it.
GM should be compatible with being used by different threads of a pthreads-based application because it uses pthreads-based mutex locking where needed.
I grabbed the openmp helloworld program off of wikipedia, it is definitly openmp that is creating and leaving the extra thread. I'm a little surprised there isn't a way to tell that thread to exit, but I couldn't find anything.
Pthreads use clone() (when it exists) as the fundamental threading mechanism. I haven't had any problems with GM, and the pthread-based locking works fine with our clone() threads, so I'm sure there are no issues the GM related to locking.
Thanks for you assistance, it looks like I'll just have to work around this last thread existing when I go to shutdown.
Mackenzie