I'm trying to use PythonQt in an analysis software for bus systems. For this I have to use PythonQt in a thread safe manner. I thought of 3 ways to active this:
1.A mutex to guard all access to PythonQts singleton instance and use unique modules as thread context. Obviously this is the easiest way out but comes with the price of not being able to scale across multiple cores and mutexes do come with an performance overhead.
2.Modify PythonQt. DeSingletonize it an make it start new Python processes with each PythonQt object. Now I have no idea if this is feasible, i have not worked with pure CPython before so I need your input on this one.
3.Modify my module to spawn a new process which intern use PythonQt and connect to them.
My question: with do you think make the most sense? How hard is #2 to realize? Is there any community interest in #2?
Kind regards Moritz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, regarding 1., that is doable but involves the locks that you mentioned. The locking needs to be on a finer granularity than the singleton access, I would do it on the different state of the singleton (e.g. the classinfo hashes).
If you have a look at PySide, they should have done the locking correctly.
The other issue is the Python GIL (global interpreter lock), this is not released by PythonQt, so while being in a Qt slot/signal call, the Python interpreter will be locked (so no other thread in Python can do anything).
Probably it would be enough to unlock the GIL at all places where the Qt code is entered, but I am not sure about that.
Apart from that, I do not understand what you mean with 2., do you really mean starting extra python processes, or do you think about running different Python interpreters in different threads? I think this feature of Python is seldomly used and I would not bet on it that it would work.
3. is definitely the way which does not involve work on PythonQt, so it might be easiest to implement.
All in all, I do not have any Python multithreading experience (mainly because the GIL seems to disallow real threading in python anyway), so maybe someone else could help you out? Having a look at PySide might also help (but PySide will not offer you easy embedding, which PythonQt does.)
regards,
Florian
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi everyone,
I'm trying to use PythonQt in an analysis software for bus systems. For this I have to use PythonQt in a thread safe manner. I thought of 3 ways to active this:
1.A mutex to guard all access to PythonQts singleton instance and use unique modules as thread context. Obviously this is the easiest way out but comes with the price of not being able to scale across multiple cores and mutexes do come with an performance overhead.
2.Modify PythonQt. DeSingletonize it an make it start new Python processes with each PythonQt object. Now I have no idea if this is feasible, i have not worked with pure CPython before so I need your input on this one.
3.Modify my module to spawn a new process which intern use PythonQt and connect to them.
My question: with do you think make the most sense? How hard is #2 to realize? Is there any community interest in #2?
Kind regards Moritz
Well, regarding 1., that is doable but involves the locks that you mentioned. The locking needs to be on a finer granularity than the singleton access, I would do it on the different state of the singleton (e.g. the classinfo hashes).
If you have a look at PySide, they should have done the locking correctly.
The other issue is the Python GIL (global interpreter lock), this is not released by PythonQt, so while being in a Qt slot/signal call, the Python interpreter will be locked (so no other thread in Python can do anything).
Probably it would be enough to unlock the GIL at all places where the Qt code is entered, but I am not sure about that.
Apart from that, I do not understand what you mean with 2., do you really mean starting extra python processes, or do you think about running different Python interpreters in different threads? I think this feature of Python is seldomly used and I would not bet on it that it would work.
3. is definitely the way which does not involve work on PythonQt, so it might be easiest to implement.
All in all, I do not have any Python multithreading experience (mainly because the GIL seems to disallow real threading in python anyway), so maybe someone else could help you out? Having a look at PySide might also help (but PySide will not offer you easy embedding, which PythonQt does.)
regards,
Florian