[pygccxml-development] Feature request: Creating thread safe methods

[Matthias B. <ba...@ir...>]

Hi,

recently I added support for multi-threading in my Python/Maya package. 
This means whenever a thread is executing Python code it has to lock the 
interpreter before it can execute the code, so that other threads that 
also execute Python code won't interfere. This can be done by 
surrounding any Python invocation with PyGILState_Ensure() and 
PyGILState_Release().
For my own commands this is no problem as the code that invokes a Python 
function is written by myself and so I can add the above functions.
But when a user registers a Python plugin class, some methods of this 
class will be called by Maya (i.e. from C++ code) and then the only 
place where the above functions can be called is the wrapper of the 
plugin class. See here:

http://mail.python.org/pipermail/c++-sig/2005-December/009953.html

(also see the other mails of the thread. As I understand the issue, the 
PyGILState_Ensure() function actually has to be called before 
get_override() is called)

However, this code is generated by pyplusplus and currently I don't see 
a way to inject the extra code. So my suggestion would be to add a new 
decoration attribute "thread_safe" to the calldef decls which is a 
boolean that controls whether the code creators add the appropriate 
locking code or not. Maybe this flag could even have three states so 
that in addition to True and False there's the option to use a global 
default value. This means the user could simply set the default value to 
specify if he wants to create thread-safe bindings or not.
Another question is if pyplusplus should ever add the locking functions 
to non-virtual methods. In principle, non-virtual methods are always 
thread-safe unless the method invokes Python code internally (which is 
probably a really rare case).

Then there's another issue related to thread-safety. Locking the Python 
interpreter is one thing, but you also have to catch any exception that 
is raised in the Python code and take some appropriate actions (like 
reporting the exception and, in my case, returning an appropriate error 
status code to Maya). This is something that cannot be done 
automatically by pyplusplus because it depends on the application how 
errors are reported back to the application. So this is where that "arg 
policies" mechanism to inject user code would be handy.
I'm not sure if even the above locking code could be added via this 
mechanism.

While adding the "thread_safe" flag and updating the code creators seems 
to be pretty straightforward to me, I'm afraid adding the "arg policy" 
thing is a more delicate operation. I thought before I try to do this 
myself I rather post a message to discuss these issues before I get my 
hands dirty.

So, any comments?

- Matthias -