Re: [pygccxml-development] Feature request: Creating thread safe methods

[Roman Y. <rom...@gm...>]

On 7/31/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> recently I added support for multi-threading in my Python/Maya package.
> This means whenever a thread is executing Python code it has to lock the
> interpreter before it can execute the code, so that other threads that
> also execute Python code won't interfere. This can be done by
> surrounding any Python invocation with PyGILState_Ensure() and
> PyGILState_Release().

May be we should add to boost::python library simple guard:
struct gilguard{
    gilguard(){ PyGILState_Ensure() ; }
    ~gilguard(){ PyGILState_Release(); }
};

Thus the generated code will be able to use the guard and it will be exception
neutral.

> However, this code is generated by pyplusplus and currently I don't see
> a way to inject the extra code.

Now can you see the problem with virtual functions and arguments policies?
They don't work well together :-(. The main problem is that function
signature is changed.

> So my suggestion would be to add a new
> decoration attribute "thread_safe" to the calldef decls which is a
> boolean that controls whether the code creators add the appropriate
> locking code or not.

This is really simple and easy to implement solution.

> Maybe this flag could even have three states so
> that in addition to True and False there's the option to use a global
> default value. This means the user could simply set the default value to
> specify if he wants to create thread-safe bindings or not.

mb.calldefs().thread_safe = True

The point is that we don't need global flag.

> Another question is if pyplusplus should ever add the locking functions
> to non-virtual methods. In principle, non-virtual methods are always
> thread-safe unless the method invokes Python code internally (which is
> probably a really rare case).

I don't know. Also I prefer to develop py++ having real use cases.
So the answer is next: if you don't have such use case we will not
write the code that
will treat it.

> Then there's another issue related to thread-safety. Locking the Python
> interpreter is one thing, but you also have to catch any exception that
> is raised in the Python code and take some appropriate actions (like
> reporting the exception and, in my case, returning an appropriate error
> status code to Maya).

Will exception neutral code will solve this problem? If not, I think py++
should have functionality, that will allow user to add/modify/rewrite
code generated
by py++. If this functionality will be added, we could implement
"arguments policy"
functionality out side of py++.

> This is something that cannot be done
> automatically by pyplusplus because it depends on the application how
> errors are reported back to the application. So this is where that "arg
> policies" mechanism to inject user code would be handy.
> I'm not sure if even the above locking code could be added via this
> mechanism.

Every overridable virtual member function looks like:

0 virtual int pure_virtual( int x ){
1
2    bp::override func_pure_virtual = this->get_override( "pure_virtual" );
3
4 return func_pure_virtual( x );
5
6 }

It seems, that user needs to add code to 1, 3, 5 and to be able to modify 2,4.
Am I right?

> While adding the "thread_safe" flag and updating the code creators seems
> to be pretty straightforward to me, I'm afraid adding the "arg policy"
> thing is a more delicate operation. I thought before I try to do this
> myself I rather post a message to discuss these issues before I get my
> hands dirty.
>
> So, any comments?

One more comment. How I see this feature is developed. py++ will
provide all services
you need to implement this feature out side of it. After "arguments
policy" functionality
will become stable, we will merge it with py++. Thus you will be able
to experiment
and move faster. From the other side, I will be able to concentrate on
next release.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/