Thread: [pygccxml-development] Help needed with embedding

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi, we are trying the Py++ library for embedding Python in our project.
We need to deal with GIL since we have to run multiple Python script "concurrently".

The first problem we have encountered is that Py++ doesn't seems to support the ability to write a "member function adaptor" (that releases and aqcuires the GIL around the function call): libtorrent has a nice example of what we need here http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup
At the moment we are "emulating" this feature with a custom transformer, here is the code:

class allow_threading_transformer_t(transformer.transformer_t):
   def __init__(self, function):
      transformer.transformer_t.__init__(self, function)        
   def __configure_sealed(self, controller):
      controller.add_pre_call_code("allow_threading_guard guard;")                
   def configure_mem_fun(self, controller):
      self.__configure_sealed(controller)
   def configure_free_fun(self, controller):
      self.__configure_sealed(controller)
   def configure_virtual_mem_fun(self, controller):
      self.__configure_sealed(controller.default_controller)

The only problem with this solution is that it creates too much code since it requires a static function for each wrapped methods, for example:
Given the "Foo" class:

class Foo
{
public:
   void test_method_1();
   void test_method_2();
};

Py++ creates:

struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {

   static void test_method_1( ::Foo & inst) {
      allow_threading_guard guard;
      inst.test_method_1();
   }

    static void test_method_2( ::Foo & inst) {
        allow_threading_guard guard;
        inst.test_method_2();
    }

}

::boost::python::class_< Foo_wrapper >( "Foo" )    
    .def("test_method_1", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_1 ) )    
    .def("test_method_2", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_2 ) )    
    ;

insted of (based on the libtorrent code) only:

boost::python::class_< Foo >("Foo")
    .def("test_method_1", allow_threads(&Foo::test_method_1))
    .def("test_method_2", allow_threads(&Foo::test_method_2))
    ;    

Is there any hope for a patch to support "member function adaptors"?

The second problem is related to virtual member functions, let's consider a "test_method" in the Foo class

class Foo
{
public:
    virtual void test_method();   
};

Py++ now creates:

struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {

    virtual void test_method(  ) {
        namespace bpl = boost::python;
        if( bpl::override func_test_method = this->get_override( "test_method" ) ){
            bpl::object py_result = bpl::call<bpl::object>( func_test_method.ptr() );
        }
        else{
            ::Foo::test_method(  );
        }
    }

    static void default_test_method( ::Foo & inst ){
        allow_threading_guard guard;
        if( dynamic_cast< Foo_wrapper * >( boost::addressof( inst ) ) ){
            inst.::Foo::test_method();
        }
        else{
            inst.test_method();
        }
    }

}

::boost::python::class_< Foo_wrapper >( "Foo", ::boost::python::init< >() )    
    .def( "test_method", (void (*)( ::Foo & ))( &Foo_wrapper::default_test_method ) )
    ;

default_test_method works fine as in the first example but here we need to restore the right PyThreadState before calling any CPython function in the overridden "test_method" (this->get_override and bpl::call in this case).
At the moment we have no solution for this problem but the idea is to create a custom base class for the Foo_wrapper class instead of boost::python::wrapper (where we can save the PyThreadState and restore that state before any call to the CPython API), here is the code:

Py++ code:

def override_create_identifier(creator, full_name ):
    if full_name == "::boost::python::wrapper":
        return "allow_threads_wrapper"

    return full_name

pyplusplus.code_creators.algorithm.create_identifier = override_create_identifier

C++:

template <typename T>
class allow_threads_wrapper : public boost::python::wrapper<T>
{
public:
    allow_threads_wrapper() : m_state(PyThreadState_Get()) { }
    PyThreadState *m_state;
};

The last step should be the ability to add custom code before the "get_override" call, something like:

struct Foo_wrapper : ::Foo, allow_threads_wrapper< ::Foo > {

    virtual void test_method(  ) {
        allow_threading_swap swap(m_state); // allow_threading_swap is a class that swap/restore the PyThreadState

        namespace bpl = boost::python;
        if( bpl::override func_test_method = this->get_override( "test_method" ) ){
            bpl::object py_result = bpl::call<bpl::object>( func_test_method.ptr() );
        }
        else{
            ::Foo::test_method(  );
        }
    }

}

Unfortunately the template of virtual_mem_fun in Py++ doesn't support the ability to add custom code at the beginning of the function, is it possible to make a small patch for this problem?

Sorry for the long post, plz help
Thanks

_________________________________________________________________
Porta Messenger in Vacanza! Scaricalo sul cellulare!
http://new.windowslivemobile.msn.com/IT-IT/windows-live-messenger/default.aspx