Re: [pygccxml-development] Help needed with embedding
Brought to you by:
mbaas,
roman_yakovenko
Menu
▾
▴
Re: [pygccxml-development] Help needed with embedding
|
From: Roman Y. <rom...@gm...> - 2009-08-03 18:59:58
|
On Mon, Aug 3, 2009 at 6:02 PM, Berserker<ber...@ho...> wrote: > Hi, we are trying the Py++ library for embedding Python in our project. > We need to deal with GIL since we have to run multiple Python script > "concurrently". > > The first problem we have encountered is that Py++ doesn't seems to support > the ability to write a "member function adaptor" (that releases and aqcuires > the GIL around the function call): libtorrent has a nice example of what we > need here > http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup > At the moment we are "emulating" this feature with a custom transformer, > here is the code: > > class allow_threading_transformer_t(transformer.transformer_t): > def __init__(self, function): > transformer.transformer_t.__init__(self, function) > def __configure_sealed(self, controller): > controller.add_pre_call_code("allow_threading_guard guard;") > > def configure_mem_fun(self, controller): > self.__configure_sealed(controller) > def configure_free_fun(self, controller): > self.__configure_sealed(controller) > def configure_virtual_mem_fun(self, controller): > self.__configure_sealed(controller.default_controller) > > The only problem with this solution is that it creates too much code since > it requires a static function for each wrapped methods, for example: > Given the "Foo" class: > > class Foo > { > public: > void test_method_1(); > void test_method_2(); > }; > > Py++ creates: > > struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > { > > static void test_method_1( ::Foo & inst) { > allow_threading_guard guard; > inst.test_method_1(); > } > > static void test_method_2( ::Foo & inst) { > allow_threading_guard guard; > inst.test_method_2(); > } > > } > > ::boost::python::class_< Foo_wrapper >( "Foo" ) > .def("test_method_1", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_1 > ) ) > .def("test_method_2", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_2 > ) ) > ; > > insted of (based on the libtorrent code) only: > > boost::python::class_< Foo >("Foo") > .def("test_method_1", allow_threads(&Foo::test_method_1)) > .def("test_method_2", allow_threads(&Foo::test_method_2)) > ; > > Is there any hope for a patch to support "member function adaptors"? This is a quick solution. By the way Py++ defines gil_guard_t class( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/code_repository/gil_guard.py?revision=1428&view=markup ) I guess it is possible to create custom call policy, so the code will look like: > boost::python::class_< Foo >("Foo") > .def("test_method_1", &Foo::test_method_1, pythread_safe< return_value_policy< ... > >() ); Is this an open source project? > The second problem is related to virtual member functions, let's consider a > "test_method" in the Foo class > > class Foo > { > public: > virtual void test_method(); > }; > > Py++ now creates: > > struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > { > > virtual void test_method( ) { > namespace bpl = boost::python; > if( bpl::override func_test_method = this->get_override( > "test_method" ) ){ > bpl::object py_result = bpl::call<bpl::object>( > func_test_method.ptr() ); > } > else{ > ::Foo::test_method( ); > } > } > > static void default_test_method( ::Foo & inst ){ > allow_threading_guard guard; > if( dynamic_cast< Foo_wrapper * >( boost::addressof( inst ) ) ){ > inst.::Foo::test_method(); > } > else{ > inst.test_method(); > } > } > > } > > ::boost::python::class_< Foo_wrapper >( "Foo", ::boost::python::init< >() > ) > .def( "test_method", (void (*)( ::Foo & ))( > &Foo_wrapper::default_test_method ) ) > ; > > default_test_method works fine as in the first example but here we need to > restore the right PyThreadState before calling any CPython function in the > overridden "test_method" (this->get_override and bpl::call in this case). > At the moment we have no solution for this problem but the idea is to create > a custom base class for the Foo_wrapper class instead of > boost::python::wrapper (where we can save the PyThreadState and restore that > state before any call to the CPython API), here is the code: Well this problem is actually solved: mb = module_builder_t( ... ) tm = mb.mem_fun( 'test_method' ) gil_guard_code = """ pyplusplus::threading::gil_guard_t guard; """ tm.add_override_precall_code( gil_guard_code ) tm.add_default_precall_code( gil_guard_code ) tm.include_files.append( code_repository.gil_guard.file_name ) > Py++ code: > > def override_create_identifier(creator, full_name ): > if full_name == "::boost::python::wrapper": > return "allow_threads_wrapper" > > return full_name > > pyplusplus.code_creators.algorithm.create_identifier = > override_create_identifier > > C++: > > template <typename T> > class allow_threads_wrapper : public boost::python::wrapper<T> > { > public: > allow_threads_wrapper() : m_state(PyThreadState_Get()) { } > PyThreadState *m_state; > }; > > The last step should be the ability to add custom code before the > "get_override" call, something like: > > struct Foo_wrapper : ::Foo, allow_threads_wrapper< ::Foo > { > > virtual void test_method( ) { > allow_threading_swap swap(m_state); // allow_threading_swap is a > class that swap/restore the PyThreadState > > namespace bpl = boost::python; > if( bpl::override func_test_method = this->get_override( > "test_method" ) ){ > bpl::object py_result = bpl::call<bpl::object>( > func_test_method.ptr() ); > } > else{ > ::Foo::test_method( ); > } > } > > } > > Unfortunately the template of virtual_mem_fun in Py++ doesn't support the > ability to add custom code at the beginning of the function, is it possible > to make a small patch for this problem? Do you still think Py++ needs the patch? > Sorry for the long post, plz help HTH -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
