pygccxml-development

Re: [pygccxml-development] Function adaptors‏

[Berserker <ber...@ho...>]

> Right now I have some free time, I suggest you to verify that the
> committed code does what you want and report any problem with it.

Hi again Roman :)
I'm testing right now the svn version (updated to revision 1751) but 
I still have some problems in controlling the GIL (with virtual protected functions in particular).
The "adaptors" implementations is working for me, 
even if (because of my inexperience with Py++) I still don't understand why I can't 
define an adaptor in conjunction with a transformer (required to control the GIL in virtual functions), 
here it is the code I'm actually using:

# This is code where I iterate over each exported "c" class
>   for f in c.member_functions(allow_empty=True):
>      f.adaptor = "allow_threads"
>      if f.virtuality:
>         f.add_transformation(allow_threading_transformer_creator())                

# This code is still required since I need to "guard" the GIL 
# for virtual functions (default_controller in particular)
>   class allow_threading_transformer_t(transformer.transformer_t):
>      def __init__(self, function):
>         transformer.transformer_t.__init__(self, function)
>
>      def __configure_sealed(self, controller):
>         controller.add_pre_call_code("allow_threading_guard guard;")
>      
>      def configure_mem_fun(self, controller):
>         self.__configure_sealed(controller)
>
>      def configure_free_fun(self, controller):
>         self.__configure_sealed(controller)
>
>      def configure_virtual_mem_fun(self, controller):
>         self.__configure_sealed(controller.default_controller)        
>
>      def required_headers(self):
>         return []

# This code customizes the base wrapper class allowing to save a 
# reference to "PyThreadState_Get" and to restore the correct 
# interpreter later when the Python code in directly invoked from C++
>   def override_create_identifier(creator, full_name ):
>      if full_name == "::boost::python::wrapper":
>        return "osiris::PythonWrapper"
>
>      return full_name

# This is the templates customization code (getPythonThreadState 
# is a method of PythonWrapper)
>   templates.virtual_mem_fun.override = Template( os.linesep.join([
>      'virtual $return_type $function_name( $arg_declarations )$constness $throw{'
>      , '    osiris::PythonState state(getPythonThreadState());'
>      , '    namespace bpl = boost::python;'
>      , '    if( bpl::override $py_function_var = this->get_override( "$function_alias" ) ){'
>      , '        $declare_py_variables'
>      , '        $py_pre_call'
>      , '        ${save_py_result}bpl::call<bpl::object>( $py_function_var.ptr()$py_arg_expressions );'
>      , '        $py_post_call'
>      , '        $py_return'
>      , '    }'
>      , '    else{'
>      , '        state.leave();' #Release the lock since from here no CPython API will be called
>      , '        $cpp_return$wrapped_class::$function_name( $cpp_arg_expressions );'
>      , '    }'
>      , '}'
>      ]))
        
# Pure virtual functions is a little more "tricky" since we need to hold the GIL lock during
# the exception propagation: osiris::throw_locked_error_already_set is a function just
# throw a derived boost::python::error_already_set's class holding the GIL lock
>templates.pure_virtual_mem_fun.override = Template( os.linesep.join([
>      'virtual $return_type $function_name( $arg_declarations )$constness $throw{'
>      , '    boost::shared_ptr<osiris::PythonState> state(new osiris::PythonState(getPythonThreadState()));'
>      , '    namespace bpl = boost::python;'
>      , '    if( bpl::override $py_function_var = this->get_override( "$function_alias" ) ){'
>      , '        $declare_py_variables'
>      , '        $py_pre_call'
>      , '        ${save_py_result}bpl::call<bpl::object>( $py_function_var.ptr()$py_arg_expressions );'
>      , '        $py_post_call'
>      , '        $py_return'
>      , '    }'
>      , '    else{'
>      , '          PyErr_SetString(PyExc_NotImplementedError, "Attempted calling Pure Virtual function that is not implemented :$function_name");'
>      , '          osiris::throw_locked_error_already_set(state);'
>      , '    }'
>      , '}'
>      ]))

And now problems :)
Is it possible to customize "virtual protected functions" and 
"pure virtual protected functions" as above to have control of the GIL? 
It seems that those templates are ignored in case of 
protected methods or I'm maybe missing something...

Thanks again for your help, 
bye

 
_________________________________________________________________
Taglia i costi con Messenger! Chiama e videochiama da PC a PC!
http://www.messenger.it/videoconversazioni.aspx

Re: [pygccxml-development] Function adaptors‏

[Roman Y. <rom...@gm...>]

2009/8/10 Berserker <ber...@ho...>:
>> Right now I have some free time, I suggest you to verify that the
>> committed code does what you want and report any problem with it.
>
> Hi again Roman :)
> I'm testing right now the svn version (updated to revision 1751) but
> I still have some problems in controlling the GIL (with virtual protected
> functions in particular).
> The "adaptors" implementations is working for me,

Good

> even if (because of my inexperience with Py++) I still don't understand why
> I can't
> define an adaptor in conjunction with a transformer (required to control the
> GIL in virtual functions),

The reason is ideological. A function transformation feature allows
user to do virtually anything.
If I will start to add to the feature something else, like adaptors,
it will introduce constraints and thus will break the initial idea.

> here it is the code I'm actually using:
>
> # This is code where I iterate over each exported "c" class
>>   for f in c.member_functions(allow_empty=True):
>>      f.adaptor = "allow_threads"
>>      if f.virtuality:
>>         f.add_transformation(allow_threading_transformer_creator())
>>
>
> # This code is still required since I need to "guard" the GIL
> # for virtual functions (default_controller in particular)
>>   class allow_threading_transformer_t(transformer.transformer_t):
>>      def __init__(self, function):
>>         transformer.transformer_t.__init__(self, function)
>>
>>      def __configure_sealed(self, controller):
>>         controller.add_pre_call_code("allow_threading_guard guard;")
>>
>>      def configure_mem_fun(self, controller):
>>         self.__configure_sealed(controller)
>>
>>      def configure_free_fun(self, controller):
>>         self.__configure_sealed(controller)
>>
>>      def configure_virtual_mem_fun(self, controller):
>>         self.__configure_sealed(controller.default_controller)
>>
>>      def required_headers(self):
>>         return []
>
> # This code customizes the base wrapper class allowing to save a
> # reference to "PyThreadState_Get" and to restore the correct
> # interpreter later when the Python code in directly invoked from C++
>>   def override_create_identifier(creator, full_name ):
>>      if full_name == "::boost::python::wrapper":
>>        return "osiris::PythonWrapper"
>>
>>      return full_name
>
> # This is the templates customization code (getPythonThreadState
> # is a method of PythonWrapper)
>>   templates.virtual_mem_fun.override = Template( os.linesep.join([
>>      'virtual $return_type $function_name( $arg_declarations )$constness
>> $throw{'
>>      , '    osiris::PythonState state(getPythonThreadState());'
>>      , '    namespace bpl = boost::python;'
>>      , '    if( bpl::override $py_function_var = this->get_override(
>> "$function_alias" ) ){'
>>      , '        $declare_py_variables'
>>      , '        $py_pre_call'
>>      , '        ${save_py_result}bpl::call<bpl::object>(
>> $py_function_var.ptr()$py_arg_expressions );'
>>      , '        $py_post_call'
>>      , '        $py_return'
>>      , '    }'
>>      , '    else{'
>>      , '        state.leave();' #Release the lock since from here no
>> CPython API will be called
>>      , '        $cpp_return$wrapped_class::$function_name(
>> $cpp_arg_expressions );'
>>      , '    }'
>>      , '}'
>>      ]))

This is only +- 20 lines of code :-)

> # Pure virtual functions is a little more "tricky" since we need to hold the
> GIL lock during
> # the exception propagation: osiris::throw_locked_error_already_set is a
> function just
> # throw a derived boost::python::error_already_set's class holding the GIL
> lock
>>templates.pure_virtual_mem_fun.override = Template( os.linesep.join([
>>      'virtual $return_type $function_name( $arg_declarations )$constness
>> $throw{'
>>      , '    boost::shared_ptr<osiris::PythonState> state(new
>> osiris::PythonState(getPythonThreadState()));'
>>      , '    namespace bpl = boost::python;'
>>      , '    if( bpl::override $py_function_var = this->get_override(
>> "$function_alias" ) ){'
>>      , '        $declare_py_variables'
>>      , '        $py_pre_call'
>>      , '        ${save_py_result}bpl::call<bpl::object>(
>> $py_function_var.ptr()$py_arg_expressions );'
>>      , '        $py_post_call'
>>      , '        $py_return'
>>      , '    }'
>>      , '    else{'
>>      , '          PyErr_SetString(PyExc_NotImplementedError, "Attempted
>> calling Pure Virtual function that is not implemented :$function_name");'
>>      , '          osiris::throw_locked_error_already_set(state);'
>>      , '    }'
>>      , '}'
>>      ]))

And few more lines.

In my opinion the Py++ design works. May be I need to think how to
make templates "more" accessible.

> And now problems :)
> Is it possible to customize "virtual protected functions" and

Depends. It depends on how much of Py++ internals you want to know.

Function transformation feature can only work with public functions. (
hint: the "function wrapper" is free function without friendship )

What you can do is to to change
mem_fun_protected_v_wrapper_t::create_virtual_body method.

Take a look on it, I believe it will be easy to you to find out what
should be changed.

Of course you can do it without modifying Py++ code

class H:
    def xyz( self ):
      ....

def xyz( self ): <your code>

H.xyz = xyz

The only catch is that you are going to deal with internals and I
don't promise to be backward compatible here.

> "pure virtual protected functions" as above to have control of the GIL?

Can you show the use case?

> It seems that those templates are ignored in case of
> protected methods or I'm maybe missing something...

Yes as I explained FT feature works only for public functions. You can
always to do

#define private public

and include all your code :-))))

> Thanks again for your help,

You are welcome.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Function adaptors‏

[Berserker <ber...@ho...>]

>> Is it possible to customize "virtual protected functions" and
>> "pure virtual protected functions" as above to have control of the GIL?
>> It seems that those templates are ignored in case of
>> protected methods or I'm maybe missing something...

> Can you show the use case?

Sure :)
Let's suppose that we have two classes in C++: 
JobsManager (a "Singleton" class) and an interface class of a Job.

class JobsManager : public Singleton<JobsManager>
{
// ...

public:
   void addJob(boost::shared_ptr<Job> j)
   {
      m_jobs.push_back(j);
   }

   void runJobs()
   {
      for(Jobs::const_iterator i = m_jobs.begin(); i != m_jobs.end(); (*i)->run(), ++i);
   }
};

class Job
{
// ...

public:
   void run()
   {
      onInit();
      onRun();
   }

protected:
   virtual void onInit() 
   { 
      // ... 
   }

   virtual void onRun() = 0;
};

The "C++ framework" will load at startup all the python scripts where we 
can implement and register our custom jobs, something like:

class MyJob(Job):
   def onInit(self):
      # ...
      Job.onInit(self)

   def onRun(self):
      # ...

JobsManager.instance().addJob( myJobInstance )

The last step in C++ is to call JobsManager::instance().runJobs() (after loading all the python
scripts), but this function call will be from C++ to Python (not from Python to C++ as "usual"),
so we need to restore the correct PyThreadState where MyJob has been created: that's why I need
a custom "base wrapper class" (where I save the PyThreadState in the constructor).

Since I'm not a Python expert (this is my first experience with Python :) ), can you please 
help me, as suggested, in customizing the mem_fun_protected_v_wrapper_t and 
mem_fun_protected_pv_wrapper_t so that they will "share" the virtual_mem_fun/pure_virtual_mem_fun 
templates like in public virtual functions? Isn't possible to write something like: 

mem_fun_protected_v_wrapper_t.create_virtual_body = mem_fun_public_v_wrapper_t.create_virtual_body
mem_fun_protected_pv_wrapper_t.create_virtual_body = mem_fun_public_vp_wrapper_t.create_virtual_body

I just need to share the same code (in this way I don't have to deal with Py++ internals).
What about adding two more templates for protected functions instead?

Bye



_________________________________________________________________
Taglia i costi con Messenger! Chiama e videochiama da PC a PC!
http://www.messenger.it/videoconversazioni.aspx