pygccxml-development

[pygccxml-development] Code inserters/Arg policies

[Matthias B. <ba...@ir...>]

Hi,

I've just updated the "Code inserters" page in the wiki and added a 
section about non-pure virtual member functions:

https://realityforge.vrsource.org/view/PyPP/CodeInserter#Non_pure_virtual_member_function

The new section examines how the Boost.Python code would have to look 
like to be able to wrap non-pure virtual functions that have a signature 
that is not compatible with Python.

Any comments so far? Are there any use-cases I have forgotten? Are there 
other options how this could be implemented?

The next question (which the text does not deal with yet) is how to 
accomplish this using pyplusplus. The current proposal of a "code 
inserter" API is not able to deal with such things yet (not even with 
pure virtual functions), so this API still needs modifications.
Another question is whether pyplusplus should always create the code 
from option 3 or if the user should be able to specify that some 
use-cases are not important and it is ok that pyplusplus creates the 
simpler code. (So far I would tend to use option 3 in all cases as it is 
the most general option and I don't think it has a noticeable 
performance penalty).

- Matthias-

Re: [pygccxml-development] Code inserters/Arg policies

[Roman Y. <rom...@gm...>]

On 8/6/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> I've just updated the "Code inserters" page in the wiki and added a
> section about non-pure virtual member functions:
>
> https://realityforge.vrsource.org/view/PyPP/CodeInserter#Non_pure_virtual_member_function
>
> The new section examines how the Boost.Python code would have to look
> like to be able to wrap non-pure virtual functions that have a signature
> that is not compatible with Python.
>
> Any comments so far? Are there any use-cases I have forgotten? Are there
> other options how this could be implemented?
>
> The next question (which the text does not deal with yet) is how to
> accomplish this using pyplusplus. The current proposal of a "code
> inserter" API is not able to deal with such things yet (not even with
> pure virtual functions), so this API still needs modifications.
> Another question is whether pyplusplus should always create the code
> from option 3 or if the user should be able to specify that some
> use-cases are not important and it is ok that pyplusplus creates the
> simpler code. (So far I would tend to use option 3 in all cases as it is
> the most general option and I don't think it has a noticeable
> performance penalty).

I will comment your ideas later. Just wanted to drop another idea to
the existing ones:
Did you use ctypes package? If not, try to find time and to read about it
( http://starship.python.net/crew/theller/ctypes/ ). I think about introducing
fundamental data types to boost.python library or to pyplusplus and
teaching py++ to generate Python code.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Code inserters/Arg policies

[Roman Y. <rom...@gm...>]

On 8/6/06, Roman Yakovenko <rom...@gm...> wrote:
> On 8/6/06, Matthias Baas <ba...@ir...> wrote:
> > Hi,
> >
> > I've just updated the "Code inserters" page in the wiki and added a
> > section about non-pure virtual member functions:
> >
> > https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Non_pure_virtual_member_function
> >
> > The new section examines how the Boost.Python code would have to look
> > like to be able to wrap non-pure virtual functions that have a signature
> > that is not compatible with Python.
> >
> > Any comments so far?

I like very much the way you did it.
One question: it is not clear how is registration will look like?

> Are there any use-cases I have forgotten? Are there
> > other options how this could be implemented?

There is one more test I'd like to see: multiple inheritance.

Also, it could be nice to see, how this policy plays well with other.
For example,
return status as exception.

> > The next question (which the text does not deal with yet) is how to
> > accomplish this using pyplusplus. The current proposal of a "code
> > inserter" API is not able to deal with such things yet (not even with
> > pure virtual functions), so this API still needs modifications.

I think it is too early to discuss implementation details. In my
opinion the document
you wrote is missing one section: usage example.

mb = module_builder_t( ... )

spam = mb.member_function( '::Viking::spam' )
spam.what I should write here in order to get the desired effect

May be the right question to ask: what user interface decl_wrapper
should provide?

> > Another question is whether pyplusplus should always create the code
> > from option 3 or if the user should be able to specify that some
> > use-cases are not important and it is ok that pyplusplus creates the
> > simpler code. (So far I would tend to use option 3 in all cases as it is
> > the most general option and I don't think it has a noticeable
> > performance penalty).

This is an experimental feature. In my opinion first we should see how
well it works
and only then to make it more customizable. So, I agree with you.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Code inserters/Arg policies

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> One question: it is not clear how is registration will look like?

I have added a section "Summary" that contains the full code for option 
3 with additional comments:

https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Summary

I have also slightly modified the code from option 3 so that now the 
default_spam() method looks the same as in the initial version (i.e. it 
doesn't need any code modifiers).

>> Are there any use-cases I have forgotten? Are there
>> > other options how this could be implemented?
> 
> There is one more test I'd like to see: multiple inheritance.

Could you please provide more details about that? Multiple inheritance 
in C++ or in Python? Who is deriving from who? And on what instance 
should the method call be made?

> Also, it could be nice to see, how this policy plays well with other.
> For example, return status as exception.

Right, that's one of the next steps to examine at which places such 
policies need to add their own code. The above summary is supposed to 
help out with that.

> mb = module_builder_t( ... )
> 
> spam = mb.member_function( '::Viking::spam' )
> spam.what I should write here in order to get the desired effect

I think that should still be the decoration interface as already 
mentioned here:
https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Decoration

That is, there should be at least one decoration operation that allows 
me to attach "code modifier" objects to one particular function.
At this level the user doesn't even have to know about the problem with 
non-pure virtual functions, it just works the same as with non-virtual 
functions. Knowing about the details is only necessary when you want to 
write your own "code modifiers".

> Did you use ctypes package? If not, try to find time and to read about it
> ( http://starship.python.net/crew/theller/ctypes/ ). I think about 
> introducing
> fundamental data types to boost.python library or to pyplusplus and
> teaching py++ to generate Python code.

With Python 2.5 ctypes will be part of the standard Python library, so 
at least it might be worth investigating if we could re-use some code 
here. But to be able to comment on it I would need to know more details 
about what you're planning on doing...

- Matthias -

Re: [pygccxml-development] Code inserters/Arg policies

[Roman Y. <rom...@gm...>]

On 8/8/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> > One question: it is not clear how is registration will look like?
>
> I have added a section "Summary" that contains the full code for option
> 3 with additional comments:
>
> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Summary
>
> I have also slightly modified the code from option 3 so that now the
> default_spam() method looks the same as in the initial version (i.e. it
> doesn't need any code modifiers).

Two comments:
1.  May be I missed something, but it seems to me that registration should also
   contain Viking_Wrapper::spam, am I right?

2. In C++ arguments by reference used as: in, out, in and out. May be
   func_spam callable should take int and return int.

> > There is one more test I'd like to see: multiple inheritance.
>
> Could you please provide more details about that? Multiple inheritance
> in C++ or in Python? Who is deriving from who? And on what instance
> should the method call be made?

Sorry I was not clear, multiple inheritance in C++. py++ creates wrapper class
for every class in the hierarchy. I am sure this will work. But still,
I think it
is worse to add such test case.

>
> > mb = module_builder_t( ... )
> >
> > spam = mb.member_function( '::Viking::spam' )
> > spam.what I should write here in order to get the desired effect
>
> I think that should still be the decoration interface as already
> mentioned here:
> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Decoration

I see. What should be done with wrappers py++ generates? Do you wan to define
"default policies" ? "Default policies" does not have to be a class,
it could be just
a terminology.

> That is, there should be at least one decoration operation that allows
> me to attach "code modifier" objects to one particular function.
> At this level the user doesn't even have to know about the problem with
> non-pure virtual functions, it just works the same as with non-virtual
> functions. Knowing about the details is only necessary when you want to
> write your own "code modifiers".

I agree with the solution, but we will have to find better name. May
be "policies"
will be good enough?

> > Did you use ctypes package? If not, try to find time and to read about it
> > ( http://starship.python.net/crew/theller/ctypes/ ). I think about
> > introducing
> > fundamental data types to boost.python library or to pyplusplus and
> > teaching py++ to generate Python code.
>
> With Python 2.5 ctypes will be part of the standard Python library, so
> at least it might be worth investigating if we could re-use some code
> here. But to be able to comment on it I would need to know more details
> about what you're planning on doing...

Actually I thought about extending boost.python library with classes that have
interface similar to "fundamental types" in ctypes:

For example:

struct native_int{
    explicit native_int( int v ) : value( v ){}

   int& get_ref(){ return value; }

   const int& get_ref() const { return value; }

   int get_value(){ return value; }

private:
    int value;
};

C++:

void spam( int & ) - will be exported as is

>From Python:
import my_module
i = my_module.native_types.native_int( 23 )
my_module.spam( i )
print i.value

In my opinion, boost.python should understand that user passes native_int
and to generate an appropriate call.

If I understand right the implementation of ctypes, it works directly with
the C stack, we can do the same thing in C++.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Code inserters/Arg policies

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> I have added a section "Summary" that contains the full code for option
>> 3 with additional comments:
>>
>> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Summary
>>
>> I have also slightly modified the code from option 3 so that now the
>> default_spam() method looks the same as in the initial version (i.e. it
>> doesn't need any code modifiers).
> 
> Two comments:
> 1.  May be I missed something, but it seems to me that registration 
> should also
>   contain Viking_Wrapper::spam, am I right?

Well, no. Or let's rather say, yes, it should, but it doesn't work. When 
I changed the signature of default_spam() on the original code generated 
by py++ I got compile errors. This was the reason why I started 
investigating other options.

Actually, I haven't found any documentation about this special version 
of the def() function in the Boost.Python reference manual (it's only 
mentioned in the tutorial), so I don't know the requirements on the two 
function references (but it seems that they must have the same 
signature). And I also have to confess that I don't fully understand how 
it works and what it actually does....

> 2. In C++ arguments by reference used as: in, out, in and out. May be
>   func_spam callable should take int and return int.

Oh, sorry, I didn't mention it but I was assuming the argument of the 
spam() function is an output argument. But you're right, if it was in or 
in/out, the actual code would have to look slightly different.

> Sorry I was not clear, multiple inheritance in C++. py++ creates wrapper 
> class
> for every class in the hierarchy. I am sure this will work. But still,
> I think it is worse to add such test case.

You mean a C++ class Norwegian that is derived from Viking and that is 
exposed to Python and that has to pass all the tests that Viking had to 
pass, right?

>> I think that should still be the decoration interface as already
>> mentioned here:
>> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Decoration
> 
> I see. What should be done with wrappers py++ generates? 

Eventually, I hope that those wrappers are generated by py++ (and not by 
the high level API) so that there will never be two conflicting wrappers.

> Do you wan to define
> "default policies" ? "Default policies" does not have to be a class,
> it could be just a terminology.

Well, if a function only has the "default policies" (=lack of any "code 
modifiers") it just means that there will be no additional code inside 
the wrappers that transforms arguments. But a user doesn't have to 
specify that explicitly, so I'm afraid I don't exactly understand where 
we need to define any "default policies"...?

>> That is, there should be at least one decoration operation that allows
>> me to attach "code modifier" objects to one particular function.
>> At this level the user doesn't even have to know about the problem with
>> non-pure virtual functions, it just works the same as with non-virtual
>> functions. Knowing about the details is only necessary when you want to
>> write your own "code modifiers".
> 
> I agree with the solution, but we will have to find better name. May
> be "policies" will be good enough?

Couldn't this lead to confusion because people might mix it up with 
"call policies"?

- Matthias -

Re: [pygccxml-development] Code inserters/Arg policies

[Roman Y. <rom...@gm...>]

On 8/9/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> >> I have added a section "Summary" that contains the full code for option
> >> 3 with additional comments:
> >>
> >> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Summary
> >>
> >> I have also slightly modified the code from option 3 so that now the
> >> default_spam() method looks the same as in the initial version (i.e. it
> >> doesn't need any code modifiers).
> >
> > Two comments:
> > 1.  May be I missed something, but it seems to me that registration
> > should also
> >   contain Viking_Wrapper::spam, am I right?
>
> Well, no. Or let's rather say, yes, it should, but it doesn't work. When
> I changed the signature of default_spam() on the original code generated
> by py++ I got compile errors. This was the reason why I started
> investigating other options.
>
> Actually, I haven't found any documentation about this special version
> of the def() function in the Boost.Python reference manual (it's only
> mentioned in the tutorial), so I don't know the requirements on the two
> function references (but it seems that they must have the same
> signature). And I also have to confess that I don't fully understand how
> it works and what it actually does....

I see. We will have to get support from boost.python developers.

> > Sorry I was not clear, multiple inheritance in C++. py++ creates wrapper
> > class
> > for every class in the hierarchy. I am sure this will work. But still,
> > I think it is worse to add such test case.
>
> You mean a C++ class Norwegian that is derived from Viking and that is
> exposed to Python and that has to pass all the tests that Viking had to
> pass, right?

Yes.

> >> I think that should still be the decoration interface as already
> >> mentioned here:
> >> https://realityforge.vrsource.org/view/PyppApi/CodeInserter#Decoration
> >
> > I see. What should be done with wrappers py++ generates?
>
> Eventually, I hope that those wrappers are generated by py++ (and not by
> the high level API) so that there will never be two conflicting wrappers.
>
> > Do you wan to define
> > "default policies" ? "Default policies" does not have to be a class,
> > it could be just a terminology.
>
> Well, if a function only has the "default policies" (=lack of any "code
> modifiers") it just means that there will be no additional code inside
> the wrappers that transforms arguments. But a user doesn't have to
> specify that explicitly, so I'm afraid I don't exactly understand where
> we need to define any "default policies"...?

You just defined them. Also as I said, "default policies" is not a
class, but rather a
behaviour of py++, in case user did not set policies.

>
> Couldn't this lead to confusion because people might mix it up with
> "call policies"?

I forgot about it. May we think about "call policies" as another policy?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/