pygccxml-development

[pygccxml-development] Basic Py++ questions

[Paul M. <pa...@sc...>]

Hello Roman,

Thanks for your help so far!

I have a few beginners questions w.r.t. Py++:
- When selecting a class with the class_() method how intelligent is the 
matching? Does it do purely a string match on the argument you provide, 
or does it actually parse the argument and go from there?  I.e. I can 
write for example class_('::mynamespace::C'), class_('mynamespace::C') 
or class_('C'). Will these match different classes?
- I get warnings (W1005) during code generation about methods/functions 
using instances of non-public classes, with the message that the 
generated code will not compile :)  Is there a way to automatically 
disable generation of method wrappers that use non-public classes?
- Finally, if I .exclude() a certain class it seems that this doesn't 
propagate into excluding all methods/functions using that class (related 
to the previous question)?

Thanks in advance,
Paul

Re: [pygccxml-development] Basic Py++ questions

[Roman Y. <rom...@gm...>]

On Tue, Oct 28, 2008 at 3:27 PM, Paul Melis <pa...@sc...> wrote:
> Hello Roman,
>
> Thanks for your help so far!

You are welcome.

> I have a few beginners questions w.r.t. Py++:
> - When selecting a class with the class_() method how intelligent is the
> matching?

Well, it is able to do few things.

>Does it do purely a string match on the argument you provide,
> or does it actually parse the argument and go from there?

Neither. Parsing is error prone approach.

You can take a look on it:
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pygccxml_dev/pygccxml/declarations/matchers.py?revision=1428&view=markup

search for declaration_matcher_t class

> I.e. I can
> write for example class_('::mynamespace::C'), class_('mynamespace::C')
> or class_('C'). Will these match different classes?

The first and the last examples could match the different classes.
This depends on flags you pass to the method. If you use class_
method, you will get exception. I am not sure about the second
example. I guess it will not match any class, but I could be wrong.

I suggest you to take a look on the following document:
http://language-binding.net/pygccxml/query_interface.html

> - I get warnings (W1005) during code generation about methods/functions
> using instances of non-public classes, with the message that the
> generated code will not compile :)  Is there a way to automatically
> disable generation of method wrappers that use non-public classes?

Yes. Py++ warnings are explained here:
http://language-binding.net/pyplusplus/documentation/warnings.html

Basically you can check why declarations is not exportable :

<<untested>>

from pyplusplus import messages

f = <<<some function>>>
if not f.exportable and f.why_not_exportable().identifier == messages.W1005:
    f.exclude()

> - Finally, if I .exclude() a certain class it seems that this doesn't
> propagate into excluding all methods/functions using that class (related
> to the previous question)?

You mean you have class A{}; and some function f( A& ) and excluding
A, doesn't exclude f, right?

If so, Py++ doesn't do it for performance\memory reasons.

Now, when I think about it, it should be easy to add to the class_t
new function - "exclude_me_and_references_to_me" ( better name is
needed ) which will does exactly what you want. We can use
"i_depend_on_them" functionality.

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Basic Py++ questions

[Paul M. <pa...@sc...>]

Hi Roman,

Roman Yakovenko wrote:
> On Tue, Oct 28, 2008 at 3:27 PM, Paul Melis <pa...@sc...> wrote:
>   
>
>> I.e. I can
>> write for example class_('::mynamespace::C'), class_('mynamespace::C')
>> or class_('C'). Will these match different classes?
>>     
>
> The first and the last examples could match the different classes.
> This depends on flags you pass to the method. If you use class_
> method, you will get exception. I am not sure about the second
> example. I guess it will not match any class, but I could be wrong.
>
> I suggest you to take a look on the following document:
> http://language-binding.net/pygccxml/query_interface.html
>   
Surprisingly you say that class_('mynamespace::C') might not match any 
class, as the following example line is in the query interface docs:

do_smth = my_class.member_function( 'my_namespace::my_class::do_smth' )

I suppose that this form should match something as it is basically the most used form in C++ code. I haven't seen much code that consistently uses an absolute namespace path, like ::mynamespace::C, to refer to classes.

>> - I get warnings (W1005) during code generation about methods/functions
>> using instances of non-public classes, with the message that the
>> generated code will not compile :)  Is there a way to automatically
>> disable generation of method wrappers that use non-public classes?
>>     
>
> Yes. Py++ warnings are explained here:
> http://language-binding.net/pyplusplus/documentation/warnings.html
>
> Basically you can check why declarations is not exportable :
>
> <<untested>>
>
> from pyplusplus import messages
>
> f = <<<some function>>>
> if not f.exportable and f.why_not_exportable().identifier == messages.W1005:
>     f.exclude()
>
>   
Right, I did notice the docs about this and saw a quote from you 
somewhere that it allows you to exclude items based on warnings they 
generate.
The above indeed shows that, but the warnings API doesn't allow you to 
find WHICH items generated a certain warning so this would mean 
iterating over ALL items being exported and checking them for a certain 
warning?
>> - Finally, if I .exclude() a certain class it seems that this doesn't
>> propagate into excluding all methods/functions using that class (related
>> to the previous question)?
>>     
>
> You mean you have class A{}; and some function f( A& ) and excluding
> A, doesn't exclude f, right?
>   
Yes. I'm not certain how the generated wrapper would behave when class A 
is excluded, but f(A&) WOULD be wrapped.
If there was a class B derived from A that would be included in the 
wrappers then I could still call f(B)?
> If so, Py++ doesn't do it for performance\memory reasons.
>
> Now, when I think about it, it should be easy to add to the class_t
> new function - "exclude_me_and_references_to_me" ( better name is
> needed ) which will does exactly what you want. We can use
> "i_depend_on_them" functionality.
>   
It would be a nice addition to give users a choice of how exclusion of a 
class is treated. The current option is simply excluding only the 
wrapper for the class, but another variant would then be to exclude the 
class and all its uses.

Regards,
Paul

PS Why did you choose to name classes with the "_t" postfix, e.g. 
declaration_matcher_t versus the more common/pythonic 
DeclarationMatcher? And could a statement like "if not None is 
self.decl_type:" not also be written as "if self.decl_type is not None:"?

Re: [pygccxml-development] Basic Py++ questions

[Roman Y. <rom...@gm...>]

On Wed, Oct 29, 2008 at 11:38 AM, Paul Melis <pa...@sc...> wrote:
> Hi Roman,

Hello.

> Surprisingly you say that class_('mynamespace::C') might not match any
> class, as the following example line is in the query interface docs:
>
> do_smth = my_class.member_function( 'my_namespace::my_class::do_smth' )
>
> I suppose that this form should match something as it is basically the most
> used form in C++ code. I haven't seen much code that consistently uses an
> absolute namespace path, like ::mynamespace::C, to refer to classes.

Hmm. The functionality was implemented few years ago. I will create
tester which will show "the real behavior".

I will let you know the results.

> Right, I did notice the docs about this and saw a quote from you somewhere
> that it allows you to exclude items based on warnings they generate.
> The above indeed shows that, but the warnings API doesn't allow you to find
> WHICH items generated a certain warning so this would mean iterating over
> ALL items being exported and checking them for a certain warning?

You are right and this is the only solution. Not Py++, neither user
can implement such functionality efficiently.

Why do you want to exclude such declarations:
* to get "clean" build - without warnings. If so, in this specific
case you can decide that you trust Py++ and suppress the warning.
* other reasons - please explain.

>>>
>>> - Finally, if I .exclude() a certain class it seems that this doesn't
>>> propagate into excluding all methods/functions using that class (related
>>> to the previous question)?
>>>
>>
>> You mean you have class A{}; and some function f( A& ) and excluding
>> A, doesn't exclude f, right?
>>
>
> Yes. I'm not certain how the generated wrapper would behave when class A is
> excluded, but f(A&) WOULD be wrapped.

As warning says - you will get Boost.Python run-time error. It will
say that it can not invoke the function with the arguments you gave.

> If there was a class B derived from A that would be included in the wrappers
> then I could still call f(B)?

You should check, but if I remember right, you still can do this. It
is possible, that few functions defined in class A, will be redefined
in class B wrapper - for example pure virtual functions.

>>
>> If so, Py++ doesn't do it for performance\memory reasons.
>>
>> Now, when I think about it, it should be easy to add to the class_t
>> new function - "exclude_me_and_references_to_me" ( better name is
>> needed ) which will does exactly what you want. We can use
>> "i_depend_on_them" functionality.
>>
>
> It would be a nice addition to give users a choice of how exclusion of a
> class is treated. The current option is simply excluding only the wrapper
> for the class, but another variant would then be to exclude the class and
> all its uses.

I can do this. Please open the ticket on SourceForge.

> Regards,
> Paul
>
> PS Why did you choose to name classes with the "_t" postfix, e.g.
> declaration_matcher_t versus the more common/pythonic DeclarationMatcher?

This is the code convention I am used to. I am coming from C++ world.

> And could a statement like "if not None is self.decl_type:" not also be
> written as "if self.decl_type is not None:"?

It seems that this specific statement was written late at night :-)

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Basic Py++ questions

[Paul M. <pyp...@as...>]

Roman Yakovenko wrote:
>>> Why do you want to exclude such declarations:
>>> * to get "clean" build - without warnings. If so, in this specific
>>> case you can decide that you trust Py++ and suppress the warning.
>>> * other reasons - please explain.
>>>
>>>       
>> The reason is quite practical. Up till now I have not been able to even
>> compile my generated wrappers because of W1005 warnings.
>> I simply want to get some version of my wrappers working that I can get
>> started with. From there I can work on replacing/handling the issues w.r.t.
>> use of non-public classes.
>>     
>
> It sounds like a bug. Can you send me simple test case. I would like
> to improve the situation.
>   

After some digging it seems to be caused by the following pattern:
- the code I'm wrapping uses its own smart pointer class, say smart_ptr<T>
- the code contains a class, say C, with a protected inner class, say I
- the smart_ptr class has one or more methods that either return or use
a pointer to C::I

In the gccxml I see lines <Class ... artificial="1"
demangled="smart_ptr<C::I>;" mangled="..."  name="smart_ptr<C::I>" />,
which probably make Py++ generate wrappers for the smart_ptr methods.

Does this make sense?
Paul

Re: [pygccxml-development] Basic Py++ questions

[Roman Y. <rom...@gm...>]

On Wed, Oct 29, 2008 at 10:31 PM, Paul Melis
<pyp...@as...> wrote:
> Roman Yakovenko wrote:
>>>> Why do you want to exclude such declarations:
>>>> * to get "clean" build - without warnings. If so, in this specific
>>>> case you can decide that you trust Py++ and suppress the warning.
>>>> * other reasons - please explain.
>>>>
>>>>
>>> The reason is quite practical. Up till now I have not been able to even
>>> compile my generated wrappers because of W1005 warnings.
>>> I simply want to get some version of my wrappers working that I can get
>>> started with. From there I can work on replacing/handling the issues w.r.t.
>>> use of non-public classes.
>>>
>>
>> It sounds like a bug. Can you send me simple test case. I would like
>> to improve the situation.
>>
>
> After some digging it seems to be caused by the following pattern:
> - the code I'm wrapping uses its own smart pointer class, say smart_ptr<T>
> - the code contains a class, say C, with a protected inner class, say I
> - the smart_ptr class has one or more methods that either return or use
> a pointer to C::I
>
> In the gccxml I see lines <Class ... artificial="1"
> demangled="smart_ptr<C::I>;" mangled="..."  name="smart_ptr<C::I>" />,
> which probably make Py++ generate wrappers for the smart_ptr methods.

No, it doesn't help. I don't understand why Py++ still generates them.
Are the functions [pure] virtual?

Also, I suggest you to read the following link:
http://language-binding.net/pyplusplus/troubleshooting_guide/smart_ptrs/smart_ptrs.html

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Basic Py++ questions

[Paul M. <pa...@sc...>]

Roman Yakovenko wrote:
> On Wed, Oct 29, 2008 at 10:31 PM, Paul Melis
> <pyp...@as...> wrote:
>   
>> Roman Yakovenko wrote:
>>     
>>>>> Why do you want to exclude such declarations:
>>>>> * to get "clean" build - without warnings. If so, in this specific
>>>>> case you can decide that you trust Py++ and suppress the warning.
>>>>> * other reasons - please explain.
>>>>>
>>>>>
>>>>>           
>>>> The reason is quite practical. Up till now I have not been able to even
>>>> compile my generated wrappers because of W1005 warnings.
>>>> I simply want to get some version of my wrappers working that I can get
>>>> started with. From there I can work on replacing/handling the issues w.r.t.
>>>> use of non-public classes.
>>>>
>>>>         
>>> It sounds like a bug. Can you send me simple test case. I would like
>>> to improve the situation.
>>>
>>>       
>> After some digging it seems to be caused by the following pattern:
>> - the code I'm wrapping uses its own smart pointer class, say smart_ptr<T>
>> - the code contains a class, say C, with a protected inner class, say I
>> - the smart_ptr class has one or more methods that either return or use
>> a pointer to C::I
>>
>> In the gccxml I see lines <Class ... artificial="1"
>> demangled="smart_ptr<C::I>;" mangled="..."  name="smart_ptr<C::I>" />,
>> which probably make Py++ generate wrappers for the smart_ptr methods.
>>     
>
> No, it doesn't help. I don't understand why Py++ still generates them.
> Are the functions [pure] virtual?
>
> Also, I suggest you to read the following link:
> http://language-binding.net/pyplusplus/troubleshooting_guide/smart_ptrs/smart_ptrs.html
>   
I thought I had found what was going on. Take, for example, the 
following class:

class Outer
{
protected:
    struct Inner
    {
        Inner() {}

        Inner(const Inner& obj)
        {
            field = obj.field;
        }

        Inner& operator = (const Inner& obj)
        {
            field = obj.field;
            return *this;
        }

        int field;
    };
};

When I run the Python script at the end of this mail I get

INFO Parsing source file "all.h" ...
INFO gccxml cmd: /home/paul/local/bin/gccxml  -I"."   "all.h" 
-fxml="/tmp/tmpQ34w-p.xml"
INFO GCCXML version - 0.9

WARNING: Outer::Inner::Inner(Outer::Inner const & obj) [copy constructor]
 > compilation error W1005: Py++ cannot expose function that takes as
 > argument/returns instance of non-public class. Generated code will not
 > compile.

WARNING: Outer::Inner & Outer::Inner::operator=(Outer::Inner const & 
obj) [member operator]
 > compilation error W1005: Py++ cannot expose function that takes as
 > argument/returns instance of non-public class. Generated code will not
 > compile.

INFO: file "generated.cpp" - updated( 0.000000 seconds )

So it LOOKS like Py++ is trying to wrap the constructors of the 
protected inner class, which is obviously not going to work.
However, when I look in the generated code there's no wrappers for 
Outer::Inner, so the warning message's claim "Generated code will not 
compile." is misleading...
All the W1005 warnings I get might, however, still contain legimate ones 
that cause my compile problems. I'll need to dig further.

Regards,
Paul



The script:

#! /usr/bin/python
import os
import sys
import pygccxml
from pyplusplus import module_builder

mb = module_builder.module_builder_t(
        files=['all.h'],
        gccxml_path='/home/paul/local/bin/gccxml') #path to gccxml 
executable

mb.global_ns.include()

#Now it is the time to give a name to our module
mb.build_code_creator( module_name='doh' )

#I don't want absolute includes within code
mb.code_creator.user_defined_directories.append( os.path.abspath('.') )

#And finally we can write code to the disk
mb.write_module( os.path.join( os.path.abspath('.'), 'generated.cpp' ) )