pygccxml-development

[pygccxml-development] Splitting main

[Allen B. <al...@vr...>]

Is there anyway to ask pyplusplus to split the module registration code 
in the module.main.cpp file?

I have a huge number of implicit conversions, ptr_to_python and 
overloaded methods that need to be registered.  (5 per class in the 
library)  Right now I am registering those all in the main module 
registration section (by adding code through the module builder).  The 
generated code works but it takes forever to compile this file.

In my specific case it would also work to register these methods inside 
the register_XXX_class() method that is created for each exposed class, 
but I would need a way to guarantee that the methods get registered 
against the global module scope.

Any suggestions on how to do either of these things?

Thanks,
Allen

Re: [pygccxml-development] Splitting main

[Roman Y. <rom...@gm...>]

On 9/3/06, Allen Bierbaum <al...@vr...> wrote:
> Is there anyway to ask pyplusplus to split the module registration code
> in the module.main.cpp file?
>
> I have a huge number of implicit conversions, ptr_to_python and
> overloaded methods that need to be registered.  (5 per class in the
> library)  Right now I am registering those all in the main module
> registration section (by adding code through the module builder).  The
> generated code works but it takes forever to compile this file.

You can generate some cpp and header file and add the call to the main.

mb.add_declaration_code( 'extern void register_my_xxx();' )
mb.add_registration_code( ' register_my_xxx();' )

In this way you don't have to introduce include or something else.
> In my specific case it would also work to register these methods inside
> the register_XXX_class() method that is created for each exposed class,
> but I would need a way to guarantee that the methods get registered
> against the global module scope.

Why?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Splitting main

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:
> On 9/3/06, Allen Bierbaum <al...@vr...> wrote:
>> Is there anyway to ask pyplusplus to split the module registration code
>> in the module.main.cpp file?
>>
>> I have a huge number of implicit conversions, ptr_to_python and
>> overloaded methods that need to be registered.  (5 per class in the
>> library)  Right now I am registering those all in the main module
>> registration section (by adding code through the module builder).  The
>> generated code works but it takes forever to compile this file.
> 
> You can generate some cpp and header file and add the call to the main.
> 
> mb.add_declaration_code( 'extern void register_my_xxx();' )
> mb.add_registration_code( ' register_my_xxx();' )

Although this would work I would rather generate all the code at the 
same time from the same generation script using py++.  As I understand 
this option I would have to generate the global scope methods separately.


> In this way you don't have to introduce include or something else.
>> In my specific case it would also work to register these methods inside
>> the register_XXX_class() method that is created for each exposed class,
>> but I would need a way to guarantee that the methods get registered
>> against the global module scope.
> 
> Why?

So the registered methods and conversions show up in the scope of the 
main module.  Or can I register a global scope method within the class 
registration code?

-Allen

Re: [pygccxml-development] Splitting main

[Roman Y. <rom...@gm...>]

On 9/4/06, Allen Bierbaum <al...@vr...> wrote:
> Roman Yakovenko wrote:
> > On 9/3/06, Allen Bierbaum <al...@vr...> wrote:
> >> Is there anyway to ask pyplusplus to split the module registration code
> >> in the module.main.cpp file?
> >>
> >> I have a huge number of implicit conversions, ptr_to_python and
> >> overloaded methods that need to be registered.  (5 per class in the
> >> library)  Right now I am registering those all in the main module
> >> registration section (by adding code through the module builder).  The
> >> generated code works but it takes forever to compile this file.
> >
> > You can generate some cpp and header file and add the call to the main.
> >
> > mb.add_declaration_code( 'extern void register_my_xxx();' )
> > mb.add_registration_code( ' register_my_xxx();' )
>
> Although this would work I would rather generate all the code at the
> same time from the same generation script using py++.  As I understand
> this option I would have to generate the global scope methods separately.

No. See this document:
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/techniques.html#python.reducing_compiling_time

> > In this way you don't have to introduce include or something else.
> >> In my specific case it would also work to register these methods inside
> >> the register_XXX_class() method that is created for each exposed class,
> >> but I would need a way to guarantee that the methods get registered
> >> against the global module scope.
> >
> > Why?
>
> So the registered methods and conversions show up in the scope of the
> main module.  Or can I register a global scope method within the class
> registration code?

Why do you care about conversions?


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Splitting main

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

> On 9/4/06, Allen Bierbaum <al...@vr...> wrote:
>
>> Roman Yakovenko wrote:
>> > On 9/3/06, Allen Bierbaum <al...@vr...> wrote:
>> >> Is there anyway to ask pyplusplus to split the module registration 
>> code
>> >> in the module.main.cpp file?
>> >>
>> >> I have a huge number of implicit conversions, ptr_to_python and
>> >> overloaded methods that need to be registered.  (5 per class in the
>> >> library)  Right now I am registering those all in the main module
>> >> registration section (by adding code through the module builder).  
>> The
>> >> generated code works but it takes forever to compile this file.
>> >
>> > You can generate some cpp and header file and add the call to the 
>> main.
>> >
>> > mb.add_declaration_code( 'extern void register_my_xxx();' )
>> > mb.add_registration_code( ' register_my_xxx();' )
>>
>> Although this would work I would rather generate all the code at the
>> same time from the same generation script using py++.  As I understand
>> this option I would have to generate the global scope methods 
>> separately.
>
>
> No. See this document:
> http://www.boost.org/libs/python/doc/tutorial/doc/html/python/techniques.html#python.reducing_compiling_time 
>

I think there is some confusiong here, so I probably didn't make myself 
clear to start with.  I am wrapping a very large C++ library.  I have 
been splitting the code generation across multiple files for several 
years now.  (with pyste it was using the --multiple options and with 
Py++ I am using the split module capabilities).  This all works 
correctly and the split is happening.

What is causing a performance problem currently is that I have to do the 
following code for every class (about 170 classes).

 bp::register_ptr_to_python< osg::LineChunkPtr >();
 bp::implicitly_convertible< osg::LineChunkRefPtr, osg::LineChunkPtr >();
 bp::implicitly_convertible< osg::LineChunkPtr, osg::StateChunkPtr >();
 bp::def("RefPtr", &pyopensg::ToRefPtr<osg::LineChunkPtr>);
 bp::def("FCPtr", &pyopensg::ToFcPtr<osg::LineChunkPtr>);

I need these registrations and methods so the libraries use of multiple 
smart pointer types will be supported correctly. 

I am currently registering them with the module builder:

      mb.add_registration_code("bp::register_ptr_to_python< osg::%sPtr 
 >();"%cinfo.name)  
      mb.add_registration_code("bp::implicitly_convertible< 
osg::%sRefPtr, osg::%sPtr >();"%(cinfo.name,cinfo.name))  
      mb.add_registration_code("bp::implicitly_convertible< osg::%sPtr, 
osg::%sPtr >();"%(cinfo.name,cinfo.parent))  
      mb.add_registration_code('bp::def("RefPtr", 
&pyopensg::ToRefPtr<osg::%sPtr>);'%cinfo.name)
      mb.add_registration_code('bp::def("FCPtr", 
&pyopensg::ToFcPtr<osg::%sPtr>);'%cinfo.name)

This works and compiles, but there is a performance hit during compiling 
because all of these registrations happen in the module.main.cpp file.  
I was hoping there may be some way to ask Py++ to split the 
registrations in this file across multiple .cpp files.  In a way 
basically extend the existing call hierarchy another level so that 
instead of just calling the class registration methods, the code in 
module.main.cpp could be output in a way that calls several other helper 
methods that in turn call the class registration methods and module 
scope registration methods like these.

It doesn't seem that this is supported though so I will investigate some 
other options.  I had tried to register some of this code with the class 
code adders in the past and it did not work.  I will try this again and 
see if I have better luck now.

-Allen

>
>> > In this way you don't have to introduce include or something else.
>> >> In my specific case it would also work to register these methods 
>> inside
>> >> the register_XXX_class() method that is created for each exposed 
>> class,
>> >> but I would need a way to guarantee that the methods get registered
>> >> against the global module scope.
>> >
>> > Why?
>>
>> So the registered methods and conversions show up in the scope of the
>> main module.  Or can I register a global scope method within the class
>> registration code?
>
>
> Why do you care about conversions?
>
>

Re: [pygccxml-development] Splitting main

[Roman Y. <rom...@gm...>]

On 9/4/06, Allen Bierbaum <al...@vr...> wrote:
> Roman Yakovenko wrote:
>
> > On 9/4/06, Allen Bierbaum <al...@vr...> wrote:
> >
> >> Roman Yakovenko wrote:
> >> > On 9/3/06, Allen Bierbaum <al...@vr...> wrote:
> >> >> Is there anyway to ask pyplusplus to split the module registration
> >> code
> >> >> in the module.main.cpp file?
> >> >>
> >> >> I have a huge number of implicit conversions, ptr_to_python and
> >> >> overloaded methods that need to be registered.  (5 per class in the
> >> >> library)  Right now I am registering those all in the main module
> >> >> registration section (by adding code through the module builder).
> >> The
> >> >> generated code works but it takes forever to compile this file.
> >> >
> >> > You can generate some cpp and header file and add the call to the
> >> main.
> >> >
> >> > mb.add_declaration_code( 'extern void register_my_xxx();' )
> >> > mb.add_registration_code( ' register_my_xxx();' )
> >>
> >> Although this would work I would rather generate all the code at the
> >> same time from the same generation script using py++.  As I understand
> >> this option I would have to generate the global scope methods
> >> separately.
> >
> >
> > No. See this document:
> > http://www.boost.org/libs/python/doc/tutorial/doc/html/python/techniques.html#python.reducing_compiling_time


> What is causing a performance problem currently is that I have to do the
> following code for every class (about 170 classes).
>
>  bp::register_ptr_to_python< osg::LineChunkPtr >();
>  bp::implicitly_convertible< osg::LineChunkRefPtr, osg::LineChunkPtr >();
>  bp::implicitly_convertible< osg::LineChunkPtr, osg::StateChunkPtr >();

You can attach these declarations to the class. They don't have to be registered
at "global" scope.

>  bp::def("RefPtr", &pyopensg::ToRefPtr<osg::LineChunkPtr>);
>  bp::def("FCPtr", &pyopensg::ToFcPtr<osg::LineChunkPtr>);

These one are the problematic ones. You can split them to different C++ files.
You can read and apply the technique described here:
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/techniques.html#python.reducing_compiling_time

> This works and compiles, but there is a performance hit during compiling
> because all of these registrations happen in the module.main.cpp file.
> I was hoping there may be some way to ask Py++ to split the
> registrations in this file across multiple .cpp files.  In a way
> basically extend the existing call hierarchy another level so that
> instead of just calling the class registration methods, the code in
> module.main.cpp could be output in a way that calls several other helper
> methods that in turn call the class registration methods and module
> scope registration methods like these.
>
> It doesn't seem that this is supported though so I will investigate some
> other options.  I had tried to register some of this code with the class
> code adders in the past and it did not work.  I will try this again and
> see if I have better luck now.

Allen, I understand your problem and I don't think I have a solution,
other I described here.
May be you can take a look on class_multiple_files.py file and to
propose a patch?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/