pygccxml-development

Re: [pygccxml-development] Function adaptors

[Roman Y. <rom...@gm...>]

On Wed, Aug 5, 2009 at 8:46 PM, Roman
Yakovenko<rom...@gm...> wrote:
> Hello Berserker
>
> I am starting to work on "function adaptors" feature and the following
> is what I think to implement:
>
> Declarations, which will support this feature:
> * member function which are: public, non virtual, without transformation,
> * free functions.
>
> The user will specify fully qualified adaptor name, and Py++ will take
> it and generate the following code:
>
> <addaptor name>( function reference )
>
> Is it enough? Will it suite your needs?

I commited the code and testers to SVN:
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml?view=rev&revision=1741

pyplusplus_dev/unittests/function_adaptor_tester.py files contains
usage example.

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Function adaptors

[Roman Y. <rom...@gm...>]

Hello Berserker

I am starting to work on "function adaptors" feature and the following
is what I think to implement:

Declarations, which will support this feature:
* member function which are: public, non virtual, without transformation,
* free functions.

The user will specify fully qualified adaptor name, and Py++ will take
it and generate the following code:

<addaptor name>( function reference )

Is it enough? Will it suite your needs?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Help needed with embedding

[Berserker <ber...@ho...>]

> let me think about it and I will back to you soon, okey?

Ok, thanks again!

_________________________________________________________________
Porta Messenger in Vacanza! Scaricalo sul cellulare!
http://new.windowslivemobile.msn.com/IT-IT/windows-live-messenger/default.aspx

Re: [pygccxml-development] Help needed with embedding

[Roman Y. <rom...@gm...>]

On Tue, Aug 4, 2009 at 2:02 PM, Berserker<ber...@ho...> wrote:
> I don't understand why do you think that this question should be posted in
> the Boost.Python mailing list.

Sorry, it seems that I need few more hours to sleep

> "allow_threads" is just the code extracted from
> http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup
> ,
> maybe a property like "adaptor" or "visitor" for each Py++ function would do
> the work, but I'm not so skilled with the Py++ API and probably there are
> other better solutions, btw something like:
>
>> tm = foo_class.mem_fun( 'test_method' )
>> tm.adaptor = "my_adaptor"
>
> and then Py++ will create:
>
>> boost::python::class_< Foo >("Foo")
>>    .def("test_method_1", my_adaptor(&Foo::test_method_1))
>
> Another solution (probably more "clean") could be a list of Templates for
> each boost::python API (boost::python::class_::def,
> boost::python::class_::add_property ecc...) so that we can customize them,
> what about?

Let me think about it. I will back to you in a day or two. I need to
check how it could be integrated with Py++.


>> Yes:
>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/function_transformers/templates.py?revision=1467&view=markup
>>
>> The short version: if you applied a transformation, than everything
>> should be done in it.
>>
>> The long version: you can modify the templates so it will release the
>> GIL. You can do it without changing Py++ code.
>
> Thanks, that was useful! Btw I have to change the "whole template" and pay
> attention to keep it aligned in the next releases of Py++ (that's ok for the
> moment),

More or less. You can change it value without touching Py++ code.
For example:

import pyplusplus import function_transformation as ft
from ft import templates as ft_tmpl

ft_tmpl.<class name>.<template> = NewTemplate

You will only have to preserve the keywords and add your keywords/code.

> what about adding $pre_call and $post_call params as in the
> "default" Template?

As above: let me think about it and I will back to you soon, okey?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Help needed with embedding

[Berserker <ber...@ho...>]

> Did you implement "allow_threads"? If yes, can you publish it and some
> usage example?
> If not, I guess you thought about it, right ? Do you mind to share
> your thoughts?
>
> This is an interesting question, and may be you should take it to
> Boost.Python mailing list. Whatever solution the list will come with,
> it will be possible to integrate it with Py++.

I don't understand why do you think that this question should be posted in the Boost.Python mailing list.
"allow_threads" is just the code extracted from http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup ,
maybe a property like "adaptor" or "visitor" for each Py++ function would do the work, but I'm not so skilled with the Py++ API and probably there are other better solutions, btw something like: 

> tm = foo_class.mem_fun( 'test_method' )
> tm.adaptor = "my_adaptor"

and then Py++ will create:

> boost::python::class_< Foo >("Foo")
>    .def("test_method_1", my_adaptor(&Foo::test_method_1))

Another solution (probably more "clean") could be a list of Templates for each boost::python API (boost::python::class_::def, boost::python::class_::add_property ecc...) so that we can customize them, what about?

> Yes: http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/function_transformers/templates.py?revision=1467&view=markup
>
> The short version: if you applied a transformation, than everything
> should be done in it.
>
> The long version: you can modify the templates so it will release the
> GIL. You can do it without changing Py++ code.

Thanks, that was useful! Btw I have to change the "whole template" and pay attention to keep it aligned in the next releases of Py++ (that's ok for the moment), what about adding $pre_call and $post_call params as in the "default" Template?

> > Here it is the link http://osiris.kodeware.net/ if you are interested :)
>
> :-). Very cool project.

Thanks :)

_________________________________________________________________
Messenger è su Hotmail. Scopri le novità.
http://www.messenger.it/accediWebMessengerHotmail.aspx

Re: [pygccxml-development] Help needed with embedding

[Roman Y. <rom...@gm...>]

On Tue, Aug 4, 2009 at 12:27 PM, Berserker<ber...@ho...> wrote:
> Thanks for the quick reply Roman
>
>> This is a quick solution. By the way Py++ defines gil_guard_t class(
>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/code_repository/gil_guard.py?revision=1428&view=markup
>> )
>
> Thanks for the link but gil_guard_t acquires and releases the GIL allowing
> to operate on the CPython API, instead we need to release the lock
> (PyEval_SaveThread) before the c++ function invocation and to acquire it
> again (PyEval_RestoreThread) after allowing multiple interpreters to run
> concurrently during the c++ code execution where no CPython API will be
> touched

Sorry for misunderstanding.

>> I guess it is possible to create custom call policy, so the code will look
>> like:
>
>> boost::python::class_< Foo >("Foo")
>> .def("test_method_1", &Foo::test_method_1,
>> pythread_safe<return_value_policy< ... > >() );
>
> I have tried this solution with a custom policy that calls PyEval_SaveThread
> in the "precall" function and PyEval_RestoreThread in the "postcall"
> function (according to
> http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/default_call_policies.html
> ) but unfortunately after the precall's policy boost::python works on the
> CPython API before of the handler invocation (converting arguments for
> example), instead with a custom visitor (like that in libtorrent) we can
> surround the c++ code invocation with PyEval_SaveThread/PyEval_RestoreThread
> exactly before/after its execution where, as I said before, no CPython API
> will be touched for sure.
> I still think that something like:
>
>> boost::python::class_< Foo >("Foo")
>>     .def("test_method_1", allow_threads(&Foo::test_method_1))>;
>
> is the best solution, any hope to support this?

Did you implement "allow_threads"? If yes, can you publish it and some
usage example?
If not, I guess you thought about it, right ? Do you mind to share
your thoughts?

This is an interesting question, and may be you should take it to
Boost.Python mailing list. Whatever solution the list will come with,
it will be possible to integrate it with Py++.

>> tm.add_override_precall_code( ... )
>> tm.add_default_precall_code( ... )
>
> add_override_precall_code is exactly what we need but "strangely" no code is
> added when the allow_threading_transformer_t is added to the function (we
> still need the transformer as I mentioned above until the "visitor" patch),
> my code:
>
>> tm.add_transformation(allow_threading_transformer_creator())
>> tm.add_override_precall_code(gil_guard_code)
>
> If I comment the "add_transformation" line, the override_precall_code is
> correctly added to the wrapper. Any advice?

Yes: http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/function_transformers/templates.py?revision=1467&view=markup

The short version: if you applied a transformation, than everything
should be done in it.

The long version: you can modify the templates so it will release the
GIL. You can do it without changing Py++ code.

>> Is this an open source project?
>
> It will be :)
> Here it is the link http://osiris.kodeware.net/ if you are interested :)

:-). Very cool project.

> Thanks again for your help

You are welcome.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Help needed with embedding

[Berserker <ber...@ho...>]

Thanks for the quick reply Roman

> This is a quick solution. By the way Py++ defines gil_guard_t class( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/code_repository/gil_guard.py?revision=1428&view=markup )

Thanks for the link but gil_guard_t acquires and releases the GIL allowing to operate on the CPython API, instead we need to release the lock (PyEval_SaveThread) before the c++ function invocation and to acquire it again (PyEval_RestoreThread) after allowing multiple interpreters to run concurrently during the c++ code execution where no CPython API will be touched

> I guess it is possible to create custom call policy, so the code will look like:

> boost::python::class_< Foo >("Foo")
> .def("test_method_1", &Foo::test_method_1, pythread_safe<return_value_policy< ... > >() );

I have tried this solution with a custom policy that calls PyEval_SaveThread in the "precall" function and PyEval_RestoreThread in the "postcall" function (according to http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/default_call_policies.html ) but unfortunately after the precall's policy boost::python works on the CPython API before of the handler invocation (converting arguments for example), instead with a custom visitor (like that in libtorrent) we can surround the c++ code invocation with PyEval_SaveThread/PyEval_RestoreThread exactly before/after its execution where, as I said before, no CPython API will be touched for sure.
I still think that something like:

> boost::python::class_< Foo >("Foo")
>     .def("test_method_1", allow_threads(&Foo::test_method_1))>; 

is the best solution, any hope to support this?

> tm.add_override_precall_code( ... )
> tm.add_default_precall_code( ... )

add_override_precall_code is exactly what we need but "strangely" no code is added when the allow_threading_transformer_t is added to the function (we still need the transformer as I mentioned above until the "visitor" patch), my code:

> tm.add_transformation(allow_threading_transformer_creator()) 
> tm.add_override_precall_code(gil_guard_code)

If I comment the "add_transformation" line, the override_precall_code is correctly added to the wrapper. Any advice?

> Is this an open source project?

It will be :)
Here it is the link http://osiris.kodeware.net/ if you are interested :)

Thanks again for your help


_________________________________________________________________
Scarica i nuovi gadget per personalizzare Messenger!
http://www.messenger.it/home_gadget.aspx

Re: [pygccxml-development] Help needed with embedding

[Roman Y. <rom...@gm...>]

On Mon, Aug 3, 2009 at 6:02 PM, Berserker<ber...@ho...> wrote:
> Hi, we are trying the Py++ library for embedding Python in our project.
> We need to deal with GIL since we have to run multiple Python script
> "concurrently".
>
> The first problem we have encountered is that Py++ doesn't seems to support
> the ability to write a "member function adaptor" (that releases and aqcuires
> the GIL around the function call): libtorrent has a nice example of what we
> need here
> http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup
> At the moment we are "emulating" this feature with a custom transformer,
> here is the code:
>
> class allow_threading_transformer_t(transformer.transformer_t):
>    def __init__(self, function):
>       transformer.transformer_t.__init__(self, function)
>    def __configure_sealed(self, controller):
>       controller.add_pre_call_code("allow_threading_guard guard;")
>
>    def configure_mem_fun(self, controller):
>       self.__configure_sealed(controller)
>    def configure_free_fun(self, controller):
>       self.__configure_sealed(controller)
>    def configure_virtual_mem_fun(self, controller):
>       self.__configure_sealed(controller.default_controller)
>
> The only problem with this solution is that it creates too much code since
> it requires a static function for each wrapped methods, for example:
> Given the "Foo" class:
>
> class Foo
> {
> public:
>    void test_method_1();
>    void test_method_2();
> };
>
> Py++ creates:
>
> struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {
>
>    static void test_method_1( ::Foo & inst) {
>       allow_threading_guard guard;
>       inst.test_method_1();
>    }
>
>     static void test_method_2( ::Foo & inst) {
>         allow_threading_guard guard;
>         inst.test_method_2();
>     }
>
> }
>
> ::boost::python::class_< Foo_wrapper >( "Foo" )
>     .def("test_method_1", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_1
> ) )
>     .def("test_method_2", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_2
> ) )
>     ;
>
> insted of (based on the libtorrent code) only:
>
> boost::python::class_< Foo >("Foo")
>     .def("test_method_1", allow_threads(&Foo::test_method_1))
>     .def("test_method_2", allow_threads(&Foo::test_method_2))
>     ;
>
> Is there any hope for a patch to support "member function adaptors"?

This is a quick solution. By the way Py++ defines gil_guard_t class(
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/code_repository/gil_guard.py?revision=1428&view=markup
)

I guess it is possible to create custom call policy, so the code will look like:

> boost::python::class_< Foo >("Foo")
>     .def("test_method_1", &Foo::test_method_1, pythread_safe< return_value_policy< ... > >() );

Is this an open source project?

> The second problem is related to virtual member functions, let's consider a
> "test_method" in the Foo class
>
> class Foo
> {
> public:
>     virtual void test_method();
> };
>
> Py++ now creates:
>
> struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {
>
>     virtual void test_method(  ) {
>         namespace bpl = boost::python;
>         if( bpl::override func_test_method = this->get_override(
> "test_method" ) ){
>             bpl::object py_result = bpl::call<bpl::object>(
> func_test_method.ptr() );
>         }
>         else{
>             ::Foo::test_method(  );
>         }
>     }
>
>     static void default_test_method( ::Foo & inst ){
>         allow_threading_guard guard;
>         if( dynamic_cast< Foo_wrapper * >( boost::addressof( inst ) ) ){
>             inst.::Foo::test_method();
>         }
>         else{
>             inst.test_method();
>         }
>     }
>
> }
>
> ::boost::python::class_< Foo_wrapper >( "Foo", ::boost::python::init< >()
> )
>     .def( "test_method", (void (*)( ::Foo & ))(
> &Foo_wrapper::default_test_method ) )
>     ;
>
> default_test_method works fine as in the first example but here we need to
> restore the right PyThreadState before calling any CPython function in the
> overridden "test_method" (this->get_override and bpl::call in this case).
> At the moment we have no solution for this problem but the idea is to create
> a custom base class for the Foo_wrapper class instead of
> boost::python::wrapper (where we can save the PyThreadState and restore that
> state before any call to the CPython API), here is the code:

Well this problem is actually solved:

mb = module_builder_t( ... )
tm = mb.mem_fun( 'test_method' )

gil_guard_code = """
    pyplusplus::threading::gil_guard_t guard;
"""

tm.add_override_precall_code( gil_guard_code )
tm.add_default_precall_code( gil_guard_code )

tm.include_files.append( code_repository.gil_guard.file_name )


> Py++ code:
>
> def override_create_identifier(creator, full_name ):
>     if full_name == "::boost::python::wrapper":
>         return "allow_threads_wrapper"
>
>     return full_name
>
> pyplusplus.code_creators.algorithm.create_identifier =
> override_create_identifier
>
> C++:
>
> template <typename T>
> class allow_threads_wrapper : public boost::python::wrapper<T>
> {
> public:
>     allow_threads_wrapper() : m_state(PyThreadState_Get()) { }
>     PyThreadState *m_state;
> };
>
> The last step should be the ability to add custom code before the
> "get_override" call, something like:
>
> struct Foo_wrapper : ::Foo, allow_threads_wrapper< ::Foo > {
>
>     virtual void test_method(  ) {
>         allow_threading_swap swap(m_state); // allow_threading_swap is a
> class that swap/restore the PyThreadState
>
>         namespace bpl = boost::python;
>         if( bpl::override func_test_method = this->get_override(
> "test_method" ) ){
>             bpl::object py_result = bpl::call<bpl::object>(
> func_test_method.ptr() );
>         }
>         else{
>             ::Foo::test_method(  );
>         }
>     }
>
> }
>
> Unfortunately the template of virtual_mem_fun in Py++ doesn't support the
> ability to add custom code at the beginning of the function, is it possible
> to make a small patch for this problem?

Do you still think Py++ needs the patch?

> Sorry for the long post, plz help

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Help needed with embedding

[Berserker <ber...@ho...>]

Hi, we are trying the Py++ library for embedding Python in our project.
We need to deal with GIL since we have to run multiple Python script "concurrently".

The first problem we have encountered is that Py++ doesn't seems to support the ability to write a "member function adaptor" (that releases and aqcuires the GIL around the function call): libtorrent has a nice example of what we need here http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/trunk/bindings/python/src/gil.hpp?revision=2460&view=markup
At the moment we are "emulating" this feature with a custom transformer, here is the code:

class allow_threading_transformer_t(transformer.transformer_t):
   def __init__(self, function):
      transformer.transformer_t.__init__(self, function)        
   def __configure_sealed(self, controller):
      controller.add_pre_call_code("allow_threading_guard guard;")                
   def configure_mem_fun(self, controller):
      self.__configure_sealed(controller)
   def configure_free_fun(self, controller):
      self.__configure_sealed(controller)
   def configure_virtual_mem_fun(self, controller):
      self.__configure_sealed(controller.default_controller)
      
The only problem with this solution is that it creates too much code since it requires a static function for each wrapped methods, for example:
Given the "Foo" class:

class Foo
{
public:
   void test_method_1();
   void test_method_2();
};

Py++ creates:

struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {
    
   static void test_method_1( ::Foo & inst) {
      allow_threading_guard guard;
      inst.test_method_1();
   }

    static void test_method_2( ::Foo & inst) {
        allow_threading_guard guard;
        inst.test_method_2();
    }
    
}

::boost::python::class_< Foo_wrapper >( "Foo" )    
    .def("test_method_1", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_1 ) )    
    .def("test_method_2", (void (*)( ::Foo & ))( &Foo_wrapper::test_method_2 ) )    
    ;
    
insted of (based on the libtorrent code) only:

boost::python::class_< Foo >("Foo")
    .def("test_method_1", allow_threads(&Foo::test_method_1))
    .def("test_method_2", allow_threads(&Foo::test_method_2))
    ;    

Is there any hope for a patch to support "member function adaptors"?

The second problem is related to virtual member functions, let's consider a "test_method" in the Foo class

class Foo
{
public:
    virtual void test_method();   
};

Py++ now creates:

struct Foo_wrapper : ::Foo, ::boost::python::wrapper< ::Foo > {
    
    virtual void test_method(  ) {
        namespace bpl = boost::python;
        if( bpl::override func_test_method = this->get_override( "test_method" ) ){
            bpl::object py_result = bpl::call<bpl::object>( func_test_method.ptr() );
        }
        else{
            ::Foo::test_method(  );
        }
    }

    static void default_test_method( ::Foo & inst ){
        allow_threading_guard guard;
        if( dynamic_cast< Foo_wrapper * >( boost::addressof( inst ) ) ){
            inst.::Foo::test_method();
        }
        else{
            inst.test_method();
        }
    }

}

::boost::python::class_< Foo_wrapper >( "Foo", ::boost::python::init< >() )    
    .def( "test_method", (void (*)( ::Foo & ))( &Foo_wrapper::default_test_method ) )
    ;
    
default_test_method works fine as in the first example but here we need to restore the right PyThreadState before calling any CPython function in the overridden "test_method" (this->get_override and bpl::call in this case).
At the moment we have no solution for this problem but the idea is to create a custom base class for the Foo_wrapper class instead of boost::python::wrapper (where we can save the PyThreadState and restore that state before any call to the CPython API), here is the code:

Py++ code:

def override_create_identifier(creator, full_name ):
    if full_name == "::boost::python::wrapper":
        return "allow_threads_wrapper"
                        
    return full_name
    
pyplusplus.code_creators.algorithm.create_identifier = override_create_identifier

C++:

template <typename T>
class allow_threads_wrapper : public boost::python::wrapper<T>
{
public:
    allow_threads_wrapper() : m_state(PyThreadState_Get()) { }
    PyThreadState *m_state;
};

The last step should be the ability to add custom code before the "get_override" call, something like:

struct Foo_wrapper : ::Foo, allow_threads_wrapper< ::Foo > {
    
    virtual void test_method(  ) {
        allow_threading_swap swap(m_state); // allow_threading_swap is a class that swap/restore the PyThreadState

        namespace bpl = boost::python;
        if( bpl::override func_test_method = this->get_override( "test_method" ) ){
            bpl::object py_result = bpl::call<bpl::object>( func_test_method.ptr() );
        }
        else{
            ::Foo::test_method(  );
        }
    }
    
}

Unfortunately the template of virtual_mem_fun in Py++ doesn't support the ability to add custom code at the beginning of the function, is it possible to make a small patch for this problem?

Sorry for the long post, plz help
Thanks


_________________________________________________________________
Porta Messenger in Vacanza! Scaricalo sul cellulare!
http://new.windowslivemobile.msn.com/IT-IT/windows-live-messenger/default.aspx

[pygccxml-development] test

[<uol...@ti...>]

simple test


Presentaci un amico! Invita i tuoi amici a scoprire i servizi Tiscali: per ogni amico che attiverà un servizio Tiscali, potrai avere 20 euro di sconto e lo stesso sconto di 20 euro verrà poi offerto al tuo amico. Scopri come! http://abbonati.tiscali.it/promo/presentaciunamico/

Re: [pygccxml-development] Help with single header setup

[Roman Y. <rom...@gm...>]

On Sat, Aug 1, 2009 at 12:47 AM, Brendan
Duncan<Bre...@di...> wrote:
> Thanks Roman, this helps me out.
>
> Interesting. I would like to investigate the problem. Is this an open
> source project?
>
>
> Unfortunately the project is proprietary in-house software.  Using the
> single header rather than passing all of the headers to module_builder seems
> to solve the memory issue.

Okey. Now I understand the reasons.

> Now I just have to solve the multi-module dependency errors that have
> started to show up, although I didn't get them when I was using the method
> of passing all headers individually.  I'll be sure to ask more questions if
> I hit another roadblock.

Did you see the following document:
http://language-binding.net/pyplusplus/documentation/multi_module_development.html

That functionality was tested by many projects and it worked ( for them :-) ).

> The Py++ project is quite nice, by the way.  Thanks for providing it, and
> offering help.

Thank you!


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Help with single header setup

[Brendan D. <Bre...@di...>]

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Thanks Roman, this helps me out.<br>
<br>
<blockquote
 cite="mid:746...@ma..."
 type="cite">
  <pre wrap=""><!---->
Interesting. I would like to investigate the problem. Is this an open
source project?
  </pre>
</blockquote>
Unfortunately the project is proprietary in-house software.&nbsp; Using the
single header rather than passing all of the headers to module_builder
seems to solve the memory issue.&nbsp; It was the fact that classes are
included by default when you pass headers in individually, and not
included by default when included into a single header, that was
throwing me off.<br>
<br>
Now I just have to solve the multi-module dependency errors that have
started to show up, although I didn't get them when I was using the
method of passing all headers individually.&nbsp; I'll be sure to ask more
questions if I hit another roadblock.&nbsp; <br>
<br>
The Py++ project is quite nice, by the way.&nbsp; Thanks for providing it,
and offering help.<br>
<br>
~Brendan<br>
<br>
<blockquote
 cite="mid:746...@ma..."
 type="cite">
  <pre wrap="">
  </pre>
  <blockquote type="cite">
    <pre wrap="">I read the section of documentation that said putting all of the headers
into a single header is the way to go. &nbsp;This is where I'm stuck, and I
don't have a lot of time to try and figure it out.

Here's a greatly simplified version of what I'm currently doing. &nbsp;Can
anyone tell me what I need to be doing instead?

Foo.h:
class Foo {
public:
&nbsp; &nbsp;Foo(void) { }
&nbsp; &nbsp;void test(void) { }
};

headers.h:
#include "Foo.h"

generate.py:
from pyplusplus import module_builder
GCCXML = '/path/to/gccxml'
mb = module_builder.module_builder_t(files=['headers.h'],
gccxml_path=GCCXML)
mb.build_code_creator(module_name='_test')
targets = mb.split_module('build')

When I run generate.py, it produces _test.main.cpp:
#include "boost/python.hpp"
#include "headers.h"
namespace bp = boost::python;
BOOST_PYTHON_MODULE(_test){
}


If instead of headers.h I pass Foo.h to module_builder_t, then it builds
fine, except this is the case where in my large project it consumes far
too much memory and takes too long. &nbsp;I tried having the pygccxml parser
generate an xml for headers.h and passing that to module_builder, but
that didn't work either. &nbsp;Looking at the xml file I do see the Foo
declarations though, so it's just module_builder that's ignoring them.

Any ideas would be greatly appreciated. &nbsp;I'm using the SVN version, on
Linux (if that matters).
    </pre>
  </blockquote>
  <pre wrap=""><!---->
The following documents should help you:
* <a class="moz-txt-link-freetext" href="http://language-binding.net/pyplusplus/documentation/tutorials/module_builder/module_builder.html">http://language-binding.net/pyplusplus/documentation/tutorials/module_builder/module_builder.html</a>
* <a class="moz-txt-link-freetext" href="http://language-binding.net/pygccxml/query_interface.html">http://language-binding.net/pygccxml/query_interface.html</a>

In general you can write:

mb.class_( 'Foo' ).include()

  </pre>
</blockquote>
<br>
</body>
</html>

Re: [pygccxml-development] Help with single header setup

[Roman Y. <rom...@gm...>]

On Thu, Jul 30, 2009 at 6:53 PM, Brendan
Duncan<Bre...@di...> wrote:
> I've been testing Py++ lately as an alternative to swig.  Some of the
> libraries I'm trying to wrap cause Py++/PyGCCXML to consume well over
> 6GB of RAM, and take quite a while to process (considerably longer than
> swig).

Interesting. I would like to investigate the problem. Is this an open
source project?

> I read the section of documentation that said putting all of the headers
> into a single header is the way to go.  This is where I'm stuck, and I
> don't have a lot of time to try and figure it out.
>
> Here's a greatly simplified version of what I'm currently doing.  Can
> anyone tell me what I need to be doing instead?
>
> Foo.h:
> class Foo {
> public:
>    Foo(void) { }
>    void test(void) { }
> };
>
> headers.h:
> #include "Foo.h"
>
> generate.py:
> from pyplusplus import module_builder
> GCCXML = '/path/to/gccxml'
> mb = module_builder.module_builder_t(files=['headers.h'],
> gccxml_path=GCCXML)
> mb.build_code_creator(module_name='_test')
> targets = mb.split_module('build')
>
> When I run generate.py, it produces _test.main.cpp:
> #include "boost/python.hpp"
> #include "headers.h"
> namespace bp = boost::python;
> BOOST_PYTHON_MODULE(_test){
> }
>
>
> If instead of headers.h I pass Foo.h to module_builder_t, then it builds
> fine, except this is the case where in my large project it consumes far
> too much memory and takes too long.  I tried having the pygccxml parser
> generate an xml for headers.h and passing that to module_builder, but
> that didn't work either.  Looking at the xml file I do see the Foo
> declarations though, so it's just module_builder that's ignoring them.
>
> Any ideas would be greatly appreciated.  I'm using the SVN version, on
> Linux (if that matters).

The following documents should help you:
* http://language-binding.net/pyplusplus/documentation/tutorials/module_builder/module_builder.html
* http://language-binding.net/pygccxml/query_interface.html

In general you can write:

mb.class_( 'Foo' ).include()

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Help with single header setup

[Brendan D. <Bre...@di...>]

I've been testing Py++ lately as an alternative to swig.  Some of the 
libraries I'm trying to wrap cause Py++/PyGCCXML to consume well over 
6GB of RAM, and take quite a while to process (considerably longer than 
swig). 

I read the section of documentation that said putting all of the headers 
into a single header is the way to go.  This is where I'm stuck, and I 
don't have a lot of time to try and figure it out.

Here's a greatly simplified version of what I'm currently doing.  Can 
anyone tell me what I need to be doing instead?

Foo.h:
class Foo {
public:
    Foo(void) { }
    void test(void) { }
};

headers.h:
#include "Foo.h"

generate.py:
from pyplusplus import module_builder
GCCXML = '/path/to/gccxml'
mb = module_builder.module_builder_t(files=['headers.h'], 
gccxml_path=GCCXML)
mb.build_code_creator(module_name='_test')
targets = mb.split_module('build')

When I run generate.py, it produces _test.main.cpp:
#include "boost/python.hpp"
#include "headers.h"
namespace bp = boost::python;
BOOST_PYTHON_MODULE(_test){
}


If instead of headers.h I pass Foo.h to module_builder_t, then it builds 
fine, except this is the case where in my large project it consumes far 
too much memory and takes too long.  I tried having the pygccxml parser 
generate an xml for headers.h and passing that to module_builder, but 
that didn't work either.  Looking at the xml file I do see the Foo 
declarations though, so it's just module_builder that's ignoring them.

Any ideas would be greatly appreciated.  I'm using the SVN version, on 
Linux (if that matters).

Thanks,
Brendan

Re: [pygccxml-development] Declarations order

[Roman Y. <rom...@gm...>]

On Thu, Jun 4, 2009 at 1:26 PM, Daniele Pianu<mu...@gm...> wrote:
> Hi to all,
> In my wrappers-generator program, I'm using the pygccxml declarations
> hierarchy to retrieve all declarations from a C library. Now I've some
> problems with the order of the declarations obtained from the
> header-parsing process. Supposing those two structures in the global
> namespace:
>
> typedef struct Foo
> {
>  ....
> } Foo;
>
> typedef struct Bar
> {
>  Foo *foo;
>  ......
> } Bar;
>
> I need to wrap the Foo structure first than Bar because the Foo type
> is used as a Bar's field type. The problem is that, when I iter among
> the pygccxml definitions (by example, among the ones returned by the
> mdecl_wrapper_t instance obtained by the "global_namespace.classes()"
> call) the definitions are often not sorted by declaration-order. The
> question is: do I need to order the declarations manually or there's
> some other easier way?

You will have to sort the declarations.

> Sorting the declarations of a small library
> it's not a big problem, but when I've to wrap libraries with thousand
> of declarations the situation changes...

Not really, at least this is not the functionality I saw in profilers.

I needed the same functionality for my code generators ( Py++ and
ctypes ), so I wrote few classes that sort classes:
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/creators_factory/sort_algorithms.py?revision=1663&view=markup

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Declarations order

[Daniele P. <mu...@gm...>]

Hi to all,
In my wrappers-generator program, I'm using the pygccxml declarations
hierarchy to retrieve all declarations from a C library. Now I've some
problems with the order of the declarations obtained from the
header-parsing process. Supposing those two structures in the global
namespace:

typedef struct Foo
{
  ....
} Foo;

typedef struct Bar
{
  Foo *foo;
  ......
} Bar;

I need to wrap the Foo structure first than Bar because the Foo type
is used as a Bar's field type. The problem is that, when I iter among
the pygccxml definitions (by example, among the ones returned by the
mdecl_wrapper_t instance obtained by the "global_namespace.classes()"
call) the definitions are often not sorted by declaration-order. The
question is: do I need to order the declarations manually or there's
some other easier way? Sorting the declarations of a small library
it's not a big problem, but when I've to wrap libraries with thousand
of declarations the situation changes...

Thanks for any advice,
Daniele

Re: [pygccxml-development] Obtain class definition from class declaration

[Roman Y. <rom...@gm...>]

On Thu, May 28, 2009 at 2:15 PM, Daniele Pianu <mu...@gm...> wrote:
> Hi all,
> I'm using the (great :D) pygccxml package to implement a wrappers
> generator for some C libraries used in my company. Given a Foo
> library/component, the pygccxml parser is launched on the Foo.h header
> (not on other headers used by the same library, ex. Foo_bar.h). Now a
> particular case has arisen:
> if a Bar structure in Foo.h is declared (but defined in another header
> NOT included by the Foo.h), I'm able to obtain the Bar declaration but
> not the Bar definition. I've checked the class_declaration_t type but
> I've not found any useful property-method. How can I solve my problem?
> Do I need to mandatory parse all header related to the Foo library to
> obtain all structure definitions or is there another way to obtain a
> structure definition from a struct declaration?

Yes, you will have to include file, that contains "Bar" definition.
The performance should not be an issue, if you will use "cache"
functionality: http://language-binding.net/pygccxml/apidocs/parser.html#pygccxml.parser.project_reader.create_cached_source_fc

> Thanks for your help and for the amazing work done with the pygccxml package,

Thank you.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Obtain class definition from class declaration

[Daniele P. <mu...@gm...>]

Hi all,
I'm using the (great :D) pygccxml package to implement a wrappers
generator for some C libraries used in my company. Given a Foo
library/component, the pygccxml parser is launched on the Foo.h header
(not on other headers used by the same library, ex. Foo_bar.h). Now a
particular case has arisen:
if a Bar structure in Foo.h is declared (but defined in another header
NOT included by the Foo.h), I'm able to obtain the Bar declaration but
not the Bar definition. I've checked the class_declaration_t type but
I've not found any useful property-method. How can I solve my problem?
Do I need to mandatory parse all header related to the Foo library to
obtain all structure definitions or is there another way to obtain a
structure definition from a struct declaration?

Thanks for your help and for the amazing work done with the pygccxml package,
Daniele

Re: [pygccxml-development] minor c++ transformation with pygccxml / py++

[Nicholas T. <nt...@nt...>]

On Tue, May 12, 2009 at 23:08, Roman Yakovenko <rom...@gm...>wrote:

> On Wed, May 13, 2009 at 4:55 AM, Nicholas Tung <nt...@nt...> wrote:
> > Hi all,
> >
> >     Is there an easy way to re-output c++ code, with minor modifications?
> > Py++ seems mostly focused on creating Boost wrappers.
>
>
> http://blog.mozilla.com/tglek/2009/01/29/semantic-rewriting-of-code-with-pork-a-bitter-recap/
>
> HTH
>
> --
> Roman Yakovenko
> C++ Python language binding
> http://www.language-binding.net/
>

Thanks very much, I'll give it a try. It seems to be exactly what I was
looking for!

Nicholas

Re: [pygccxml-development] minor c++ transformation with pygccxml / py++

[Roman Y. <rom...@gm...>]

On Wed, May 13, 2009 at 4:55 AM, Nicholas Tung <nt...@nt...> wrote:
> Hi all,
>
>     Is there an easy way to re-output c++ code, with minor modifications?
> Py++ seems mostly focused on creating Boost wrappers.

http://blog.mozilla.com/tglek/2009/01/29/semantic-rewriting-of-code-with-pork-a-bitter-recap/

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] minor c++ transformation with pygccxml / py++

[Nicholas T. <nt...@nt...>]

Sorry for the spam, I didn't know that gccxml didn't parse function bodies.

regards,
Nicholas

On Tue, May 12, 2009 at 18:55, Nicholas Tung <nt...@nt...> wrote:

> Hi all,
>
>     Is there an easy way to re-output c++ code, with minor modifications?
> Py++ seems mostly focused on creating Boost wrappers.
>
> Thanks so much,
> Nicholas
>

[pygccxml-development] minor c++ transformation with pygccxml / py++

[Nicholas T. <nt...@nt...>]

Hi all,

    Is there an easy way to re-output c++ code, with minor modifications?
Py++ seems mostly focused on creating Boost wrappers.

Thanks so much,
Nicholas

Re: [pygccxml-development] container_traits.key_type() missing in 0.9.5

[Gustavo C. <gjc...@gm...>]

2009/4/28 Roman Yakovenko <rom...@gm...>

> On Mon, Apr 27, 2009 at 8:01 PM, Gustavo Carneiro <gjc...@gm...>
> wrote:
> > Hi, I am trying to scan std::map container types.  I need to obtain the
> key
> > and element types separately, but pygccxml (0.9.5) keeps fighting me
> > (below).  Wasn't there supposed to be a key_type method/attribute in the
> > container_traits classes?
>
> I didn't need it, so it doesn't exist. I don't mind to add it for you.


OK, but it apparently is there in trunk, so no work needed for you :-)

I am wrapping std::map now, with pybindgen, that's why I need it.


>
> > Apparently key_type() method works well in SVN pygccxml.  Any idea when
> will
> > be next pygccxml release?
>
> I don't know. I hope in a month. I am pretty busy this days.
>
> In general I consider "pygccxml" as frozen, which means I will only
> update the documentation.


A month or two sounds reasonable.

I tried svn trunk, and everything worked perfectly for me.


>
>
> Did you note that I switched to Sphinx? It took some time, but the
> result worth it.


I agree, the docs look much nicer now.  Great work.

Thanks for the feedback.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

Re: [pygccxml-development] Py++ typedef handling

[Roman Y. <rom...@gm...>]

On Fri, Apr 24, 2009 at 2:03 AM, Patrick Hartling
<pat...@pr...> wrote:
>> I guess, it is possible to attach "pretty name" to a type and use it
>> for code generation, but this is a huge change.
>
> Is such an approach necessary, though? Since the type information identified
> by GCC-XML is the typedef, doesn't that mean that pygccxml or Py++ is
> "drilling down" to the un-aliased type in some cases?

Yes it does. And you know, may be you are right. May be it will be
enough to "fix" those places.

>>> What I see is that the typedef gets used in
>>> some places but not others. I haven't identified a pattern to this yet,
>>> but
>>> I am sure that there is one.
>>
>> Yes:
>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/decl_wrappers/decl_wrapper.py?revision=1664&view=markup
>>
>> Take a look on "_get_alias" method.
>
> I have used the alias property in some cases. Does it apply to what I
> posted, though? If so, then I am not seeing the full extent of what
> opportunities exist for operating on Py++ objects. For example, how can the
> alias property be used to alter the type casting done for keyword argument
> values?

I was thinking about using that alias, when I format function
arguments. For example:

#ifdef os64:
    typedef std::vector< int64 > numbers_t;
#else
    typedef std::vector< int32 > numbers_t;
#endif

void do_smth( numbers_t& );

I think, today, Py++ will generate do_smth with std::vector, which
contains int64 or int32. Finding out, that std::vector< int[32|64] >
class has alias and using it can improve the generated code.

Obviously, your patch produce the same result and it is much better
than my idea.

I need to check\verify my ideas before I raise them and may be to
sleep a little bit more :-)

>> It is very difficult. A year or two ago I tried to add something
>> similar to GCCXML ad got lost in GCC code. May be you will have more
>> luck.
>
> I cannot make any promises, and it depends on how my work hours are
> allocated. I would like to see this addressed, though, and maybe I can make
> a case for devoting time to it.

Good luck. GCC code is not the easiest code to read and understand.

>> Take a look on
>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/pyboost_dev/dev/date_time/generate_code.py?revision=1428&view=markup
>> file.
>> It contains code that does exactly this.
>>
>> Let me know if you need more help.
>
>
> That pointer is helpful. It set me on a course where I have already resolved
> several of our most frustrating issues.
>
> I have attached a small patch against the SVN trunk that makes a significant
> difference in generating more portable code--at least in our case. I have
> not yet run this change through the test suite (as I still have to learn how
> to run the test suite).

I committed the patch.

Thank you.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] container_traits.key_type() missing in 0.9.5

[Roman Y. <rom...@gm...>]

On Mon, Apr 27, 2009 at 8:01 PM, Gustavo Carneiro <gjc...@gm...> wrote:
> Hi, I am trying to scan std::map container types.  I need to obtain the key
> and element types separately, but pygccxml (0.9.5) keeps fighting me
> (below).  Wasn't there supposed to be a key_type method/attribute in the
> container_traits classes?

I didn't need it, so it doesn't exist. I don't mind to add it for you.

> Apparently key_type() method works well in SVN pygccxml.  Any idea when will
> be next pygccxml release?

I don't know. I hope in a month. I am pretty busy this days.

In general I consider "pygccxml" as frozen, which means I will only
update the documentation.

Did you note that I switched to Sphinx? It took some time, but the
result worth it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/