pygccxml-development

[pygccxml-development] Parsing a single header file for the whole project

[Benoît L. <ben...@gm...>]

Hi all,

There's something that's driving me mad, and I'm sure it's really
basic or simple to fix.
I have a big project to bind and parsing all the files takes a lot of
time, and finally hit the 8GB point at which I prefer to kill the
process.
In the "Best practices" page, it is suggested to "Create one header
file, which will include all project header files.". I've tried that,
both manually (creating the file on disk and then only parsing this
file), or automatically (using module_builder.create_text_fc to
construct a file based on a list of files), and in both ways py++
doesn't generate any binding.

I mean if I try to parse:
- A.h and B.h I will get the wrappers for the classes from these files,
- but if I try with a X.h which contains
#include "A.h"
#include "B.h"
nothing is wrapped by Py++.

I only get these files generated:
exposed_decl.pypp.txt
PROJECTNAME.main.cpp (this file is almost empty, ie. it doesn't
register any class)
named_tuple.py

Any idea, clue?

Thanks for your help.

Re: [pygccxml-development] Parsing a single header file for the whole project

[Benoît L. <ben...@gm...>]

I've just realized this is the same topic as another thread, which
should help me fix my problem.
Don't know how I could miss it the first time....
So, you can ignore this thread.

Le 1 avril 2010 14:38, Benoît Leveau <ben...@gm...> a écrit :
> Hi all,
>
> There's something that's driving me mad, and I'm sure it's really
> basic or simple to fix.
> I have a big project to bind and parsing all the files takes a lot of
> time, and finally hit the 8GB point at which I prefer to kill the
> process.
> In the "Best practices" page, it is suggested to "Create one header
> file, which will include all project header files.". I've tried that,
> both manually (creating the file on disk and then only parsing this
> file), or automatically (using module_builder.create_text_fc to
> construct a file based on a list of files), and in both ways py++
> doesn't generate any binding.
>
> I mean if I try to parse:
> - A.h and B.h I will get the wrappers for the classes from these files,
> - but if I try with a X.h which contains
> #include "A.h"
> #include "B.h"
> nothing is wrapped by Py++.
>
> I only get these files generated:
> exposed_decl.pypp.txt
> PROJECTNAME.main.cpp (this file is almost empty, ie. it doesn't
> register any class)
> named_tuple.py
>
> Any idea, clue?
>
> Thanks for your help.
>

Re: [pygccxml-development] Parsing a single header file for the whole project

[Roman Y. <rom...@gm...>]

2010/4/1 Benoît Leveau <ben...@gm...>:
> Hi all,

Good evening.

> There's something that's driving me mad, and I'm sure it's really
> basic or simple to fix.
> I have a big project to bind and parsing all the files takes a lot of
> time, and finally hit the 8GB point at which I prefer to kill the
> process.

Take a look on "pygccxml.parser.project_reader.create_cached_source_fc(header,
cached_source_file)" function (
http://language-binding.net/pygccxml/apidocs/parser.html )

It will allow you to save gccxml compilation time.

If you can send me( privatly ) gccxml generated file, I can try to
improve py++/pygccxml performance.

> In the "Best practices" page, it is suggested to "Create one header
> file, which will include all project header files."

pygccxml is able to merge results from different gccxml invocation,
but from my experience it will take more time and memory.

Did you include all your projects in that file or a single one? Ogre
3D engine is pretty big project, but even it doesn't consumes so much
memory.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Parsing a single header file for the whole project

[Benoît L. <ben...@gm...>]

What I ended up doing is:
creating a temporary file with a list of #include "header_path" then
constructing the module_builder_t with this file.

I first tried with create_text_fc but if you do that, in all the cpp
files generated by py++ , you won't get any include, so nothing will
compile.
The only weird thing now is that in all the cpp files, they all
include this temp file, so I need to keep this temp file for building.
I didn't know this create_cached_source... I'll try that to see if
that solves my problem.

I remember I tried the cache=cache_file_path option to
module_builder_t constructor, but it was not helping at all.
Each time the cache file was read, then recreated, and the generation
time was almost the same as without any cache.

Le 1 avril 2010 17:27, Roman Yakovenko <rom...@gm...> a écrit :
> 2010/4/1 Benoît Leveau <ben...@gm...>:
>> Hi all,
>
> Good evening.
>
>> There's something that's driving me mad, and I'm sure it's really
>> basic or simple to fix.
>> I have a big project to bind and parsing all the files takes a lot of
>> time, and finally hit the 8GB point at which I prefer to kill the
>> process.
>
> Take a look on "pygccxml.parser.project_reader.create_cached_source_fc(header,
> cached_source_file)" function (
> http://language-binding.net/pygccxml/apidocs/parser.html )
>
> It will allow you to save gccxml compilation time.
>
> If you can send me( privatly ) gccxml generated file, I can try to
> improve py++/pygccxml performance.
>
>> In the "Best practices" page, it is suggested to "Create one header
>> file, which will include all project header files."
>
> pygccxml is able to merge results from different gccxml invocation,
> but from my experience it will take more time and memory.
>
> Did you include all your projects in that file or a single one? Ogre
> 3D engine is pretty big project, but even it doesn't consumes so much
> memory.
>
> --
> Roman Yakovenko
> C++ Python language binding
> http://www.language-binding.net/
>

Re: [pygccxml-development] Parsing a single header file for the whole project

[Roman Y. <rom...@gm...>]

2010/4/1 Benoît Leveau <ben...@gm...>:
> What I ended up doing is:
> creating a temporary file with a list of #include "header_path" then
> constructing the module_builder_t with this file.
>
> I first tried with create_text_fc but if you do that, in all the cpp
> files generated by py++ , you won't get any include, so nothing will
> compile.

create_text_fc takes as argument valid C++ code. It is very useful for
small tests:

mb = module_builder.module_builder_t(  [
module_builder.create_text_fc( 'namespace enums{ enum color{ red = 1};
}' )]
            , gccxml_path=autoconfig.gccxml.executable,
compiler=pygccxml.utils.native_compiler.get_gccxml_compiler() )

mb.namespace( name='::enums' ).include()

> The only weird thing now is that in all the cpp files, they all
> include this temp file, so I need to keep this temp file for building.

I believe you will end making that file "permanent". You will add
there few work around and template instantiations.

> I didn't know this create_cached_source... I'll try that to see if
> that solves my problem.

It will save you compilation of source files with gccxml, py++ will
still have to parse the generated xml file.

> I remember I tried the cache=cache_file_path option to
> module_builder_t constructor, but it was not helping at all.
> Each time the cache file was read, then recreated, and the generation
> time was almost the same as without any cache.

I guess, it was recreated because you generated temp file each time (
may be I wrong ).

>From the benchmarks I did, "pickling" declarations tree was much
slower then recreating it from the existing xml file.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/