Re: [pygccxml-development] Another way to speed up the generation process...

[Allen B. <al...@vr...>]

Matthias Baas wrote:
> Allen Bierbaum wrote:
>>> What I did were two things: pruning the declaration tree and caching 
>>> query operations. I don't use the pruning function from my earlier 
>>> mail anymore because this pruning is only done after the header files 
>>> were already parsed and stored in the cache. So while the function 
>>> could speed up the decoration stage it could not speed up the parsing 
>>> stage (which took more than a minute on OSX using the regular cache). 
>>> So now I prune the declarations at an earlier stage, namely right 
>>> after the XML file was created by gccxml and before pygccxml reads it. 
>>> I wrote a little utility that directly operates on the XML file and 
>>> outputs a pruned XML file which I use as input for Py++ (I think I'll 
>>> put it into contrib eventually). This should now even be safer than 
>>> before because I also keep all dependencies even when they are not 
>>> inside any of the allowed headers. With the pruned tree alone I am 
>>> already at the above parsing time (when the cache is used which 
>>> required a small modification to pygccxml) and decoration was also 
>>> noticeably faster (~1:40min) because there were less declarations to 
>>> consider (as already mentioned in my earlier mail).
>> Is this similar to the code I have in goodies that customizes the module 
>> builder calls to cache the module builder's state?  I guess what I am 
>> asking, is will your changes replace the customizations I am doing in 
>> goodies?
> 
> I'm not exactly sure which customizations you mean, but I don't think 
> I'm interfering with your customizations. The only modification to Py++ 
> that I needed is already checked in (the modification to pygccxml so 
> that XML files get cached). The actual pruning happens outside Py++ and 
> doesn't need any further customization.
> 
> And for the query cache the only customization I needed was to override 
> the constructor of declaration_t which I can do within my own code so 
> it's not necessary to modify Py++ itself.

Ok. The modification I did in goodies was to override the module builder.

I as not worried at all about interference, I was actually hoping that 
what you were doing would find it's way into Py++ so I could get rid of 
my extensions.  I would like to see a way that everyone can see the 
additional performance of better caching.  I hope to try out some of 
your performance improvements soon.  I can always find a need for better 
performance. :)

Thanks for taking the time to look into this and improve the project.

-Allen