Re: [pygccxml-development] Performance regression?

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:
> On 8/2/07, Allen Bierbaum <al...@vr...> wrote:
>> One of the big projects I use Py++ with seems to be suffering from
>> performance issues.  I used to be able to generate the bindings in
>> approximately 10 minutes, now it is taking a little over 80 minutes.
> 
> :-(. I will try to find out what happened. Can you provide the
> versions you are using?

Is is the latest version of Py++ with Python 2.5 and Ubuntu 7.04.  The 
previous work was done with Python 2.4 and FC6.

> Also I must admit that something strange is going on. I am testing
> Py++ on Python-Ogre project and I didn't see x8 performance
> deterioration ( hope this is the right word to use).
> 
> Do you run same version of script?

In the meantime the script has been extended to use some of the more 
advanced Py++ functionality in places.  I could probably back some of 
these changes out if we need to see the performance of an old version of 
Py++.

That said, nothing significant has changed in the binding generation 
script (just adding a couple transforms to a few classes).  The other 
thing of note is that the amount of time spent in the script is about 
the same as before.  The internal code of Py++ is where the increase is 
occuring.

> 
>> I have been so happy with Py++ as of late that I haven't even been
>> paying attention to the changes going into Py++ so I don't know what may
>> be causing the slow down.  :)
> 
> May be unicode support?

I guess this is possible.  If this is it, then there is not much that 
can be done to solve this part of it.  The best that could be done is to 
minimize the amount of comparisons performed by making some of the code 
smarter or caching some values.

> 
>> I did collect some timing information though in the hopes that someone
>> smarter then me may be able to spot some corners of Py++ that could be
>> optimized.
>>
>> During my run I output basic timings of:
>>
>> Code generation complete.  took: 5099.32698894
>>      ...
>>      module builder init: 253.73842907
>>      ...
>>      build creators: 1991.69337511
>>      write module: 2821.93180895
>>
>> I ran the whole thing through the cProfile module to see where python
>> was spending it's time. The raw data is available at:
>> https://realityforge.vrsource.org/twiki/pub/PyOpenSG/TempArea/gen_bindings.prof.bz2
>>
>> A report summarizing the results (sorted by time spent exclusively
>> within the method) is available here:
>> https://realityforge.vrsource.org/twiki/pub/PyOpenSG/TempArea/stats.txt
> 
> Can you also provide the profile information for your previous version of Py++?

That is quite a bit harder.  The script doesn't work with older versions 
of Py++ as far as I know.  If this is something that you really really 
need, I could try to find some time over the weekend.

I was hoping that you may just see something in the profile output that 
would point to a corner case in Py++ that is causing the slowdown.

One other thing that may be useful is converting the cProfile (lsprof) 
output into a calltree.  This should show who is calling the various 
methods and how often.  This may help in pointing out some particularly 
expensive method.  You can use the script here: 
http://ddaa.net/blog/python/lsprof-calltree to convert the data.

-Allen