Re: [pygccxml-development] Another performance tweak

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

> On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
>
>> Roman Yakovenko wrote:
>>
>> > On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
>> >
>> >> I now have preliminary numbers. My build that used to take 58 seconds
>> >> now takes 76 seconds with your caching changes. So it looks right now
>> >> like the implementations I was using were about 25% faster for some
>> >> reason.
>> >>
>> >> Any thoughts?
>> >
>> >
>> > I don't know why, but I think I prefer to pay this price. I prefer
>> > code, that I can maintain. Sorry.
>>
>> It is probably no surprise that I disagree.  The extra 25% performance
>> seems like a good thing to me.
>
>
> :-). It is not 25%. The original time was 12 minutes. Your
> optimization brought it to
> 1 minutes, than my changes brought it to 1.25 minutes.

The 25% is that in my current version, my implementations take 58 
seconds and with yours it takes 76 seconds.   That is a 25% difference 
with my math at least. :)

>
>> Why were the extra layers of indirection needed in your implementation?
>>
>> In my implementation I just tried to keep everything local to the method
>> I was optimizing.  In that way I thought it was pretty maintainable
>> because that method was the only place in the code that set or used the
>> cache value.  This still allowed for disabling caching by using a module
>> level variable to prevent the cache from being set.
>>
>> What caused this local encapsulation to be less maintainable?
>
>
> You can not answer the question "what optimization pygccxml does" without
> scanning the whole source code. pygccxml supports "file by file"
> compilation mode.
> When it joins the declarations tree, it have to clear all declaration
> and type caches.
> It is very easy to write decl.cache.reset(). While in your case,
> developer ( me ) has
> always scan all sources and to find out what attributes should be
> reset. Another problem
> is when new "cache value" is introduced. I can bet, that I will forget
> to add it to all
> places where I need reset. Thus the software will become buggy.
>
> This is just an example to problem my implementation solves. There is
> new concept
> in pygccxml "cache" and I want it to be presented in a right way.

Why didn't I need the ability to reset in my versions?  Do all the 
methods really need reset.  I mean why do we have to reset something 
that is always constant for a given decl?  (ex: access type)

I agree, if there are multiple places in the code where the cache would 
have to be reset or touched then that is an issue.  But I didn't run 
across that with my implementations.  I know you had a corner case with 
remove_alias that took a long time to track down, was that because of 
needing a reset?

>> > You already achieved x10 improvement. This is a grate result. Lets
>> > stop here.
>> >
>> I don't know if I will ever stop looking for ways to make this code run
>> faster, but I will probably stop soon so I can just use the code instead
>> of trying to improve it.
>
>
> Please don't take it personal, without your ideas and work Py++ would 
> not become
> such powerful and good tool.

I don't take it personal.  I just really need py++ to run faster so I 
can work with it more easily.  It is much better now then it was, but 
now that I have seen how many opportunities there are for improvement I 
just want to make sure I don't miss any easy fixes. :)

-Allen