Re: [pygccxml-development] Another performance tweak

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

> On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
>
>> Roman Yakovenko wrote:
>>
>> > On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
>> >
>> >> Roman Yakovenko wrote:
>> >>
>> >> > On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
>> >> >
>> >> >> I now have preliminary numbers. My build that used to take 58 
>> seconds
>> >> >> now takes 76 seconds with your caching changes. So it looks 
>> right now
>> >> >> like the implementations I was using were about 25% faster for 
>> some
>> >> >> reason.
>> >> >>
>> >> >> Any thoughts?
>> >> >
>> >> >
>> >> > I don't know why, but I think I prefer to pay this price. I prefer
>> >> > code, that I can maintain. Sorry.
>> >>
>> >> It is probably no surprise that I disagree.  The extra 25% 
>> performance
>> >> seems like a good thing to me.
>> >
>> >
>> > :-). It is not 25%. The original time was 12 minutes. Your
>> > optimization brought it to
>> > 1 minutes, than my changes brought it to 1.25 minutes.
>>
>> The 25% is that in my current version, my implementations take 58
>> seconds and with yours it takes 76 seconds.   That is a 25% difference
>> with my math at least. :)
>
>
> I count from 12 minutes.

Well that is just cheating then. :)

>
>> >
>> >> Why were the extra layers of indirection needed in your 
>> implementation?
>> >>
>> >> In my implementation I just tried to keep everything local to the 
>> method
>> >> I was optimizing.  In that way I thought it was pretty maintainable
>> >> because that method was the only place in the code that set or 
>> used the
>> >> cache value.  This still allowed for disabling caching by using a 
>> module
>> >> level variable to prevent the cache from being set.
>> >>
>> >> What caused this local encapsulation to be less maintainable?
>> >
>> >
>> > You can not answer the question "what optimization pygccxml does" 
>> without
>> > scanning the whole source code. pygccxml supports "file by file"
>> > compilation mode.
>> > When it joins the declarations tree, it have to clear all declaration
>> > and type caches.
>> > It is very easy to write decl.cache.reset(). While in your case,
>> > developer ( me ) has
>> > always scan all sources and to find out what attributes should be
>> > reset. Another problem
>> > is when new "cache value" is introduced. I can bet, that I will forget
>> > to add it to all
>> > places where I need reset. Thus the software will become buggy.
>> >
>> > This is just an example to problem my implementation solves. There is
>> > new concept
>> > in pygccxml "cache" and I want it to be presented in a right way.
>>
>> Why didn't I need the ability to reset in my versions?
>
>
> You do. You didn't run unit tests, right? Otherwise typedef_tester.py
> would fail.

I ran the unit tests but maybe I missed it for remove_alias.

>
>> Do all the
>> methods really need reset.  I mean why do we have to reset something
>> that is always constant for a given decl?  (ex: access type)
>
>
> You treat pygccxml declarations tree as read only, right? If you think 
> about
> read\write than you really need to reset it.
>
>> I agree, if there are multiple places in the code where the cache would
>> have to be reset or touched then that is an issue.  But I didn't run
>> across that with my implementations.  I know you had a corner case with
>> remove_alias that took a long time to track down, was that because of
>> needing a reset?
>
>
> Yes.
> removed_alias on a typedef, after join of declarations tree contained
> reference to
> the removed class. Some operation stopped working. After this I
> understand, that to be
> on a safe side I need to reset declaration cache too.
>
>> >> > You already achieved x10 improvement. This is a grate result. Lets
>> >> > stop here.
>> >> >
>> >> I don't know if I will ever stop looking for ways to make this 
>> code run
>> >> faster, but I will probably stop soon so I can just use the code 
>> instead
>> >> of trying to improve it.
>> >
>> >
>> > Please don't take it personal, without your ideas and work Py++ would
>> > not become
>> > such powerful and good tool.
>>
>> I don't take it personal.  I just really need py++ to run faster so I
>> can work with it more easily.  It is much better now then it was, but
>> now that I have seen how many opportunities there are for improvement I
>> just want to make sure I don't miss any easy fixes. :)
>
>
> I did not optimize Py++ at all. I just make it to run "fast enough" on
> my projects.
> So please do it.
>
I am still trying to make it "fast enough" for my projects. :)

-Allen