Re: [pygccxml-development] Another performance tweak

[Roman Y. <rom...@gm...>]

On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
> Roman Yakovenko wrote:
>
> > On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
> >
> >> Roman Yakovenko wrote:
> >>
> >> > On 8/28/06, Allen Bierbaum <al...@vr...> wrote:
> >> >
> >> >> I now have preliminary numbers. My build that used to take 58 seconds
> >> >> now takes 76 seconds with your caching changes. So it looks right now
> >> >> like the implementations I was using were about 25% faster for some
> >> >> reason.
> >> >>
> >> >> Any thoughts?
> >> >
> >> >
> >> > I don't know why, but I think I prefer to pay this price. I prefer
> >> > code, that I can maintain. Sorry.
> >>
> >> It is probably no surprise that I disagree.  The extra 25% performance
> >> seems like a good thing to me.
> >
> >
> > :-). It is not 25%. The original time was 12 minutes. Your
> > optimization brought it to
> > 1 minutes, than my changes brought it to 1.25 minutes.
>
> The 25% is that in my current version, my implementations take 58
> seconds and with yours it takes 76 seconds.   That is a 25% difference
> with my math at least. :)

I count from 12 minutes.

> >
> >> Why were the extra layers of indirection needed in your implementation?
> >>
> >> In my implementation I just tried to keep everything local to the method
> >> I was optimizing.  In that way I thought it was pretty maintainable
> >> because that method was the only place in the code that set or used the
> >> cache value.  This still allowed for disabling caching by using a module
> >> level variable to prevent the cache from being set.
> >>
> >> What caused this local encapsulation to be less maintainable?
> >
> >
> > You can not answer the question "what optimization pygccxml does" without
> > scanning the whole source code. pygccxml supports "file by file"
> > compilation mode.
> > When it joins the declarations tree, it have to clear all declaration
> > and type caches.
> > It is very easy to write decl.cache.reset(). While in your case,
> > developer ( me ) has
> > always scan all sources and to find out what attributes should be
> > reset. Another problem
> > is when new "cache value" is introduced. I can bet, that I will forget
> > to add it to all
> > places where I need reset. Thus the software will become buggy.
> >
> > This is just an example to problem my implementation solves. There is
> > new concept
> > in pygccxml "cache" and I want it to be presented in a right way.
>
> Why didn't I need the ability to reset in my versions?

You do. You didn't run unit tests, right? Otherwise typedef_tester.py
would fail.

> Do all the
> methods really need reset.  I mean why do we have to reset something
> that is always constant for a given decl?  (ex: access type)

You treat pygccxml declarations tree as read only, right? If you think about
read\write than you really need to reset it.

> I agree, if there are multiple places in the code where the cache would
> have to be reset or touched then that is an issue.  But I didn't run
> across that with my implementations.  I know you had a corner case with
> remove_alias that took a long time to track down, was that because of
> needing a reset?

Yes.
removed_alias on a typedef, after join of declarations tree contained
reference to
the removed class. Some operation stopped working. After this I
understand, that to be
on a safe side I need to reset declaration cache too.

> >> > You already achieved x10 improvement. This is a grate result. Lets
> >> > stop here.
> >> >
> >> I don't know if I will ever stop looking for ways to make this code run
> >> faster, but I will probably stop soon so I can just use the code instead
> >> of trying to improve it.
> >
> >
> > Please don't take it personal, without your ideas and work Py++ would
> > not become
> > such powerful and good tool.
>
> I don't take it personal.  I just really need py++ to run faster so I
> can work with it more easily.  It is much better now then it was, but
> now that I have seen how many opportunities there are for improvement I
> just want to make sure I don't miss any easy fixes. :)

I did not optimize Py++ at all. I just make it to run "fast enough" on
my projects.
So please do it.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/