Re: [pygccxml-development] Another performance tweak
Brought to you by:
mbaas,
roman_yakovenko
Menu
▾
▴
Re: [pygccxml-development] Another performance tweak
|
From: Allen B. <al...@vr...> - 2006-08-28 19:22:59
|
I found one other place. Using make_flatten_generator in pyplusplus (exposing it and then using it actually) shaves off another 8-10%. The profile list now looks pretty flat. In other words there is nothing dominating. All of the cached methods are now showing up as the most costly in the profile but they are all about equal in terms of time. I would still like to see a faster implementation of these things so we can get back that extra 25% increase but I guess i will live with it if you don't want to optimize it further. I think any remaining optimizations will need to be done by making algorithms smarter so they do less work. For now I think I am done with my optimization efforts though. I need to get real work done. -Allen Allen Bierbaum wrote: >Roman Yakovenko wrote: > > > >>On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >> >> >> >>>Roman Yakovenko wrote: >>> >>> >>> >>>>On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >>>> >>>> >>>> >>>>>Roman Yakovenko wrote: >>>>> >>>>> >>>>> >>>>>>On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >>>>>> >>>>>> >>>>>> >>>>>>>I now have preliminary numbers. My build that used to take 58 >>>>>>> >>>>>>> >>>seconds >>> >>> >>>>>>>now takes 76 seconds with your caching changes. So it looks >>>>>>> >>>>>>> >>>right now >>> >>> >>>>>>>like the implementations I was using were about 25% faster for >>>>>>> >>>>>>> >>>some >>> >>> >>>>>>>reason. >>>>>>> >>>>>>>Any thoughts? >>>>>>> >>>>>>> >>>>>>I don't know why, but I think I prefer to pay this price. I prefer >>>>>>code, that I can maintain. Sorry. >>>>>> >>>>>> >>>>>It is probably no surprise that I disagree. The extra 25% >>>>> >>>>> >>>performance >>> >>> >>>>>seems like a good thing to me. >>>>> >>>>> >>>>:-). It is not 25%. The original time was 12 minutes. Your >>>>optimization brought it to >>>>1 minutes, than my changes brought it to 1.25 minutes. >>>> >>>> >>>The 25% is that in my current version, my implementations take 58 >>>seconds and with yours it takes 76 seconds. That is a 25% difference >>>with my math at least. :) >>> >>> >>I count from 12 minutes. >> >> > >Well that is just cheating then. :) > > > >>>>>Why were the extra layers of indirection needed in your >>>>> >>>>> >>>implementation? >>> >>> >>>>>In my implementation I just tried to keep everything local to the >>>>> >>>>> >>>method >>> >>> >>>>>I was optimizing. In that way I thought it was pretty maintainable >>>>>because that method was the only place in the code that set or >>>>> >>>>> >>>used the >>> >>> >>>>>cache value. This still allowed for disabling caching by using a >>>>> >>>>> >>>module >>> >>> >>>>>level variable to prevent the cache from being set. >>>>> >>>>>What caused this local encapsulation to be less maintainable? >>>>> >>>>> >>>>You can not answer the question "what optimization pygccxml does" >>>> >>>> >>>without >>> >>> >>>>scanning the whole source code. pygccxml supports "file by file" >>>>compilation mode. >>>>When it joins the declarations tree, it have to clear all declaration >>>>and type caches. >>>>It is very easy to write decl.cache.reset(). While in your case, >>>>developer ( me ) has >>>>always scan all sources and to find out what attributes should be >>>>reset. Another problem >>>>is when new "cache value" is introduced. I can bet, that I will forget >>>>to add it to all >>>>places where I need reset. Thus the software will become buggy. >>>> >>>>This is just an example to problem my implementation solves. There is >>>>new concept >>>>in pygccxml "cache" and I want it to be presented in a right way. >>>> >>>> >>>Why didn't I need the ability to reset in my versions? >>> >>> >>You do. You didn't run unit tests, right? Otherwise typedef_tester.py >>would fail. >> >> > >I ran the unit tests but maybe I missed it for remove_alias. > > > >>>Do all the >>>methods really need reset. I mean why do we have to reset something >>>that is always constant for a given decl? (ex: access type) >>> >>> >>You treat pygccxml declarations tree as read only, right? If you think >>about >>read\write than you really need to reset it. >> >> >> >>>I agree, if there are multiple places in the code where the cache would >>>have to be reset or touched then that is an issue. But I didn't run >>>across that with my implementations. I know you had a corner case with >>>remove_alias that took a long time to track down, was that because of >>>needing a reset? >>> >>> >>Yes. >>removed_alias on a typedef, after join of declarations tree contained >>reference to >>the removed class. Some operation stopped working. After this I >>understand, that to be >>on a safe side I need to reset declaration cache too. >> >> >> >>>>>>You already achieved x10 improvement. This is a grate result. Lets >>>>>>stop here. >>>>>> >>>>>> >>>>>> >>>>>I don't know if I will ever stop looking for ways to make this >>>>> >>>>> >>>code run >>> >>> >>>>>faster, but I will probably stop soon so I can just use the code >>>>> >>>>> >>>instead >>> >>> >>>>>of trying to improve it. >>>>> >>>>> >>>>Please don't take it personal, without your ideas and work Py++ would >>>>not become >>>>such powerful and good tool. >>>> >>>> >>>I don't take it personal. I just really need py++ to run faster so I >>>can work with it more easily. It is much better now then it was, but >>>now that I have seen how many opportunities there are for improvement I >>>just want to make sure I don't miss any easy fixes. :) >>> >>> >>I did not optimize Py++ at all. I just make it to run "fast enough" on >>my projects. >>So please do it. >> >> >> >I am still trying to make it "fast enough" for my projects. :) > >-Allen > > > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >pygccxml-development mailing list >pyg...@li... >https://lists.sourceforge.net/lists/listinfo/pygccxml-development > > > |
