Re: [pygccxml-development] Another performance tweak
Brought to you by:
mbaas,
roman_yakovenko
Menu
▾
▴
Re: [pygccxml-development] Another performance tweak
|
From: Allen B. <al...@vr...> - 2006-08-28 19:24:08
|
Allen Bierbaum wrote: > I found one other place. Using make_flatten_generator in pyplusplus > (exposing it and then using it actually) shaves off another 8-10%. > > The profile list now looks pretty flat. In other words there is > nothing dominating. All of the cached methods are now showing up as > the most costly in the profile but they are all about equal in terms > of time. I would still like to see a faster implementation of these > things so we can get back that extra 25% increase but I guess i will > live with it if you don't want to optimize it further. One correction here. remove_alias is definitely the most expensive method remaining. It currently takes up about 13% of the run-time. -Allen > > I think any remaining optimizations will need to be done by making > algorithms smarter so they do less work. For now I think I am done > with my optimization efforts though. I need to get real work done. > > -Allen > > > Allen Bierbaum wrote: > >> Roman Yakovenko wrote: >> >> >> >>> On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >>> >>> >>> >>>> Roman Yakovenko wrote: >>>> >>>> >>>> >>>>> On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >>>>> >>>>> >>>>> >>>>>> Roman Yakovenko wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On 8/28/06, Allen Bierbaum <al...@vr...> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> I now have preliminary numbers. My build that used to take 58 >>>>>>>> >>>>>>> >>>> seconds >>>> >>>> >>>>>>>> now takes 76 seconds with your caching changes. So it looks >>>>>>>> >>>>>>> >>>> right now >>>> >>>> >>>>>>>> like the implementations I was using were about 25% faster for >>>>>>>> >>>>>>> >>>> some >>>> >>>> >>>>>>>> reason. >>>>>>>> >>>>>>>> Any thoughts? >>>>>>>> >>>>>>> >>>>>>> I don't know why, but I think I prefer to pay this price. I prefer >>>>>>> code, that I can maintain. Sorry. >>>>>>> >>>>>> >>>>>> It is probably no surprise that I disagree. The extra 25% >>>>> >>>> performance >>>> >>>> >>>>>> seems like a good thing to me. >>>>>> >>>>> >>>>> :-). It is not 25%. The original time was 12 minutes. Your >>>>> optimization brought it to >>>>> 1 minutes, than my changes brought it to 1.25 minutes. >>>>> >>>> >>>> The 25% is that in my current version, my implementations take 58 >>>> seconds and with yours it takes 76 seconds. That is a 25% difference >>>> with my math at least. :) >>>> >>> >>> I count from 12 minutes. >>> >> >> >> Well that is just cheating then. :) >> >> >> >>>>>> Why were the extra layers of indirection needed in your >>>>> >>>> implementation? >>>> >>>> >>>>>> In my implementation I just tried to keep everything local to the >>>>>> >>>>> >>>> method >>>> >>>> >>>>>> I was optimizing. In that way I thought it was pretty maintainable >>>>>> because that method was the only place in the code that set or >>>>>> >>>>> >>>> used the >>>> >>>> >>>>>> cache value. This still allowed for disabling caching by using a >>>>>> >>>>> >>>> module >>>> >>>> >>>>>> level variable to prevent the cache from being set. >>>>>> >>>>>> What caused this local encapsulation to be less maintainable? >>>>>> >>>>> >>>>> You can not answer the question "what optimization pygccxml does" >>>>> >>>> >>>> without >>>> >>>> >>>>> scanning the whole source code. pygccxml supports "file by file" >>>>> compilation mode. >>>>> When it joins the declarations tree, it have to clear all declaration >>>>> and type caches. >>>>> It is very easy to write decl.cache.reset(). While in your case, >>>>> developer ( me ) has >>>>> always scan all sources and to find out what attributes should be >>>>> reset. Another problem >>>>> is when new "cache value" is introduced. I can bet, that I will >>>>> forget >>>>> to add it to all >>>>> places where I need reset. Thus the software will become buggy. >>>>> >>>>> This is just an example to problem my implementation solves. There is >>>>> new concept >>>>> in pygccxml "cache" and I want it to be presented in a right way. >>>>> >>>> >>>> Why didn't I need the ability to reset in my versions? >>>> >>> >>> You do. You didn't run unit tests, right? Otherwise typedef_tester.py >>> would fail. >>> >> >> >> I ran the unit tests but maybe I missed it for remove_alias. >> >> >> >>>> Do all the >>>> methods really need reset. I mean why do we have to reset something >>>> that is always constant for a given decl? (ex: access type) >>>> >>> >>> You treat pygccxml declarations tree as read only, right? If you >>> think about >>> read\write than you really need to reset it. >>> >>> >>> >>>> I agree, if there are multiple places in the code where the cache >>>> would >>>> have to be reset or touched then that is an issue. But I didn't run >>>> across that with my implementations. I know you had a corner case >>>> with >>>> remove_alias that took a long time to track down, was that because of >>>> needing a reset? >>>> >>> >>> Yes. >>> removed_alias on a typedef, after join of declarations tree contained >>> reference to >>> the removed class. Some operation stopped working. After this I >>> understand, that to be >>> on a safe side I need to reset declaration cache too. >>> >>> >>> >>>>>>> You already achieved x10 improvement. This is a grate result. Lets >>>>>>> stop here. >>>>>>> >>>>>>> >>>>>> >>>>>> I don't know if I will ever stop looking for ways to make this >>>>>> >>>>> >>>> code run >>>> >>>> >>>>>> faster, but I will probably stop soon so I can just use the code >>>>>> >>>>> >>>> instead >>>> >>>> >>>>>> of trying to improve it. >>>>>> >>>>> >>>>> Please don't take it personal, without your ideas and work Py++ would >>>>> not become >>>>> such powerful and good tool. >>>>> >>>> >>>> I don't take it personal. I just really need py++ to run faster so I >>>> can work with it more easily. It is much better now then it was, but >>>> now that I have seen how many opportunities there are for >>>> improvement I >>>> just want to make sure I don't miss any easy fixes. :) >>>> >>> >>> I did not optimize Py++ at all. I just make it to run "fast enough" on >>> my projects. >>> So please do it. >>> >>> >> >> I am still trying to make it "fast enough" for my projects. :) >> >> -Allen >> >> >> >> ------------------------------------------------------------------------- >> >> Using Tomcat but need to do more? Need to support web services, >> security? >> Get stuff done quickly with pre-integrated technology to make your >> job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache >> Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> pygccxml-development mailing list >> pyg...@li... >> https://lists.sourceforge.net/lists/listinfo/pygccxml-development >> >> >> > |
