#6388 Deep trace of single pageload

forge-dec-27
closed
General
2
2014-01-02
2013-06-24
Dave Brondsema
No

Let's do a trace (e.g. start with TimerMiddleware verbose mode to get every pymongo call; maybe try line_profiler later) of common functionality to every page. Focus on just the core parts of allura, not looking into the specifics of whatever page you're on. Might be helpful to test hitting an external link URL, so there's not much tool-specific going on, just core route handling, project/tool lookup, permission checks, etc. Identify opportunities for improvement.

Related

Chat: 52c596f60594ca5f48ca91d2
Chat: 52c596f70594ca5f48ca91d5
Chat: 52c596f70594ca5f48ca91d8
Chat: 52c596f70594ca5f48ca91db
Chat: 52c596f70594ca5f48ca91de
Chat: 52c596f70594ca5f48ca91e1
Chat: 52c596f70594ca5f48ca91e4
Chat: 52c596f70594ca5f48ca91e7
Chat: 52c5a39b0594ca5f48ca91fc
Chat: 52c5a39b0594ca5f48ca91ff
Chat: 52c5cae70594ca5f48ca920e
Chat: 52c5cae70594ca5f48ca9211

Discussion

1 2 > >> (Page 1 of 2)
  • Dave Brondsema
    Dave Brondsema
    2013-12-09

    Based on New Relic "xray sessions", potential candidates to investigate more are:

    • controllers/root.py Neighborhood.query.get: 11% of avg _nav.json request. Time spent is all in mongo. Cache it for a while?
    • model/project.py install_anchored_tools: 15% of avg _nav.json request. Most of time spent is in mongo. Seems to be the self.app_configs property load?
    • lib/security.py obj.neighborhood_project: 7.3% of avg _nav.json request. Cache it for a while?
    • post_thread/post_widget.html has repeated sequential calls to Artifact.refs taking 13% of avg ticket view, backrefs (5.2%), user icon_url which is already memozied (8.4%)
     
    Last edit: Dave Brondsema 2013-12-09
    • Dave Brondsema
      Dave Brondsema
      2013-12-19

      Neighborhood.query.get is all over the place. For example in a discussion thread, each post calls User.icon_url which calls private_project which uses self.neighborhood but even though that is a LazyProperty it is instance-level so doesn't get re-used much. There are many other places too.

      We could have a general purpose Neighborhood cache system and use it in lots of places. Some tests might not want it to be cached though. Another even generalized possibility is to extend Ming's identity map. It currently indexes objects by _id only, but we could make it do lookups by any unique key(s) for a given model (e.g. url_prefix for Neighborhood).

       
  • Dave Brondsema
    Dave Brondsema
    2013-12-20

    allura:db/6388

    There are a few performance & configuration improvements in this branch, but none of the particular areas mentioned above are addressed. Most of the work in this branch is a script to have a standard benchmark of call counts that can be run for any git revision (can't go earlier than the creation of the script though).

    Run the main script to get current call counts, and to see specific calls allura/scripts/perf$ ./call_count.py -v

    Use the shell wrapper to create a CSV file with multiple commits recorded (suitable for graphing in a spreadsheet app). E.g.: allura/scripts/perf$ ./call_count_walk_git.sh bdde98d..HEAD

    For integration with Jenkins, we should be able to run call_count.py --data-file /some/permanent/location.csv and hook that up to https://wiki.jenkins-ci.org/display/JENKINS/Plot+Plugin

     
1 2 > >> (Page 1 of 2)


Anonymous


Cancel   Add attachments