[Sqlalchemy-tickets] Issue #3307: Performance with PyPy (zzzeek/sqlalchemy)
Brought to you by:
zzzeek
Menu
▾
▴
[Sqlalchemy-tickets] Issue #3307: Performance with PyPy (zzzeek/sqlalchemy)
|
From: zoomorph <iss...@bi...> - 2015-02-13 10:20:54
|
New issue 3307: Performance with PyPy https://bitbucket.org/zzzeek/sqlalchemy/issue/3307/performance-with-pypy zoomorph: SQLAlchemy seems to run quite a bit slower (2-3x) under PyPy. I've gathered some pstats profiles comparing the two. PyPy (latest nightly build) + SQLAlchemy (master branch) + psycopg2cffi: ``` #!python 1 0.000 0.000 0.017 0.017 /myapp/tpserv/client/client.py:285(handle_client_line) 1 0.000 0.000 0.017 0.017 /myapp/tpserv/client/client.py:859(handle_request_statistics) 1 0.000 0.000 0.017 0.017 /myapp/tpserv/tank/tank.py:419(initial_load) 1 0.000 0.000 0.014 0.014 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:2435(first) 1 0.000 0.000 0.014 0.014 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:2267(__getitem__) 1 0.000 0.000 0.012 0.012 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:2516(__iter__) 1 0.000 0.000 0.010 0.010 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:2530(_execute_and_instances) 1 0.000 0.000 0.010 0.010 /myapp/env_pypy/site-packages/sqlalchemy/engine/base.py:846(execute) 1 0.000 0.000 0.010 0.010 /myapp/env_pypy/site-packages/sqlalchemy/sql/elements.py:322(_execute_on_connection) 1 0.000 0.000 0.010 0.010 /myapp/env_pypy/site-packages/sqlalchemy/engine/base.py:975(_execute_clauseelement) 1 0.000 0.000 0.005 0.005 <string>:1(<lambda>) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/elements.py:431(compile) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/elements.py:496(_compiler) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:328(__init__) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:166(__init__) 2/1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:211(process) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/engine/base.py:1062(_execute_context) 50/1 0.002 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/visitors.py:75(_compiler_dispatch) 1 0.000 0.000 0.005 0.005 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:1474(visit_select) 22 0.000 0.000 0.003 0.000 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:1254(_label_select_column) 1 0.000 0.000 0.003 0.003 /myapp/env_pypy/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py:457(get_result_proxy) 1 0.000 0.000 0.003 0.003 /myapp/env_pypy/site-packages/sqlalchemy/engine/result.py:407(__init__) 1 0.000 0.000 0.003 0.003 /myapp/env_pypy/site-packages/sqlalchemy/engine/result.py:423(_init_metadata) 1 0.003 0.003 0.003 0.003 /myapp/env_pypy/site-packages/sqlalchemy/engine/result.py:189(__init__) 22 0.000 0.000 0.003 0.000 /myapp/env_pypy/site-packages/sqlalchemy/sql/compiler.py:551(visit_label) 2 0.000 0.000 0.002 0.001 /myapp/env_pypy/site-packages/sqlalchemy/orm/loading.py:27(instances) 2 0.000 0.000 0.002 0.001 /myapp/env_pypy/site-packages/psycopg2cffi/_impl/cursor.py:18(check_closed_) 1 0.000 0.000 0.002 0.002 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:2934(_compile_context) 2 0.000 0.000 0.001 0.001 /usr/local/pypy/lib-python/2.7/logging/__init__.py:1163(info) 2 0.000 0.000 0.001 0.001 /usr/local/pypy/lib-python/2.7/logging/__init__.py:1273(_log) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/engine/default.py:441(do_execute) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/psycopg2cffi/_impl/cursor.py:193(execute) 10/5 0.000 0.000 0.001 0.000 /myapp/env_pypy/site-packages/sqlalchemy/sql/operators.py:294(__eq__) 2 0.000 0.000 0.001 0.001 /usr/local/pypy/lib-python/2.7/logging/__init__.py:1294(handle) 9/7 0.000 0.000 0.001 0.000 {operator.eq} 2 0.000 0.000 0.001 0.001 /usr/local/pypy/lib-python/2.7/logging/__init__.py:1326(callHandlers) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:3297(setup_context) 6 0.000 0.000 0.001 0.000 /usr/local/pypy/lib-python/2.7/logging/__init__.py:757(handle) 3 0.000 0.000 0.001 0.000 /myapp/env_pypy/site-packages/sqlalchemy/orm/attributes.py:174(operate) 48 0.001 0.000 0.001 0.000 /myapp/env_pypy/site-packages/sqlalchemy/orm/interfaces.py:459(_get_context_loader) 1 0.001 0.001 0.001 0.001 /myapp/env_pypy/site-packages/psycopg2cffi/_impl/cursor.py:675(_pq_execute) 24 0.000 0.000 0.001 0.000 /myapp/env_pypy/site-packages/sqlalchemy/orm/interfaces.py:491(setup) 2 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/orm/relationships.py:950(__eq__) 6 0.000 0.000 0.001 0.000 /usr/local/pypy/lib-python/2.7/logging/__init__.py:860(emit) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/orm/query.py:3258(row_processor) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/orm/loading.py:221(instance_processor) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/engine/result.py:806(fetchall) 1 0.000 0.000 0.001 0.001 /myapp/env_pypy/site-packages/sqlalchemy/engine/result.py:775(_fetchall_impl) ``` Python 2.7 + SQLAlchemy (master branch) + psycopg2: ``` #!python 1 0.000 0.000 0.005 0.005 /myapp/tpserv/client/client.py:285(handle_client_line) 1 0.000 0.000 0.005 0.005 /myapp/tpserv/client/client.py:859(handle_request_statistics) 1 0.000 0.000 0.005 0.005 /myapp/tpserv/tank/tank.py:419(initial_load) 1 0.000 0.000 0.004 0.004 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2435(first) 1 0.000 0.000 0.004 0.004 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2267(__getitem__) 1 0.000 0.000 0.003 0.003 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2516(__iter__) 1 0.000 0.000 0.003 0.003 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2530(_execute_and_instances) 1 0.000 0.000 0.003 0.003 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py:846(execute) 1 0.000 0.000 0.003 0.003 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:322(_execute_on_connection) 1 0.000 0.000 0.003 0.003 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py:975(_execute_clauseelement) 1 0.000 0.000 0.001 0.001 <string>:1(<lambda>) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:431(compile) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:496(_compiler) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:328(__init__) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:166(__init__) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1062(_execute_context) 2/1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:211(process) 50/1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py:75(_compiler_dispatch) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:1474(visit_select) 2 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:1142(info) 2 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:1252(_log) 2 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:1273(handle) 2 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:1305(callHandlers) 6 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:736(handle) 22 0.000 0.000 0.001 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:1254(_label_select_column) 1 0.000 0.000 0.001 0.001 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py:441(do_execute) 1 0.001 0.001 0.001 0.001 {method 'execute' of 'psycopg2._psycopg.cursor' objects} 6 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:839(emit) 22 0.000 0.000 0.001 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:551(visit_label) 4 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/handlers.py:68(emit) 10/5 0.000 0.000 0.001 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/operators.py:294(__eq__) 4 0.000 0.000 0.001 0.000 /usr/lib/python2.7/logging/__init__.py:933(emit) 9/7 0.000 0.000 0.001 0.000 {operator.eq} 1 0.000 0.000 0.001 0.001 /usr/lib/python2.7/contextlib.py:21(__exit__) 1 0.000 0.000 0.001 0.001 /myapp/tpcommon/profile.py:11(profile) 3 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py:174(operate) 2 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/relationships.py:950(__eq__) 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2934(_compile_context) 2 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py:27(instances) 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:1616(_compose_select_body) 26 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:590(visit_column) 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py:457(get_result_proxy) 8/2 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py:86(_compiler_dispatch) 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/result.py:407(__init__) 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/result.py:423(_init_metadata) 7/6 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects} 1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py:507(_init_compiled) 2/1 0.000 0.000 0.000 0.000 /myapp/env_python/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py:705(visit_clauselist) ``` initial_load basically executes one select query. Perhaps some optimizations can be made for better PyPy performance? |
