Thread: [Sqlalchemy-commits] [2174] sqlalchemy/trunk: - speed enhancements to ORM object instantiation, eag
Brought to you by:
zzzeek
From: <co...@sq...> - 2006-12-28 06:28:17
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head><style type="text/css"><!-- #msg dl { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; } #msg dt { float: left; width: 6em; font-weight: bold; } #msg dt:after { content:':';} #msg dl, #msg dt, #msg ul, #msg li { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; } #msg dl a { font-weight: bold} #msg dl a:link { color:#fc3; } #msg dl a:active { color:#ff0; } #msg dl a:visited { color:#cc6; } h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; } #msg pre { overflow: auto; background: #ffc; border: 1px #fc0 solid; padding: 6px; } #msg ul, pre { overflow: auto; } #patch { width: 100%; } #patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;} #patch .propset h4, #patch .binary h4 {margin:0;} #patch pre {padding:0;line-height:1.2em;margin:0;} #patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;} #patch .propset .diff, #patch .binary .diff {padding:10px 0;} #patch span {display:block;padding:0 10px;} #patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;} #patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;} #patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;} #patch .lines, .info {color:#888;background:#fff;} --></style> <title>[2174] sqlalchemy/trunk: - speed enhancements to ORM object instantiation, eager loading of rows</title> </head> <body> <div id="msg"> <dl> <dt>Revision</dt> <dd>2174</dd> <dt>Author</dt> <dd>zzzeek</dd> <dt>Date</dt> <dd>2006-12-28 02:30:43 -0600 (Thu, 28 Dec 2006)</dd> </dl> <h3>Log Message</h3> <pre>- speed enhancements to ORM object instantiation, eager loading of rows</pre> <h3>Modified Paths</h3> <ul> <li><a href="#sqlalchemytrunkCHANGES">sqlalchemy/trunk/CHANGES</a></li> <li><a href="#sqlalchemytrunklibsqlalchemyormattributespy">sqlalchemy/trunk/lib/sqlalchemy/orm/attributes.py</a></li> <li><a href="#sqlalchemytrunklibsqlalchemyorminterfacespy">sqlalchemy/trunk/lib/sqlalchemy/orm/interfaces.py</a></li> <li><a href="#sqlalchemytrunklibsqlalchemyormmapperpy">sqlalchemy/trunk/lib/sqlalchemy/orm/mapper.py</a></li> <li><a href="#sqlalchemytrunklibsqlalchemyormstrategiespy">sqlalchemy/trunk/lib/sqlalchemy/orm/strategies.py</a></li> </ul> </div> <div id="patch"> <h3>Diff</h3> <a id="sqlalchemytrunkCHANGES"></a> <div class="modfile"><h4>Modified: sqlalchemy/trunk/CHANGES (2173 => 2174)</h4> <pre class="diff"><span> <span class="info">--- sqlalchemy/trunk/CHANGES 2006-12-28 00:27:58 UTC (rev 2173) +++ sqlalchemy/trunk/CHANGES 2006-12-28 08:30:43 UTC (rev 2174) </span><span class="lines">@@ -7,6 +7,7 @@ </span><span class="cx"> is constructed with individual calls to append_column(); this fixes an ORM </span><span class="cx"> bug whereby nested select statements were not getting correlated with the </span><span class="cx"> main select generated by the Query object. </span><ins>+- speed enhancements to ORM object instantiation, eager loading of rows </ins><span class="cx"> </span><span class="cx"> 0.3.3 </span><span class="cx"> - string-based FROM clauses fixed, i.e. select(..., from_obj=["sometext"]) </span></span></pre></div> <a id="sqlalchemytrunklibsqlalchemyormattributespy"></a> <div class="modfile"><h4>Modified: sqlalchemy/trunk/lib/sqlalchemy/orm/attributes.py (2173 => 2174)</h4> <pre class="diff"><span> <span class="info">--- sqlalchemy/trunk/lib/sqlalchemy/orm/attributes.py 2006-12-28 00:27:58 UTC (rev 2173) +++ sqlalchemy/trunk/lib/sqlalchemy/orm/attributes.py 2006-12-28 08:30:43 UTC (rev 2174) </span><span class="lines">@@ -758,11 +758,9 @@ </span><span class="cx"> #print self, "register attribute", key, "for class", class_ </span><span class="cx"> if not hasattr(class_, '_state'): </span><span class="cx"> def _get_state(self): </span><del>- try: - return self.__sa_attr_state - except AttributeError: - self.__sa_attr_state = {} - return self.__sa_attr_state </del><ins>+ if not hasattr(self, '_sa_attr_state'): + self._sa_attr_state = {} + return self._sa_attr_state </ins><span class="cx"> class_._state = property(_get_state) </span><span class="cx"> </span><span class="cx"> typecallable = kwargs.pop('typecallable', None) </span></span></pre></div> <a id="sqlalchemytrunklibsqlalchemyorminterfacespy"></a> <div class="modfile"><h4>Modified: sqlalchemy/trunk/lib/sqlalchemy/orm/interfaces.py (2173 => 2174)</h4> <pre class="diff"><span> <span class="info">--- sqlalchemy/trunk/lib/sqlalchemy/orm/interfaces.py 2006-12-28 00:27:58 UTC (rev 2173) +++ sqlalchemy/trunk/lib/sqlalchemy/orm/interfaces.py 2006-12-28 08:30:43 UTC (rev 2174) </span><span class="lines">@@ -61,11 +61,18 @@ </span><span class="cx"> There is a single default strategy selected, and alternate strategies can be selected </span><span class="cx"> at selection time through the usage of StrategizedOption objects.""" </span><span class="cx"> def _get_context_strategy(self, context): </span><del>- return self._get_strategy(context.attributes.get((LoaderStrategy, self), self.strategy.__class__)) </del><ins>+ try: + return context.attributes[id(self)] + except KeyError: + # cache the located strategy per StrategizedProperty in the given context for faster re-lookup + ctx_strategy = self._get_strategy(context.attributes.get((LoaderStrategy, self), self.strategy.__class__)) + context.attributes[id(self)] = ctx_strategy + return ctx_strategy </ins><span class="cx"> def _get_strategy(self, cls): </span><span class="cx"> try: </span><span class="cx"> return self._all_strategies[cls] </span><span class="cx"> except KeyError: </span><ins>+ # cache the located strategy per class for faster re-lookup </ins><span class="cx"> strategy = cls(self) </span><span class="cx"> strategy.init() </span><span class="cx"> strategy.is_default = False </span></span></pre></div> <a id="sqlalchemytrunklibsqlalchemyormmapperpy"></a> <div class="modfile"><h4>Modified: sqlalchemy/trunk/lib/sqlalchemy/orm/mapper.py (2173 => 2174)</h4> <pre class="diff"><span> <span class="info">--- sqlalchemy/trunk/lib/sqlalchemy/orm/mapper.py 2006-12-28 00:27:58 UTC (rev 2173) +++ sqlalchemy/trunk/lib/sqlalchemy/orm/mapper.py 2006-12-28 08:30:43 UTC (rev 2174) </span><span class="lines">@@ -1352,25 +1352,44 @@ </span><span class="cx"> class _ExtensionCarrier(MapperExtension): </span><span class="cx"> def __init__(self): </span><span class="cx"> self.__elements = [] </span><del>- self.__callables = {} </del><span class="cx"> def insert(self, extension): </span><span class="cx"> """insert a MapperExtension at the beginning of this ExtensionCarrier's list.""" </span><span class="cx"> self.__elements.insert(0, extension) </span><span class="cx"> def append(self, extension): </span><span class="cx"> """append a MapperExtension at the end of this ExtensionCarrier's list.""" </span><span class="cx"> self.__elements.append(extension) </span><del>- def __getattribute__(self, key): - if key in MapperExtension.__dict__: - try: - return self.__callables[key] - except KeyError: - return self.__callables.setdefault(key, lambda *args, **kwargs:self._do(key, *args, **kwargs)) - else: - return super(_ExtensionCarrier, self).__getattribute__(key) </del><ins>+ def get_session(self, *args, **kwargs): + return self._do('get_session', *args, **kwargs) + def load(self, *args, **kwargs): + return self._do('load', *args, **kwargs) + def get(self, *args, **kwargs): + return self._do('get', *args, **kwargs) + def get_by(self, *args, **kwargs): + return self._do('get_by', *args, **kwargs) + def select_by(self, *args, **kwargs): + return self._do('select_by', *args, **kwargs) + def select(self, *args, **kwargs): + return self._do('select', *args, **kwargs) + def create_instance(self, *args, **kwargs): + return self._do('create_instance', *args, **kwargs) + def append_result(self, *args, **kwargs): + return self._do('append_result', *args, **kwargs) + def populate_instance(self, *args, **kwargs): + return self._do('populate_instance', *args, **kwargs) + def before_insert(self, *args, **kwargs): + return self._do('before_insert', *args, **kwargs) + def before_update(self, *args, **kwargs): + return self._do('before_update', *args, **kwargs) + def after_update(self, *args, **kwargs): + return self._do('after_update', *args, **kwargs) + def after_insert(self, *args, **kwargs): + return self._do('after_insert', *args, **kwargs) + def before_delete(self, *args, **kwargs): + return self._do('before_delete', *args, **kwargs) + def after_delete(self, *args, **kwargs): + return self._do('after_delete', *args, **kwargs) </ins><span class="cx"> def _do(self, funcname, *args, **kwargs): </span><span class="cx"> for elem in self.__elements: </span><del>- if elem is self: - raise exceptions.AssertionError("ExtensionCarrier set to itself") </del><span class="cx"> ret = getattr(elem, funcname)(*args, **kwargs) </span><span class="cx"> if ret is not EXT_PASS: </span><span class="cx"> return ret </span></span></pre></div> <a id="sqlalchemytrunklibsqlalchemyormstrategiespy"></a> <div class="modfile"><h4>Modified: sqlalchemy/trunk/lib/sqlalchemy/orm/strategies.py (2173 => 2174)</h4> <pre class="diff"><span> <span class="info">--- sqlalchemy/trunk/lib/sqlalchemy/orm/strategies.py 2006-12-28 00:27:58 UTC (rev 2173) +++ sqlalchemy/trunk/lib/sqlalchemy/orm/strategies.py 2006-12-28 08:30:43 UTC (rev 2174) </span><span class="lines">@@ -463,31 +463,55 @@ </span><span class="cx"> for value in self.mapper.props.values(): </span><span class="cx"> value.setup(context, eagertable=clauses.eagertarget, parentclauses=clauses, parentmapper=self.mapper) </span><span class="cx"> </span><ins>+ def _create_row_processor(self, selectcontext, row): + """create a 'row processing' function that will apply eager aliasing to the row. + + also check that an identity key can be retrieved from the row, else return None.""" + # check for a user-defined decorator in the SelectContext (which was set up by the contains_eager() option) + if selectcontext.attributes.has_key((EagerLoader, self.parent_property)): + # custom row decoration function, placed in the selectcontext by the + # contains_eager() mapper option + decorator = selectcontext.attributes[(EagerLoader, self.parent_property)] + if decorator is None: + decorator = lambda row: row + else: + try: + # decorate the row according to the stored AliasedClauses for this eager load + clauses = self.clauses_by_lead_mapper[selectcontext.mapper] + decorator = clauses._row_decorator + except KeyError: + # no stored AliasedClauses: eager loading was not set up in the query and + # AliasedClauses never got initialized + return None + + try: + decorated_row = decorator(row) + # check for identity key + identity_key = self.mapper.identity_key_from_row(decorated_row) + # and its good + return decorator + except KeyError: + # no identity key - dont return a row processor, will cause a degrade to lazy + return None + </ins><span class="cx"> def process_row(self, selectcontext, instance, row, identitykey, isnew): </span><span class="cx"> """receive a row. tell our mapper to look for a new object instance in the row, and attach </span><span class="cx"> it to a list on the parent instance.""" </span><del>- </del><span class="cx"> if self in selectcontext.recursion_stack: </span><span class="cx"> return </span><span class="cx"> </span><span class="cx"> try: </span><del>- # decorate the row according to the stored AliasedClauses for this eager load, - # or look for a user-defined decorator in the SelectContext (which was set up by the contains_eager() option) - if selectcontext.attributes.has_key((EagerLoader, self.parent_property)): - # custom row decoration function, placed in the selectcontext by the - # contains_eager() mapper option - decorator = selectcontext.attributes[(EagerLoader, self.parent_property)] - if decorator is None: - decorated_row = row - else: - decorated_row = decorator(row) - else: - clauses = self.clauses_by_lead_mapper[selectcontext.mapper] - decorated_row = clauses._decorate_row(row) - # check for identity key - identity_key = self.mapper.identity_key_from_row(decorated_row) </del><ins>+ # check for row processor + row_processor = selectcontext.attributes[id(self)] </ins><span class="cx"> except KeyError: </span><del>- # else degrade to a lazy loader </del><ins>+ # create a row processor function and cache it in the context + row_processor = self._create_row_processor(selectcontext, row) + selectcontext.attributes[id(self)] = row_processor + + if row_processor is not None: + decorated_row = row_processor(row) + else: + # row_processor was None: degrade to a lazy loader </ins><span class="cx"> if self._should_log_debug: </span><span class="cx"> self.logger.debug("degrade to lazy loader on %s" % mapperutil.attribute_str(instance, self.key)) </span><span class="cx"> self.parent_property._get_strategy(LazyLoader).process_row(selectcontext, instance, row, identitykey, isnew) </span></span></pre> </div> </div> </body> </html> |