Thread: [SQL-CVS] [ sqlobject-Patches-1407684 ] object initialization threading issue
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: SourceForge.net <no...@so...> - 2007-12-22 03:07:36
|
Patches item #1407684, was opened at 2006-01-17 08:35 Message generated for change (Comment added) made by j3z You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=540674&aid=1407684&group_id=74338 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Oleg Broytmann (phd) Summary: object initialization threading issue Initial Comment: The creation of SQL objects is not threadsafe. After stressing a TurboGears app with ab2 with multiple concurrent connections to a threaded server, the log gets filled with tracebacks like these: 15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/_cp httptools.py", line 96, in _run applyFilters('before_main') File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/fil ters/__init__.py", line 131, in applyFilters method() File "/home/alberto/turbogears/turbogears/visit.py", line 99, in beforeMain visit=self.get_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 139, in get_visit return self.new_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 122, in new_visit visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout ) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1197, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1221, in _create self.set(**kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1142, in set self._connection._SO_update(self, args) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/dbconnection.py", line 561, in _SO_update self.query("UPDATE %s SET %s WHERE %s = %s" % AttributeError: 'TG_Visit' object has no attribute 'id' 15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/_cp httptools.py", line 96, in _run applyFilters('before_main') File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/fil ters/__init__.py", line 131, in applyFilters method() File "/home/alberto/turbogears/turbogears/visit.py", line 99, in beforeMain visit=self.get_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 139, in get_visit return self.new_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 122, in new_visit visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout ) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1197, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1224, in _create self._SO_finishCreate(id) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1242, in _SO_finishCreate del self.sqlmeta._creating AttributeError: _creating I've found at http://pythonpaste.org/archives/message/ 20051202.223718.a3adcfaa.en.html a solution to the problem, which is overriding __init__ on SQLObject subclasses by wrapping SQLObject.__init__ with a mutex acquisition and release. I've made a patch for main.py which generalizes this approach and wraps __init__ for every SQLObject created. Tested under my environment and works fine. The patch is also posted at the turbogears Trac at http:// trac.turbogears.org/turbogears/attachment/ticket/397/ sqlobject_threading.patch Cheers, Alberto Valverde al...@to... ---------------------------------------------------------------------- Comment By: Jeremy Field (j3z) Date: 2007-12-22 14:07 Message: Logged In: YES user_id=932673 Originator: NO I think I am experiencing this or a similar problem in SQLObject 0.9.2. I've written a simple SQLObject subclass (A) within a fairly straightforward schema. It fails to instantiate, giving the error partially reproduced below. Interestingly, it has only started to fail since I added another SQLObject subclass (B) to the schema. This second subclass has a many-to-one relationship to the first, and was inserted between (A) and a third subclass (C) as follows: Before: A -one-to-many-> C After: A -one-to-many-> B -one-to-many-> C I have to assume that this rearrangement is what's causing the problem, but can't for the life of me work out why! The problem occurs running a procedural script from the command line, so I don't see how threading could be the problem. Traceback (most recent call last): ... File "/usr/lib/python2.4/site-packages/sqlobject/declarative.py", line 98, in _wrapper return fn(self, *args, **kwargs) File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1210, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1238, in _create self.set(**kw) File "<string>", line 1, in <lambda> File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 975, in _SO_loadValue selectResults = self._connection._SO_selectOne(self, dbNames) File "/usr/lib/python2.4/site-packages/sqlobject/dbconnection.py", line 652, in _SO_selectOne return self.queryOne( AttributeError: 'A' object has no attribute 'id' I can provide more details if anyone can suggest where I should start. (Have already spent considerable time in the debugger, but have been unable to work out what's failing.) ---------------------------------------------------------------------- Comment By: Oleg Broytmann (phd) Date: 2006-01-31 07:21 Message: Logged In: YES user_id=4799 Applied at the revision 1546. Thank you! ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2006-01-17 08:39 Message: Logged In: NO Ooops, typo: the patch is for declarative.py, not main.py ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=540674&aid=1407684&group_id=74338 |
From: SourceForge.net <no...@so...> - 2008-04-14 14:33:36
|
Patches item #1407684, was opened at 2006-01-17 00:35 Message generated for change (Comment added) made by phd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=540674&aid=1407684&group_id=74338 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed >Resolution: Invalid Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Oleg Broytmann (phd) Summary: object initialization threading issue Initial Comment: The creation of SQL objects is not threadsafe. After stressing a TurboGears app with ab2 with multiple concurrent connections to a threaded server, the log gets filled with tracebacks like these: 15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/_cp httptools.py", line 96, in _run applyFilters('before_main') File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/fil ters/__init__.py", line 131, in applyFilters method() File "/home/alberto/turbogears/turbogears/visit.py", line 99, in beforeMain visit=self.get_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 139, in get_visit return self.new_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 122, in new_visit visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout ) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1197, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1221, in _create self.set(**kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1142, in set self._connection._SO_update(self, args) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/dbconnection.py", line 561, in _SO_update self.query("UPDATE %s SET %s WHERE %s = %s" % AttributeError: 'TG_Visit' object has no attribute 'id' 15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/_cp httptools.py", line 96, in _run applyFilters('before_main') File "/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/ cherrypy/fil ters/__init__.py", line 131, in applyFilters method() File "/home/alberto/turbogears/turbogears/visit.py", line 99, in beforeMain visit=self.get_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 139, in get_visit return self.new_visit() File "/home/alberto/turbogears/turbogears/visit.py", line 122, in new_visit visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout ) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1197, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1224, in _create self._SO_finishCreate(id) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500- py2.4.egg/sqlobj ect/main.py", line 1242, in _SO_finishCreate del self.sqlmeta._creating AttributeError: _creating I've found at http://pythonpaste.org/archives/message/ 20051202.223718.a3adcfaa.en.html a solution to the problem, which is overriding __init__ on SQLObject subclasses by wrapping SQLObject.__init__ with a mutex acquisition and release. I've made a patch for main.py which generalizes this approach and wraps __init__ for every SQLObject created. Tested under my environment and works fine. The patch is also posted at the turbogears Trac at http:// trac.turbogears.org/turbogears/attachment/ticket/397/ sqlobject_threading.patch Cheers, Alberto Valverde al...@to... ---------------------------------------------------------------------- >Comment By: Oleg Broytmann (phd) Date: 2008-04-14 18:33 Message: Logged In: YES user_id=4799 Originator: NO I reverted the patch due to the bug report at http://groups.google.com/group/turbogears/msg/2b056e15ed3af886 ---------------------------------------------------------------------- Comment By: Jeremy Field (j3z) Date: 2007-12-22 06:07 Message: Logged In: YES user_id=932673 Originator: NO I think I am experiencing this or a similar problem in SQLObject 0.9.2. I've written a simple SQLObject subclass (A) within a fairly straightforward schema. It fails to instantiate, giving the error partially reproduced below. Interestingly, it has only started to fail since I added another SQLObject subclass (B) to the schema. This second subclass has a many-to-one relationship to the first, and was inserted between (A) and a third subclass (C) as follows: Before: A -one-to-many-> C After: A -one-to-many-> B -one-to-many-> C I have to assume that this rearrangement is what's causing the problem, but can't for the life of me work out why! The problem occurs running a procedural script from the command line, so I don't see how threading could be the problem. Traceback (most recent call last): ... File "/usr/lib/python2.4/site-packages/sqlobject/declarative.py", line 98, in _wrapper return fn(self, *args, **kwargs) File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1210, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1238, in _create self.set(**kw) File "<string>", line 1, in <lambda> File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 975, in _SO_loadValue selectResults = self._connection._SO_selectOne(self, dbNames) File "/usr/lib/python2.4/site-packages/sqlobject/dbconnection.py", line 652, in _SO_selectOne return self.queryOne( AttributeError: 'A' object has no attribute 'id' I can provide more details if anyone can suggest where I should start. (Have already spent considerable time in the debugger, but have been unable to work out what's failing.) ---------------------------------------------------------------------- Comment By: Oleg Broytmann (phd) Date: 2006-01-30 23:21 Message: Logged In: YES user_id=4799 Applied at the revision 1546. Thank you! ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2006-01-17 00:39 Message: Logged In: NO Ooops, typo: the patch is for declarative.py, not main.py ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=540674&aid=1407684&group_id=74338 |