#37 object initialization threading issue

closed-invalid
Oleg Broytman
None
5
2008-04-14
2006-01-16
Anonymous
No

The creation of SQL objects is not threadsafe. After stressing a
TurboGears app with ab2 with multiple concurrent connections to a
threaded server, the log gets filled with tracebacks like these:

15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last):
File
"/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/
cherrypy/_cp httptools.py",
line 96, in _run
applyFilters('before_main')
File
"/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/
cherrypy/fil ters/__init__.py",
line 131, in applyFilters
method()
File "/home/alberto/turbogears/turbogears/visit.py", line 99, in
beforeMain
visit=self.get_visit()
File "/home/alberto/turbogears/turbogears/visit.py", line 139, in
get_visit
return self.new_visit()
File "/home/alberto/turbogears/turbogears/visit.py", line 122, in
new_visit
visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout )
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1197, in __init__
self._create(id, **kw)
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1221, in _create
self.set(**kw)
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1142, in set
self._connection._SO_update(self, args)
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/dbconnection.py",
line 561, in _SO_update
self.query("UPDATE %s SET %s WHERE %s = %s" %
AttributeError: 'TG_Visit' object has no attribute 'id'

15/Jan/2006:22:05:00 HTTP INFO Traceback (most recent call last):
File
"/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/
cherrypy/_cp httptools.py",
line 96, in _run
applyFilters('before_main')
File
"/usr/lib/python2.4/site-packages/CherryPy-2.2.0beta-py2.4.egg/
cherrypy/fil ters/__init__.py",
line 131, in applyFilters
method()
File "/home/alberto/turbogears/turbogears/visit.py", line 99, in
beforeMain
visit=self.get_visit()
File "/home/alberto/turbogears/turbogears/visit.py", line 139, in
get_visit
return self.new_visit()
File "/home/alberto/turbogears/turbogears/visit.py", line 122, in
new_visit
visit= TG_Visit( visit_key=visit_key, expiry=now+self.timeout )
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1197, in __init__
self._create(id, **kw)
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1224, in _create
self._SO_finishCreate(id)
File
"/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1500-
py2.4.egg/sqlobj ect/main.py",
line 1242, in _SO_finishCreate
del self.sqlmeta._creating
AttributeError: _creating

I've found at http://pythonpaste.org/archives/message/
20051202.223718.a3adcfaa.en.html a solution to the problem, which
is overriding __init__ on SQLObject subclasses by wrapping
SQLObject.__init__ with a mutex acquisition and release.

I've made a patch for main.py which generalizes this approach and
wraps __init__ for every SQLObject created. Tested under my
environment and works fine.

The patch is also posted at the turbogears Trac at http://
trac.turbogears.org/turbogears/attachment/ticket/397/
sqlobject_threading.patch

Cheers,
Alberto Valverde
alberto@toscat.net

Discussion

  • a SVN patch for sqlobject/main.py

     
  • Logged In: NO

    Ooops, typo: the patch is for declarative.py, not main.py

     
  • Oleg Broytman
    Oleg Broytman
    2006-01-30

    Logged In: YES
    user_id=4799

    Applied at the revision 1546. Thank you!

     
  • Oleg Broytman
    Oleg Broytman
    2006-01-30

    • assigned_to: nobody --> phd
    • status: open --> closed-accepted
     
  • Jeremy Field
    Jeremy Field
    2007-12-22

    Logged In: YES
    user_id=932673
    Originator: NO

    I think I am experiencing this or a similar problem in SQLObject 0.9.2.

    I've written a simple SQLObject subclass (A) within a fairly straightforward schema. It fails to instantiate, giving the error partially reproduced below. Interestingly, it has only started to fail since I added another SQLObject subclass (B) to the schema. This second subclass has a many-to-one relationship to the first, and was inserted between (A) and a third subclass (C) as follows:

    Before: A -one-to-many-> C
    After: A -one-to-many-> B -one-to-many-> C

    I have to assume that this rearrangement is what's causing the problem, but can't for the life of me work out why! The problem occurs running a procedural script from the command line, so I don't see how threading could be the problem.

    Traceback (most recent call last):
    ...
    File "/usr/lib/python2.4/site-packages/sqlobject/declarative.py", line 98, in _wrapper
    return fn(self, *args, **kwargs)
    File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1210, in __init__
    self._create(id, **kw)
    File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 1238, in _create
    self.set(**kw)
    File "<string>", line 1, in <lambda>
    File "/usr/lib/python2.4/site-packages/sqlobject/main.py", line 975, in _SO_loadValue
    selectResults = self._connection._SO_selectOne(self, dbNames)
    File "/usr/lib/python2.4/site-packages/sqlobject/dbconnection.py", line 652, in _SO_selectOne
    return self.queryOne(
    AttributeError: 'A' object has no attribute 'id'

    I can provide more details if anyone can suggest where I should start. (Have already spent considerable time in the debugger, but have been unable to work out what's failing.)

     
  • Oleg Broytman
    Oleg Broytman
    2008-04-14

    • status: closed-accepted --> closed-invalid