Thread: [SQLObject] error while using threads
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Randall S. <ra...@tn...> - 2005-12-02 22:40:54
|
I've been doing some testing with threads. After repeatedly running my test program, I eventually get an error like the one below. I would guess it's two threads writing to the same resource. Any ideas what is causing this? The test program at the end of this message. -Randall Exception in thread 14: Traceback (most recent call last): File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap self.run() File "threadtest.py", line 29, in run lastName='Smith' File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1183, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1210, in _create self._SO_finishCreate(id) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1228, in _SO_finishCreate del self.sqlmeta._creating AttributeError: _creating # Test Program import time import threading from sqlobject import * cns="postgresql://randall:arnold1@localhost/test" con = connectionForURI(cns) threads = [] class Person(SQLObject): firstName = StringCol() middleInitial = StringCol(length=1, default=None) lastName = StringCol() Person.createTable(ifNotExists=True, connection=con) person = Person(firstName='Randall', middleInitial='B', lastName='Smith', connection=con ) print person class MyThread(threading.Thread): def run(self): sqlhub.threadConnection = con person = Person(firstName='Randall', middleInitial='B', lastName='Smith' ) print person ##print self.getName(), sqlhub.getConnection() for threadname in range(20): threads.append(MyThread(name=threadname + 1).start()) |
From: Randall S. <ra...@tn...> - 2005-12-02 22:54:57
|
Randall Smith wrote: > I've been doing some testing with threads. After repeatedly running my > test program, I eventually get an error like the one below. I would > guess it's two threads writing to the same resource. Any ideas what is > causing this? The test program at the end of this message. -Randall > > Exception in thread 14: > Traceback (most recent call last): > File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap > self.run() > File "threadtest.py", line 29, in run > lastName='Smith' > File > "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", > line 1183, in __init__ > self._create(id, **kw) > File > "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", > line 1210, in _create > self._SO_finishCreate(id) > File > "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", > line 1228, in _SO_finishCreate > del self.sqlmeta._creating > AttributeError: _creating > > # Test Program > > import time > import threading > from sqlobject import * > > cns="postgresql://randall:arnold1@localhost/test" > con = connectionForURI(cns) > > threads = [] > > class Person(SQLObject): > firstName = StringCol() > middleInitial = StringCol(length=1, default=None) > lastName = StringCol() > > Person.createTable(ifNotExists=True, connection=con) > > person = Person(firstName='Randall', > middleInitial='B', > lastName='Smith', > connection=con > ) > print person > > class MyThread(threading.Thread): > def run(self): > sqlhub.threadConnection = con > person = Person(firstName='Randall', > middleInitial='B', > lastName='Smith' > ) > print person > ##print self.getName(), sqlhub.getConnection() > > for threadname in range(20): > threads.append(MyThread(name=threadname + 1).start()) > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click Using a lock seems to fix it. I thought SQLObject was threadsafe already. I must be missing something. lock = threading.RLock() class Person(SQLObject): def __init__(self, *args, **kwargs): lock.acquire() return_value = SQLObject.__init__(self, *args, **kwargs) lock.release() return return_value firstName = StringCol() middleInitial = StringCol(length=1, default=None) lastName = StringCol() |
From: Randall S. <ra...@tn...> - 2005-12-04 05:11:27
|
I've been testing SQLObject in a threaded environment and found problems. I decided to test it on webkit, because that's what I plan to deploy my app on. webkit is a threaded app server, so I expected problems and I got them. Below is a webkit servlet and the error output. Being typical of thread problems, this error shows up randomly. Unless someone has some better advice, I'm going to start placing thread locks in my code. At this time I'm not familiar enough with the SQLObject code to make it thread safe, but I'll gladly contribute any solutions I find. --Randall from WebKit.Page import Page from sqloclasses import Person from sqlobject import * import threading import time import os cns="postgresql://myname:mypass@localhost/test" con = connectionForURI(cns) class RTest(Page): def setUpCon(self): print len(con._pool) sqlhub.threadConnection = con def writeHTML(self): self.setUpCon() person = Person(firstName='first', middleInitial='middle', lastName='last' ) self.writeln('Hello\n') first_name = person.firstName #self.writeln(str(person.firstName)) req = self.request() #self.writeln('Hello\n') I wrote a threaded app using urllib2 to test this out, hitting it repeatedly until an error occurred. Below is the error. [Sat Dec 3 22:52:43 2005] [error] WebKit: Error while executing script /home/randall/downloads/tarballs/Webware-0.9/WebKit/Examples/RTest.py Traceback (most recent call last): File "./WebKit/Application.py", line 466, in dispatchRawRequest File "./WebKit/Application.py", line 529, in runTransaction File "./WebKit/Application.py", line 553, in runTransactionViaServlet File "./WebKit/Servlet.py", line 59, in runTransaction File "./WebKit/Transaction.py", line 136, in respond File "./WebKit/HTTPServlet.py", line 67, in respond File "./WebKit/HTTPContent.py", line 63, in respondToGet File "./WebKit/HTTPContent.py", line 105, in _respond File "./WebKit/Page.py", line 30, in defaultAction File "/home/randall/downloads/tarballs/Webware-0.9/WebKit/Examples/RTest.py", line 22, in writeHTML lastName='Smith' File "/home/randall/downloads/tarballs/Webware-0.9/WebKit/Examples/sqloclasses.py", line 5, in __init__ return_value = SQLObject.__init__(self, *args, **kwargs) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1183, in __init__ self._create(id, **kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1207, in _create self.set(**kw) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/main.py", line 1128, in set self._connection._SO_update(self, args) File "/usr/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1330-py2.4.egg/sqlobject/dbconnection.py", line 561, in _SO_update self.query("UPDATE %s SET %s WHERE %s = %s" % AttributeError: 'Person' object has no attribute 'id' |
From: Randall S. <ra...@tn...> - 2006-01-01 06:43:22
|
Happy new year! This references my messages on Dec 12 and 13. I didn't get a response, so I'm assuming SQLObject is still not thread safe. I noticed that TurboGears uses CherryPy and SQLObject. CherryPy is multi-threaded, so I suspect users have or will experience occasional thread conflicts, which concerns me since it is becoming more popular. I haven't thought much about a solution, but it has been on my mind more as I am coming close to deploying my first app using SQLObject on webkit. I'll probably take the safest route and use high level locks acquired on object writes (creation and updating). The particular problem I saw in testing was manipulation of an sqlmeta attribute while creating a new instance. For those of you most familiar with SQLObject, what are your thread unsafe suspects? Anything outside of sqlmeta? How do you suggest it be made thread safe? If I've made a wrong assumption of observation, my apologies. Eat your black eyed peas today for good luck. Randall |
From: Kevin D. <da...@gm...> - 2006-01-01 14:43:05
|
Hi Randall, I don't remember the messages that you're referring to, and I didn't see them in the annoying archives over at Sourceforge (nor did I see a message on the TurboGears list). SQLObject *is* supposed to be threadsafe and I haven't seen any particular problem reports that appeared to be related to threadsafety. Can you retierate what it was you were doing to sqlmeta (or point to the messages)? sqlmeta is fairly new, and you may have a use case that wasn't considered when it was created... Kevin On 1/1/06, Randall Smith <ra...@tn...> wrote: > Happy new year! This references my messages on Dec 12 and 13. I didn't > get a response, so I'm assuming SQLObject is still not thread safe. I > noticed that TurboGears uses CherryPy and SQLObject. CherryPy is > multi-threaded, so I suspect users have or will experience occasional > thread conflicts, which concerns me since it is becoming more popular. > > I haven't thought much about a solution, but it has been on my mind more > as I am coming close to deploying my first app using SQLObject on > webkit. I'll probably take the safest route and use high level locks > acquired on object writes (creation and updating). The particular > problem I saw in testing was manipulation of an sqlmeta attribute while > creating a new instance. > > For those of you most familiar with SQLObject, what are your thread > unsafe suspects? Anything outside of sqlmeta? How do you suggest it be > made thread safe? If I've made a wrong assumption of observation, my > apologies. Eat your black eyed peas today for good luck. > > Randall > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > -- Kevin Dangoor Author of the Zesty News RSS newsreader email: ki...@bl... company: http://www.BlazingThings.com blog: http://www.BlueSkyOnMars.com |
From: Randall S. <ra...@tn...> - 2006-01-02 04:19:54
|
Randall Smith wrote: > Happy new year! This references my messages on Dec 12 and 13. I didn't > get a response, so I'm assuming SQLObject is still not thread safe. I > noticed that TurboGears uses CherryPy and SQLObject. CherryPy is > multi-threaded, so I suspect users have or will experience occasional > thread conflicts, which concerns me since it is becoming more popular. > > I haven't thought much about a solution, but it has been on my mind more > as I am coming close to deploying my first app using SQLObject on > webkit. I'll probably take the safest route and use high level locks > acquired on object writes (creation and updating). The particular > problem I saw in testing was manipulation of an sqlmeta attribute while > creating a new instance. > > For those of you most familiar with SQLObject, what are your thread > unsafe suspects? Anything outside of sqlmeta? How do you suggest it be > made thread safe? If I've made a wrong assumption of observation, my > apologies. Eat your black eyed peas today for good luck. > > Randall > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click Typo. Original messages were on 2005-12-02 and 2005-12-03. Randall |