Thread: [SQLObject] Thread Safety and SQLObject
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Brad B. <br...@bb...> - 2003-05-22 21:45:01
|
I have a Webware servlet that has code like this: pid = os.fork() if not pid: try: batch_file.process() except TooManyErrors, err: pass At some point in the code that batch_file.process() is calling, I instantiate one of my SQLObject-based classes which is mapped to a database table that acts as a logger for purchase transactions: conn = PostgresConnection( host='localhost', db='foo', user='bar', passwd='somepass' ) self.purchase_transaction = PurchaseTransaction.new( merchantProxyID = self.merchant_proxy_id, transactionID = self.txID, transactionDate = now(), amount = self.amount, productDesc = self.description, cardholderName = self.cardholder_name, firstTwoCCDigits = self.card_number[0:2], lastFourCCDigits = self.card_number[-4:], expireMonth = self.card_expire_month, expireYear = self.card_expire_year, emailAddress = self.email, connection = conn ) I'm doing this because I want to return control to the web browser whilst I go off and process lots of credit card transactions from the batch_file in the background. The problem is that if this batch_file.process() stuff is executing in the background and I continue to do database retrievals and such through the web, it's quite easy for me to create various kinds of exceptions; Error: no results to fetch, OperationalError: not connected to server, etc. The error messages aren't consistent, but what *is* consistent is that it's always dying on something to do with getting data from the database whilst this child process is executing in the background. It seems like the child and parent processes are trying to read from the same socket, with obvious trouble happening from there. Two questions then: 1. Why isn't connection = conn making this Do The Right Thing? I thought the semantics I've shown above would be creating a new DB connection every time, and thus hopefully have the child avoid interfering with the parent. 2. Is there any way to use SQLObject safely in a threaded environment like the os.fork'ing example I have above? -- Brad Bollenbach BBnet.ca |
From: Luke O. <lu...@me...> - 2003-05-22 22:03:52
|
> 1. Why isn't connection = conn making this Do The Right Thing? I > thought the semantics I've shown above would be creating a new DB > connection every time, and thus hopefully have the child avoid > interfering with the parent. I'm going to make a wild guess, and noting that CVS is still missing certain fixes that were pointed out when I last posted MemoryTransaction...: around line 683, in new(), add the following line: if kw.has_key('connection'): inst = cls(None, connection=kw['connection']) + inst._connection = kw['connection'] del kw['connection'] There are related bugs in RelatedJoin.add and RelatedJoin.remove, where they should use the connection from inst and not self.callingClass. > 2. Is there any way to use SQLObject safely in a threaded environment > like the os.fork'ing example I have above? So long as you don't have any of the potential concurrency problems of trying to update the same object from separate threads etc, I think the connection problem above will solve it. And even in those cases, SQLObject shouldn't blow up, you just don't know what data you'll end up with. :) - Luke |
From: Brad B. <br...@bb...> - 2003-05-23 01:58:49
|
On 05/22/03 16:49, Luke Opperman wrote: > > > 1. Why isn't connection = conn making this Do The Right Thing? I > > thought the semantics I've shown above would be creating a new DB > > connection every time, and thus hopefully have the child avoid > > interfering with the parent. > > I'm going to make a wild guess, and noting that CVS is still missing certain > fixes that were pointed out when I last posted MemoryTransaction...: > > around line 683, in new(), add the following line: > if kw.has_key('connection'): > inst = cls(None, connection=kw['connection']) > + inst._connection = kw['connection'] > del kw['connection'] > > There are related bugs in RelatedJoin.add and RelatedJoin.remove, where they > should use the connection from inst and not self.callingClass. > > > 2. Is there any way to use SQLObject safely in a threaded environment > > like the os.fork'ing example I have above? > > So long as you don't have any of the potential concurrency problems of trying > to update the same object from separate threads etc, I think the connection > problem above will solve it. And even in those cases, SQLObject shouldn't blow > up, you just don't know what data you'll end up with. :) Well, I tried your solution but it didn't fix my problem. This isn't because there was anything wrong with your solution, but more because I found other problems in my own code (e.g. I hadn't quite covered passing connections to *all* of my SQLObject-based classes being hit in the child process :/). For simplicty's sake, I'm doing this now: pid = os.fork() if not pid: # child process -- process the batch file whilst control # has been returned to the browser try: conn = PostgresConnection( host='localhost', db='foo', user='bar', passwd='baz' ) MerchantAccount._connection = conn MerchantProxyAccount._connection = conn MerchantUser._connection = conn PurchaseTransaction._connection = conn try: batch_file.process() except TooManyErrors, err: pass finally: os._exit(0) This also fixed another bug where the child would never return in my Webware servlet (three cheers for os._exit()! :). Thanks for helping me see what I was doing wrong. :) -- Brad Bollenbach BBnet.ca |