Thread: [SQLObject] bug: connection/transaction leak?
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Guenther S. <gs...@sy...> - 2003-12-17 21:24:19
|
=2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, i noticed that when using sqlobject together with zope after some time more and more connections are open to my postgresql server. first i thought that it was a zope bug (i am binding sqlobject transactions= to zope transactions and assumed, that the transaction objects were referenced from somewhere in the ZODB), but now it seems that it is a bug in sqlobject. =20 =2D -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- from SQLObject import * import time class Account(SQLObject): ___ name =3D StringCol(alternateID =3D True) ___ password =3D StringCol() = =20 conn =3D PostgresConnection('user=3Dzope dbname=3DWebspaceNG', autoCommit = =3D False) = =20 while 1: ___ trans =3D conn.transaction() ___ b =3D Account.new(name =3D "FOO" + str(time.time()), password =3D "bar"= , connection =3D trans) ___ trans.commit() ___ raw_input() =2D -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- if you run this code, with every iteration an additional connection gets opened to the postgresql server (until the max. connection limit is reached= & postgresql refuses new connections). cu /gst =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/4Ml1ZtF7I/+gjcERAkT8AJ0bvPL8+SoglxVTWbVrAdPmO/qRUACZAc9y lAIFFBN8XaQRxA0btc4E8iM=3D =3DXoeD =2D----END PGP SIGNATURE----- |
From: Ian B. <ia...@co...> - 2003-12-17 23:50:05
|
On Dec 17, 2003, at 3:24 PM, Guenther Starnberger wrote: > from SQLObject import * > import time > > class Account(SQLObject): > ___ name = StringCol(alternateID = True) > ___ password = StringCol() > > conn = PostgresConnection('user=zope dbname=WebspaceNG', autoCommit = > False) > > while 1: > ___ trans = conn.transaction() > ___ b = Account.new(name = "FOO" + str(time.time()), password = "bar", > connection = trans) > ___ trans.commit() > ___ raw_input() > - -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- > > if you run this code, with every iteration an additional connection > gets > opened to the postgresql server (until the max. connection limit is > reached & > postgresql refuses new connections). Hmm.. that shouldn't happen. Though looking at the code, I don't do a whole lot to return connections. The transaction should get garbage collected, and __del__ releases the connection, but I can imagine there'd be lots of reasons a transaction wouldn't get collected (though in this case it looks like it should be). Probably on commit()/rollback() the transaction should become invalid, and the connection should be returned. Or maybe you could run trans.begin() to revalidate it (at which point it'd grab another connection). -- Ian Bicking | ia...@co... | http://blog.ianbicking.org |
From: Guenther S. <gs...@sy...> - 2003-12-19 20:46:22
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 18 December 2003 00:50, you wrote: hi, > Hmm.. that shouldn't happen. Though looking at the code, I don't do a > whole lot to return connections. The transaction should get garbage > collected, and __del__ releases the connection, but I can imagine > there'd be lots of reasons a transaction wouldn't get collected (though > in this case it looks like it should be). > > Probably on commit()/rollback() the transaction should become invalid, > and the connection should be returned. Or maybe you could run > trans.begin() to revalidate it (at which point it'd grab another > connection). using the following while block: while 1: ___ b = Account.new(name = "FOO" + str(time.time()), password = "bar", connection = conn) ___ print sys.getrefcount(conn) ___ del(b) shows an increasing refcount of conn with every iteration. maybe the leak is somewhere in the sqlobject class? cu /gst -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/42OUZtF7I/+gjcERAqPsAKCuapSeoG3udyvRrytIRu67bqxRzACgzO5F FblDKu3mUSZvZ0RNtfO57Yo= =IX1s -----END PGP SIGNATURE----- |
From: Ian B. <ia...@co...> - 2003-12-19 21:04:59
|
> using the following while block: > > while 1: > ___ b = Account.new(name = "FOO" + str(time.time()), password = "bar", > connection = conn) > ___ print sys.getrefcount(conn) > ___ del(b) > > shows an increasing refcount of conn with every iteration. maybe the > leak > is somewhere in the sqlobject class? Each new Account is going to have a reference to the connection, so that's to be expected. Hmm... and the accounts won't be deleted because they are cached. That's the problem, then. Maybe the transaction simply needs to clear its cache on a commit/rollback (well, you can't really clear the cache, but you can make all the objects GCable). -- Ian Bicking | ia...@co... | http://blog.ianbicking.org |
From: Guenther S. <gs...@sy...> - 2003-12-20 15:07:19
|
=2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday 19 December 2003 22:05, Ian Bicking wrote: hi, > Each new Account is going to have a reference to the connection, so > that's to be expected. Hmm... and the accounts won't be deleted > because they are cached. That's the problem, then. Maybe the > transaction simply needs to clear its cache on a commit/rollback (well, > you can't really clear the cache, but you can make all the objects > GCable). hmm.. shouldn't objects in the cache automatically expire 'faster' when i = set=20 the cullFrequency to a lower value? i tried setting the cullFrequency to 5= =20 and running gc.collect() after every iteration, but the refcount to the=20 connection still stays the same (resp. +1 at every iteration). on the other hand when i disable caching by passing "cache =3D False" to th= e=20 connection & run the gc it works fine (the refcount to 'conn' doesn't=20 increase). cu /gst =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/5GWeZtF7I/+gjcERAva+AJ9H/M2TjDu4GeP+ypEreDrud/zOVQCgiNdp hZ13/zMOlWTM+0/bkJzXYHA=3D =3D3rEe =2D----END PGP SIGNATURE----- |