#12 Cyclic references leading to memory/connection leak

MySQLdb
closed
Andy Dustman
MySQLdb (53)
5
2012-09-19
2003-11-03
No

http://c0re.23.nu/c0de/misc/MySQL-python-0.9.2-
cyclicref.patch

MySQLdb connection Objects contain cyclic references which
keep the connection objectfrom beeing closed and dealloced
by pythons reference counting garbage collector at function
exit. The connection object contains support for the cycle
detecting garbage collector but it takes an unknowen tome
untill the cycle detecting garbage collector kicks in which
means you have an unusable but still open connection to
your database for an undefined time. This can be avoided by
structuring your code carefully to allways call
connection.close() before returning from a function but doing
so is very unpythonic.

This patch migrates the problem by changing the cyclic
references to weak references which allow reference
counting to work as desired. Probably restructuring of the
classes to avoid cyclic references would be a better solution.

Code demonstrating the problem:

import MySQLdb

def userPassOk(user, passwd):
db = MySQLdb.connect(...)
c = db.cursor()
c.execute("select USER_NAME, USER_PASSWORD from
AV_USER WHERE USER_NAME = '%s'" % user)
print c.fetchall()
c.close()
#db.close()

if name == 'main':
for i in range(20000):
print i,
userPassOk(user="guest", passwd='guest')

--- /usr/local/lib/python2.3/site-packages/MySQLdb/
connections.py.orig Mon Nov 3 23:46:07 2003
+++ /usr/local/lib/python2.3/site-packages/MySQLdb/
connections.py Mon Nov 3 23:56:22 2003
@@ -12,6 +12,7 @@
NotSupportedError, ProgrammingError
import types, _mysql

+from weakref import proxy

def defaulterrorhandler(connection, cursor, errorclass,
errorvalue):
"""
@@ -113,9 +114,9 @@
conv[FIELD_TYPE.STRING] = u
conv[FIELD_TYPE.VAR_STRING] = u
self._make_connection(args, kwargs2)
- self.converter[types.StringType] = self.string_literal
+ self.converter[types.StringType] =
proxy(self.string_literalPY)
if hasattr(types, 'UnicodeType'):
- self.converter[types.UnicodeType] =
self.unicode_literal
+ self.converter[types.UnicodeType] =
proxy(self.unicode_literal)
self._transactional = self.server_capabilities &
CLIENT.TRANSACTIONS
self.messages = []

@@ -164,6 +165,10 @@

     """
     return self.escape(o, self.converter)

+
+ def string_literalPY(self, obj):
+ """use string_literal(obj) instead!"""
+ self.string_literal(obj)

 def unicode_literal(self, u, dummy=None):
     """

Discussion

  • Andy Dustman
    Andy Dustman
    2003-11-24

    Logged In: YES
    user_id=71372

    Your patch will not be applied.

     
  • Andy Dustman
    Andy Dustman
    2003-11-24

    Logged In: YES
    user_id=71372

    Weak references don't show up until Python 2.1, and I still
    want to support Python 1.5.2 for version 0.9/1.0.

     
  • Andy Dustman
    Andy Dustman
    2004-06-22

    Logged In: YES
    user_id=71372

    This (or a variant) will probably be in 1.1.2.

     
  • Andy Dustman
    Andy Dustman
    2005-01-23

    Logged In: YES
    user_id=71372

    There's a bug open for this