[SQL-CVS] r4030 - in SQLObject/trunk: docs sqlobject

[<sub...@co...>]

Author: phd
Date: 2009-10-09 09:28:09 -0600 (Fri, 09 Oct 2009)
New Revision: 4030

Modified:
   SQLObject/trunk/docs/News.txt
   SQLObject/trunk/sqlobject/cache.py
Log:
The cache culling algorithm was enhanced by Jason Culverhouse <ja...@me...>.


Modified: SQLObject/trunk/docs/News.txt
===================================================================
--- SQLObject/trunk/docs/News.txt	2009-10-09 15:26:30 UTC (rev 4029)
+++ SQLObject/trunk/docs/News.txt	2009-10-09 15:28:09 UTC (rev 4030)
@@ -55,6 +55,11 @@
 
 * Removed the last remained string exceptions.
 
+SQLObject 0.11.3
+================
+
+* A change ported from `SQLObject 0.10.9`_.
+
 SQLObject 0.11.2
 ================
 
@@ -117,6 +122,13 @@
 * Changed the order of testing of SQLite modules - look for external
   PySQLite2 before sqlite3.
 
+SQLObject 0.10.9
+================
+
+* The cache culling algorithm was enhanced to eliminate memory leaks by
+  removing references to dead objects; tested on a website that runs around
+  4 million requests a day.
+
 SQLObject 0.10.8
 ================
 

Modified: SQLObject/trunk/sqlobject/cache.py
===================================================================
--- SQLObject/trunk/sqlobject/cache.py	2009-10-09 15:26:30 UTC (rev 4029)
+++ SQLObject/trunk/sqlobject/cache.py	2009-10-09 15:28:09 UTC (rev 4030)
@@ -180,18 +180,31 @@
             self.expiredCache[id] = ref(obj)
 
     def cull(self):
-        """
-        Runs through the cache and expires objects.  E.g., if
-        ``cullFraction`` is 3, then every third object is moved to
+        """Runs through the cache and expires objects
+
+        E.g., if ``cullFraction`` is 3, then every third object is moved to
         the 'expired' (aka weakref) cache.
+
         """
         self.lock.acquire()
         try:
+            #remove dead references from the expired cache
+            keys = self.expiredCache.keys()
+            for key in keys:
+                if self.expiredCache[key]() is None:
+                    self.expiredCache.pop(key, None)
+
             keys = self.cache.keys()
             for i in xrange(self.cullOffset, len(keys), self.cullFraction):
                 id = keys[i]
-                self.expiredCache[id] = ref(self.cache[id])
+                # create a weakref, then remove from the cache
+                obj = ref(self.cache[id])
                 del self.cache[id]
+
+                #the object may have been gc'd when removed from the cache
+                #above, no need to place in expiredCache
+                if obj() is not None:
+                    self.expiredCache[id] = obj
             # This offset tries to balance out which objects we
             # expire, so no object will just hang out in the cache
             # forever.