sqlobject-cvs Mailing List for SQLObject (Page 2)

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "SQLObject development repository".

The branch, master has been updated
       via  69c1c2f36c2893f82b9270782eb802a9f96df589 (commit)
       via  ae1ca49def118ed7a0aa897e6613ef8563ef4d3f (commit)
      from  0d633b72642bace7da86a51470aca1044b82c808 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://sourceforge.net/p/sqlobject/sqlobject/ci/69c1c2f36c2893f82b9270782eb802a9f96df589

commit 69c1c2f36c2893f82b9270782eb802a9f96df589
Author: Oleg Broytman <ph...@ph...>
Date:   Mon Sep 18 19:36:16 2023 +0300

    Fix(SQLiteConnection): Release connections to fix memory leak
    
    Release connections from threads that are no longer active.
    This fixes memory leak in multithreaded programs in Windows.
    
    `SQLite` requires different connections per thread so
    `SQLiteConnection` creates and stores connections per thread.
    When a thread finishes its connections should be closed.
    But if a program doesn't cooperate and doesn't close connections at
    the end of a thread SQLObject leaks memory as connection objects are
    stuck in `SQLiteConnection`. On Linux the leak is negligible as
    Linux reuses thread IDs so new connections replaces old and old
    connections are garbage collected. But Windows doesn't reuse thread
    IDs so old connections pile and never released. To fix the problem
    `SQLiteConnection` now enumerate threads and release connections
    from non-existing threads.
    
    Fixes: #186.

diff --git a/docs/News.rst b/docs/News.rst
index f37f366..dd8a26a 100644
--- a/docs/News.rst
+++ b/docs/News.rst
@@ -5,9 +5,28 @@ News
 .. contents:: Contents:
    :backlinks: none
 
-SQLObject (master)
+SQLObject (masterdoesn't )
 ==================
 
+Drivers
+-------
+
+* Fix(SQLiteConnection): Release connections from threads that are
+  no longer active. This fixes memory leak in multithreaded programs
+  in Windows.
+
+  ``SQLite`` requires different connections per thread so
+  ``SQLiteConnection`` creates and stores connections per thread.
+  When a thread finishes its connections should be closed.
+  But if a program doesn't cooperate and doesn't close connections at
+  the end of a thread SQLObject leaks memory as connection objects are
+  stuck in ``SQLiteConnection``. On Linux the leak is negligible as
+  Linux reuses thread IDs so new connections replaces old and old
+  connections are garbage collected. But Windows doesn't reuse thread
+  IDs so old connections pile and never released. To fix the problem
+  ``SQLiteConnection`` now enumerate threads and release connections
+  from non-existing threads.
+
 CI
 --
 
diff --git a/sqlobject/sqlite/sqliteconnection.py b/sqlobject/sqlite/sqliteconnection.py
index 89635e3..47d0a9c 100644
--- a/sqlobject/sqlite/sqliteconnection.py
+++ b/sqlobject/sqlite/sqliteconnection.py
@@ -4,6 +4,7 @@ try:
     from _thread import get_ident
 except ImportError:
     from thread import get_ident
+from threading import enumerate as enumerate_threads
 try:
     from urllib import quote
 except ImportError:
@@ -150,6 +151,7 @@ class SQLiteConnection(DBAPI):
             self._connectionNumbers[id(conn)] = self._connectionCount
             self._connectionCount += 1
             return conn
+        self._releaseUnusedConnections()
         threadid = get_ident()
         if (self._pool is not None and threadid in self._threadPool):
             conn = self._threadPool[threadid]
@@ -213,6 +215,19 @@ class SQLiteConnection(DBAPI):
         conn.text_factory = str  # Convert text data to str, not unicode
         return conn
 
+    def _releaseUnusedConnections(self):
+        """Release connections from threads that're no longer active"""
+        thread_ids = set(t.ident for t in enumerate_threads())
+        tp_set = set(self._threadPool)
+        unused_connections = [
+            self._threadPool[tid] for tid in (tp_set - thread_ids)
+        ]
+        for unused_connection in unused_connections:
+            try:
+                self.releaseConnection(unused_connection, explicit=True)
+            except self.module.ProgrammingError:
+                pass  # Ignore error in `conn.close()` from a different thread
+
     def close(self):
         DBAPI.close(self)
         self._threadPool = {}

http://sourceforge.net/p/sqlobject/sqlobject/ci/ae1ca49def118ed7a0aa897e6613ef8563ef4d3f

commit ae1ca49def118ed7a0aa897e6613ef8563ef4d3f
Author: Oleg Broytman <ph...@ph...>
Date:   Mon Sep 18 02:29:36 2023 +0300

    Fix(SQLiteConnection): Clear `_threadOrigination`

diff --git a/sqlobject/sqlite/sqliteconnection.py b/sqlobject/sqlite/sqliteconnection.py
index 06cf3c4..89635e3 100644
--- a/sqlobject/sqlite/sqliteconnection.py
+++ b/sqlobject/sqlite/sqliteconnection.py
@@ -181,6 +181,8 @@ class SQLiteConnection(DBAPI):
         else:
             if self._pool and conn in self._pool:
                 self._pool.remove(conn)
+            if threadid:
+                del self._threadOrigination[id(conn)]
             conn.close()
 
     def _setAutoCommit(self, conn, auto):
@@ -214,6 +216,7 @@ class SQLiteConnection(DBAPI):
     def close(self):
         DBAPI.close(self)
         self._threadPool = {}
+        self._threadOrigination = {}
         if self._memory:
             self._memoryConn.close()
             self.makeMemoryConnection()

-----------------------------------------------------------------------

Summary of changes:
 docs/News.rst                        | 21 ++++++++++++++++++++-
 sqlobject/sqlite/sqliteconnection.py | 18 ++++++++++++++++++
 2 files changed, 38 insertions(+), 1 deletion(-)


hooks/post-receive
-- 
SQLObject development repository




2003	Jan	Feb	Mar (9)	Apr (74)	May (29)	Jun (16)	Jul (28)	Aug (10)	Sep (57)	Oct (9)	Nov (29)	Dec (12)
2004	Jan (7)	Feb (14)	Mar (6)	Apr (3)	May (12)	Jun (34)	Jul (9)	Aug (29)	Sep (22)	Oct (2)	Nov (15)	Dec (52)
2005	Jan (47)	Feb (78)	Mar (14)	Apr (35)	May (33)	Jun (16)	Jul (26)	Aug (63)	Sep (40)	Oct (96)	Nov (96)	Dec (123)
2006	Jan (159)	Feb (144)	Mar (64)	Apr (31)	May (88)	Jun (48)	Jul (16)	Aug (64)	Sep (87)	Oct (92)	Nov (56)	Dec (76)
2007	Jan (94)	Feb (103)	Mar (126)	Apr (123)	May (85)	Jun (11)	Jul (130)	Aug (47)	Sep (65)	Oct (70)	Nov (12)	Dec (11)
2008	Jan (30)	Feb (55)	Mar (88)	Apr (20)	May (50)	Jun	Jul (38)	Aug (1)	Sep (9)	Oct (5)	Nov (6)	Dec (39)
2009	Jan (8)	Feb (16)	Mar (3)	Apr (33)	May (44)	Jun (1)	Jul (10)	Aug (33)	Sep (74)	Oct (22)	Nov	Dec (15)
2010	Jan (28)	Feb (22)	Mar (46)	Apr (29)	May (1)	Jun (1)	Jul (27)	Aug (8)	Sep (5)	Oct (33)	Nov (24)	Dec (41)
2011	Jan (4)	Feb (12)	Mar (35)	Apr (29)	May (19)	Jun (16)	Jul (32)	Aug (25)	Sep (5)	Oct (11)	Nov (21)	Dec (12)
2012	Jan (3)	Feb (4)	Mar (20)	Apr (4)	May (25)	Jun (13)	Jul	Aug	Sep (2)	Oct (25)	Nov (9)	Dec (1)
2013	Jan (6)	Feb (8)	Mar	Apr (10)	May (31)	Jun (7)	Jul (18)	Aug (33)	Sep (4)	Oct (16)	Nov	Dec (27)
2014	Jan (2)	Feb	Mar	Apr (11)	May (39)	Jun (8)	Jul (11)	Aug (4)	Sep	Oct (27)	Nov	Dec (71)
2015	Jan (17)	Feb (47)	Mar (33)	Apr	May	Jun (9)	Jul (7)	Aug	Sep	Oct	Nov	Dec (8)
2016	Jan (4)	Feb (4)	Mar	Apr	May (12)	Jun (7)	Jul (9)	Aug (31)	Sep (8)	Oct (3)	Nov (15)	Dec (1)
2017	Jan (13)	Feb (7)	Mar (14)	Apr (8)	May (10)	Jun (4)	Jul (2)	Aug (1)	Sep	Oct (8)	Nov (4)	Dec (5)
2018	Jan (2)	Feb (8)	Mar	Apr (4)	May	Jun (6)	Jul	Aug (1)	Sep	Oct	Nov (1)	Dec
2019	Jan (1)	Feb (16)	Mar (1)	Apr (3)	May (5)	Jun (1)	Jul	Aug	Sep (2)	Oct	Nov (1)	Dec (3)
2020	Jan	Feb	Mar	Apr (1)	May (1)	Jun	Jul	Aug (1)	Sep	Oct (2)	Nov	Dec (2)
2021	Jan	Feb (2)	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct (1)	Nov (1)	Dec
2022	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep (6)	Oct (1)	Nov (1)	Dec (4)
2023	Jan	Feb	Mar	Apr	May	Jun	Jul (1)	Aug (3)	Sep (2)	Oct (2)	Nov (4)	Dec
2024	Jan	Feb (2)	Mar	Apr	May	Jun	Jul (1)	Aug	Sep (1)	Oct	Nov	Dec (9)
2025	Jan	Feb (4)	Mar (2)	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec

sqlobject-cvs Mailing List for SQLObject (Page 2)

SQLObject is a Python ORM.

sqlobject-cvs — CVS notification