[SQL-CVS] SQLObject development repository branch master updated. 3.10.2-4-g69c1c2f

[phd <no...@so...>]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "SQLObject development repository".

The branch, master has been updated
       via  69c1c2f36c2893f82b9270782eb802a9f96df589 (commit)
       via  ae1ca49def118ed7a0aa897e6613ef8563ef4d3f (commit)
      from  0d633b72642bace7da86a51470aca1044b82c808 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://sourceforge.net/p/sqlobject/sqlobject/ci/69c1c2f36c2893f82b9270782eb802a9f96df589

commit 69c1c2f36c2893f82b9270782eb802a9f96df589
Author: Oleg Broytman <ph...@ph...>
Date:   Mon Sep 18 19:36:16 2023 +0300

    Fix(SQLiteConnection): Release connections to fix memory leak
    
    Release connections from threads that are no longer active.
    This fixes memory leak in multithreaded programs in Windows.
    
    `SQLite` requires different connections per thread so
    `SQLiteConnection` creates and stores connections per thread.
    When a thread finishes its connections should be closed.
    But if a program doesn't cooperate and doesn't close connections at
    the end of a thread SQLObject leaks memory as connection objects are
    stuck in `SQLiteConnection`. On Linux the leak is negligible as
    Linux reuses thread IDs so new connections replaces old and old
    connections are garbage collected. But Windows doesn't reuse thread
    IDs so old connections pile and never released. To fix the problem
    `SQLiteConnection` now enumerate threads and release connections
    from non-existing threads.
    
    Fixes: #186.

diff --git a/docs/News.rst b/docs/News.rst
index f37f366..dd8a26a 100644
--- a/docs/News.rst
+++ b/docs/News.rst
@@ -5,9 +5,28 @@ News
 .. contents:: Contents:
    :backlinks: none
 
-SQLObject (master)
+SQLObject (masterdoesn't )
 ==================
 
+Drivers
+-------
+
+* Fix(SQLiteConnection): Release connections from threads that are
+  no longer active. This fixes memory leak in multithreaded programs
+  in Windows.
+
+  ``SQLite`` requires different connections per thread so
+  ``SQLiteConnection`` creates and stores connections per thread.
+  When a thread finishes its connections should be closed.
+  But if a program doesn't cooperate and doesn't close connections at
+  the end of a thread SQLObject leaks memory as connection objects are
+  stuck in ``SQLiteConnection``. On Linux the leak is negligible as
+  Linux reuses thread IDs so new connections replaces old and old
+  connections are garbage collected. But Windows doesn't reuse thread
+  IDs so old connections pile and never released. To fix the problem
+  ``SQLiteConnection`` now enumerate threads and release connections
+  from non-existing threads.
+
 CI
 --
 
diff --git a/sqlobject/sqlite/sqliteconnection.py b/sqlobject/sqlite/sqliteconnection.py
index 89635e3..47d0a9c 100644
--- a/sqlobject/sqlite/sqliteconnection.py
+++ b/sqlobject/sqlite/sqliteconnection.py
@@ -4,6 +4,7 @@ try:
     from _thread import get_ident
 except ImportError:
     from thread import get_ident
+from threading import enumerate as enumerate_threads
 try:
     from urllib import quote
 except ImportError:
@@ -150,6 +151,7 @@ class SQLiteConnection(DBAPI):
             self._connectionNumbers[id(conn)] = self._connectionCount
             self._connectionCount += 1
             return conn
+        self._releaseUnusedConnections()
         threadid = get_ident()
         if (self._pool is not None and threadid in self._threadPool):
             conn = self._threadPool[threadid]
@@ -213,6 +215,19 @@ class SQLiteConnection(DBAPI):
         conn.text_factory = str  # Convert text data to str, not unicode
         return conn
 
+    def _releaseUnusedConnections(self):
+        """Release connections from threads that're no longer active"""
+        thread_ids = set(t.ident for t in enumerate_threads())
+        tp_set = set(self._threadPool)
+        unused_connections = [
+            self._threadPool[tid] for tid in (tp_set - thread_ids)
+        ]
+        for unused_connection in unused_connections:
+            try:
+                self.releaseConnection(unused_connection, explicit=True)
+            except self.module.ProgrammingError:
+                pass  # Ignore error in `conn.close()` from a different thread
+
     def close(self):
         DBAPI.close(self)
         self._threadPool = {}

http://sourceforge.net/p/sqlobject/sqlobject/ci/ae1ca49def118ed7a0aa897e6613ef8563ef4d3f

commit ae1ca49def118ed7a0aa897e6613ef8563ef4d3f
Author: Oleg Broytman <ph...@ph...>
Date:   Mon Sep 18 02:29:36 2023 +0300

    Fix(SQLiteConnection): Clear `_threadOrigination`

diff --git a/sqlobject/sqlite/sqliteconnection.py b/sqlobject/sqlite/sqliteconnection.py
index 06cf3c4..89635e3 100644
--- a/sqlobject/sqlite/sqliteconnection.py
+++ b/sqlobject/sqlite/sqliteconnection.py
@@ -181,6 +181,8 @@ class SQLiteConnection(DBAPI):
         else:
             if self._pool and conn in self._pool:
                 self._pool.remove(conn)
+            if threadid:
+                del self._threadOrigination[id(conn)]
             conn.close()
 
     def _setAutoCommit(self, conn, auto):
@@ -214,6 +216,7 @@ class SQLiteConnection(DBAPI):
     def close(self):
         DBAPI.close(self)
         self._threadPool = {}
+        self._threadOrigination = {}
         if self._memory:
             self._memoryConn.close()
             self.makeMemoryConnection()

-----------------------------------------------------------------------

Summary of changes:
 docs/News.rst                        | 21 ++++++++++++++++++++-
 sqlobject/sqlite/sqliteconnection.py | 18 ++++++++++++++++++
 2 files changed, 38 insertions(+), 1 deletion(-)


hooks/post-receive
-- 
SQLObject development repository