Christoph Zwerschke said:
> I haven=92t checked out DbConnectionPool yet. Which status has it?
> Warren, are you still working on it? Are you/somebody else interested
> in improving/testing this module and getting it into Webware, e.g.
> MiscUtils? If yes, I=92d like to contribute if I can.
I haven't done any work on it for quite a while, and the last work I did
never made it into the sandbox.
IIRC, I did a significant refactor of the code to extract the basic
pooling logic into a super-class so that I could use it to build pools of
other things besides database connections.
While I was at it, I extracted the core expiration logic from the
expiration thread so that it could be called by the "check in" code. Thi=
made it possible to cleanly make "aggressive" expiration non-optional wit=
a minimal performance penalty. I also implemented usageCount expiration.
The net effect of the expiration changes was to change the interface of
startExpiration(), replacing the "aggressiveExpiration" boolean with an
argument to specify the usageCount for usage based expiration.
These changes have NOT been used in a production environment, and I
haven't had time to develop a proper testing suite.
I'll try to get the latest changes into the sandbox. I would welcome any
help that you or anyone else could give in testing and further
I would like to see DbConnectionPool included in WebKit, at least as an
alternative to DBPool.
> For the time being I have only tested DBPool.py. Here are my
> * The module docstring talks about =93DB API=94. I think this should
> be specified to =93DB API 2=94, since DB API version 1 has no
> threadsafety attribute and thus will not work with DBPool.
> * There is an irritating comment that says:
> # These functions are used with DB modules
> # that have connection level threadsafety, like PostgreSQL:
> def _unthreadsafe_addConnection(self, con):
> - Shouldn=92t it rather say =93... that *DO NOT* have connection lev=
> threadsafety...=94? (threadsafety 0 =3D not threadsafe,
> 1 =3D module level, 2 =3D connection level, 3 =3D cursor level)
> - PostgreSQL is not a DB module, but a DBMS, so shouldn=92t it say
> =93like PyGreSQL=94 or something (another PostGreSQL interface,
> psycopg, has threadsafety=3D2)?
> * Wouldn=92t it be good to have minConnections and startConnections
> in addition to maxConnections? This would be analogous to the
> parameters in AppServer.config.
> * If a database connection crashes or gets disconnected, it will
> never be restarted. Wouldn=92t it be good to let the getConnection
> method check whether the connection is still usable and reopen
> or restart it in case it is not?
> * My last important question: Maybe I understand something wrong,
> but if I use DBPool for WebKit servlets, i.e. with the
> ThreadedAppServer, wouldn=92t it be much more reasonable to
> automatically bind the connections 1:1 to the servlet threads
> created by the AppServer, instead of making an independent
> database connection pool? So each thread would have it=92s own
> private database connection, servlets could never block each
> other, there would be no threadsafety issues at all,
> and you wouldn=92t need to configure any pool size,
> but there would be always just as many connections as stated
> in the AppServer.config. As far as I see it, this would have
> only advantages. Do I overlook anything?
> Could this be implemented with Webware/WebKit out of the box?
> If not, shouldn=92t such a feature be added?
I share many of these concerns with DBPool. The one about "bad"
connections being left in the pool is the one in particular that drove me
to write DbConnectionPool, as I felt this issue was at least partly to
blame for the "hung thread" issues that several were having.
As far as the need to have a connection pool at all, you are right. Ther=
is probably a way for WebKit to handle this issue internally and therefor=
sidestep some of the thread safety issues. Some of my thoughts on this
1. Because of the way WebKit handles servlets and threads (a given
instance of a servlet may be called by multiple threads during its
lifetime, though only one at any given time), a connection stored as a
servlet instance variable could still be called by more than one thread,
dictating some thread safety within the database interface code. To
mitigate this, the connection could be stored with the thread somehow.
However, I don't think even this will protect extension code that is not
at least minimally thread safe.
2. The problem of having a stored reference to a "bad" connection still
exists, so some mechanism for dealing with these would still need to be
implemented. In DbConnectionPool, I couldn't think of any good way of
finding out if a connection was "bad", so I chose to implement the
"expiration" concept as somewhat of a workaround. Although it doesn't
directly address a "bad" connection, it does address one of the most
common reasons why connections become unusable.
3. It would be preferable for the internal implementation to be able to
manage other thread-specific resources in addition to database
In the past, Ian Bicking had expressed some interest in modifications to
DbConnectionPool that would address thread safety issues in extension
modules. He is probably much more familiar with what those issues are
than I am.
Also, Ian's latest work on Paste might dovetail with this, as it is an
issue that is somewhat framework neutral.
Ian, if you are reading this, would you care to chime in?