Re: [SQLObject] Re: Re: Concurrency issues...

[Ian B. <ia...@co...>]

I just wanted to reply to some of the things you brought up in general 
design terms.

On Mar 29, 2004, at 5:55 PM, Chris Gahan wrote:
> One issue that this whole debacle has brought to my attention, however,
> is the design of DBAPI. It's not very flexible, and I think it could be
> improved.

First, no arguments here.  Well, it's reasonably flexible for what it 
does, which is abstracting out database-specific issues.  It could use 
more factoring, including moving things like pooling into separate, 
generic modules.

> Right now, the pooling code (in the DBAPI class) and the database-
> abstraction layer (in the *Connection classes) are all bundled together
> in a single class, and I think they should be separated.
>
> Why don't we simplify the *Connection classes so that they're only
> responsible for creating connections to the DB and executing SQL
> commands. This would make it easier to handle exceptions (like
> "connection timed out", which currently kills your program if it
> happens). The connection object could catch that specific exception, 
> and
> try to re-establish the connection, allowing the calling function to
> continue on its merry way! :D
>
> Then, separating the pool out could be done by creating a
> "ConnectionFactory" class. It could be subclassed depending on the
> behaviour you wanted. To use it, you'd just create an instance of it 
> and
> assign it to SQLObject's _connection attribute.

Oh no, not factories!

> Default factories could be: SimpleConnectionFactory (which just 
> creates a
> new connection every time, and if too many connections are open, it'll
> wait until some connections got closed before returning new 
> connections),
> and of course, the PooledConnectionFactory.

Oh no, not subclasses!

> Then, SQLObject could also include (or people could write their own)
> special-purpose factories. For example: a factory that always returns 
> the
> same connection to a specific Webware servlet, effectively giving each
> servlet its own DB connection (you could implement it by instantiating
> the factory with the servlet's .__hash__() as a parameter, or 
> something).

Taking the particular issue that spawned this discussion, the problem 
was a bug, not a lack of potential for customization.  It can certainly 
be argued that the bug was more likely to exist because of the coupled 
nature of the code involved.  But the solution was to fix it, it 
definitely *wasn't* to make it possible to customize the code so as to 
avoid the bug.  And I think that's what would happen in this model, 
where people would be carrying around their own customizations to the 
code, each person with their own problematic corner cases.

I want SQLObject to work correctly without too much effort.  If there's 
weird issues in some case, I think SQLObject should try to do the right 
thing for everyone.  In most cases this is quite possible.  If it's not 
possible, we should try to look for the sensible set of choices that we 
should support -- not every possible choice, but just the ones that 
have real use cases.  Then we should distill that into some sort of 
option, something that can be easily explained and documented, 
including descriptions of when you will care about the option and how 
you should choose.  I don't want to make SQLObject flexible enough that 
you simply give the user the tools to fix the problem on their own -- I 
think that's burdensome.  (It brings to mind this article: 
http://www.linuxworld.com/story/44251.htm)

And of course, the typical option should be the one that requires no 
thought at all from the user -- i.e., the default.  Unless the typical 
option would be broken for some situation, particularly broken in a way 
that causes bad bugs (e.g., lost data, incorrect data, concurrency 
issues, etc) -- in that case we have to force the user to make an 
explicit choice, and do what we can to make that bug more explicit 
(which might just mean putting in an assert statement).

So from this perspective, I'm definitely opposed to encouraging lots of 
subclasses and factories.  I am enthusiastic about robust, explicit, 
documented modules (just documented in the docstring, nothing fancy), 
and I'm very open to factoring classes like DBAPI into several modules.

That still leaves open the issue of passing in parameters to these 
'hidden' modules and classes, such as if you want to control the pool, 
or one of the caches, which are usually created by SQLObject on behalf 
of the user.  I'm open to suggestions -- but the current situation 
where the SQLObject or DBConnection class has to know what options to 
pass through isn't that bad (even though it does imply some coupling, 
in that everytime you add an option to the cache or pool object you'll 
have to update the SQLObject or DBConnection class).

--
Ian Bicking | ia...@co... | http://blog.ianbicking.org