|
From: Ian B. <ia...@co...> - 2004-03-02 23:32:25
|
Scott Russell wrote:
> David and Ian:
>
> Yes, (as you'll see in my mail that arrived the same time yours did) it
> looks like a thread concurrency issue.
>
> If I turn on debugging, the threading test survives many additional
> threads, but will die eventually. (David, 32 kills it very handily.)
> Whether because of some serialization that gets forced during printing,
> because it slows the process down enough to avoid contention, or some
> third issue, I can't say.
>
> Here is a sample run, with 32 threads. Sometimes it lasts for 30-40
> queries, sometimes (as seen here) only 3....
>
> $ python Main/TestPage.py
> 1:Dummy-1/Select : SELECT person.id, person.email, person.firstname, person.lastname FROM person WHERE (person.email = 'xxx') LIMIT 1
> 1:Dummy-2/Select : SELECT person.id, person.email, person.firstname, person.lastname FROM person WHERE (person.email = 'xxx') LIMIT 1
> 1:Dummy-3/Select : SELECT person.id, person.email, person.firstname, person.lastname FROM person WHERE (person.email = 'xxx') LIMIT 1
> Killed
>
> What next?
Well, it's reusing the same connection each time. Which it probably
shouldn't be -- it can reuse them sometimes, but you'd think it would
need to do otherwise sometimes.
Hmm...
Bah, DBConnection.iterSelect is dumb. It shouldn't be written like it
is. Right now it looks like:
def iterSelect(self, select):
return self._runWithConnection(self._iterSelect, select, self,
False)
Which means the connection is returned to the pool immediately when the
generator is returned, then reused when it shouldn't be, and worse it
can be returned to the pool another time if the select results are
exhausted. It should look like:
def iterSelect(self, select):
conn = self.getConnection()
return self._iterSelect(conn, select, self, False)
This isn't quite right either, as the connection will only be returned
if you exhaust the select. _iterSelect should be broken out into an
object that returns the connection on __del__. Anyway, this iterSelect
should resolve your concurrency problems for the moment (I haven't
tested it, though; report back if it works)
Ian
|
