Menu
▾
▴
[SQLObject] Concurrency issues...
|
From: Chris G. <ch...@il...> - 2004-03-21 12:18:49
|
Hey there. I'm having a big problem, and I need to find away around it,
otherwise I don't think I can use SQLObject. :)
It's a problem that was brought up a couple weeks ago. If I hammer SQLObject
with multiple threads all doing queries simultaneously, it'll either bomb
out by giving me an exception, or permanently lock up one of the threads,
and sometimes the entire python interpreter.
The problem was supposedly fixed with the new Iteration() class, but I don't
think it got to the root of the problem.
This weirdness happens with Postgres and MySQL drivers (the exceptions they
throw are slightly different, but their meanings are similar). It also
happens in SQLObject 0.5.2 and 0.6 (the latest SVN).
Here are the results of a program that hammers SQLObject (source attached).
It creates a simple database of 49+1 people, then spawns a bunch of
'whacker' threads which each repeatedly select all the records and iterate
over them.
---8<---8<---8<------S---N---I---P------>8--->8--->8---
$ python sqlobject_hammer.py
Making peeps...
--------------------------------------------------
49 peeps created...
WHACKING!!!
==================================================
whacker #0 | iteration: 1 (read 49 records)
whacker #0 | iteration: 2 (read 49 records)
whacker #1 | iteration: 1 (read 49 records)
whacker #0 | iteration: 3 (read 49 records)
whacker #0 | iteration: 4 (read 49 records)
whacker #0 | iteration: 5 (read 49 records)
whacker #0 | iteration: 6 (read 49 records)
whacker #1 | iteration: 2 (read 49 records)
whacker #0 | iteration: 7 (read 49 records)
whacker #1 | iteration: 3 (read 49 records)
whacker #0 | iteration: 8 (read 49 records)
whacker #1 | iteration: 4 (read 49 records)
whacker #1 | iteration: 5 (read 49 records)
whacker #1 | iteration: 6 (read 49 records)
whacker #2 | iteration: 1 (read 49 records)
whacker #0 | iteration: 9 (read 49 records)
Unhandled exception in thread started by <function whackit at 0x40459d84>
Traceback (most recent call last):
File "sqlobject_hammer.py", line 33, in whackit
del peep
UnboundLocalError: local variable 'peep' referenced before assignment
Unhandled exception in thread started by <function whackit at 0x40459d84>
Traceback (most recent call last):
File "sqlobject_hammer.py", line 29, in whackit
for count, peep in enumerate(peeps):
File "/usr/lib/python2.3/site-packages/sqlobject/main.py", line 1192, in
__iter__
return conn.iterSelect(self)
File "/usr/lib/python2.3/site-packages/sqlobject/dbconnection.py", line
199, in iterSelect
select, keepConnection=False)
File "/usr/lib/python2.3/site-packages/sqlobject/dbconnection.py", line
393, in __init__
self.cursor.execute(self.query)
File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 95, in
execute
return self._execute(query, args)
File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 114, in
_execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.3/site-packages/MySQLdb/connections.py", line 33,
in defaulterrorhandler
raise errorclass, errorvalue
ValueError: invalid literal for long(): w1
whacker #2 | iteration: 2 (read 49 records)
Unhandled exception in thread started by <function whackit at 0x40459d84>
Traceback (most recent call last):
File "sqlobject_hammer.py", line 33, in whackit
del peep
UnboundLocalError: local variable 'peep' referenced before assignment
whacker #1 | iteration: 7 (read 49 records)
Unhandled exception in thread started by <function whackit at 0x40459d84>
Traceback (most recent call last):
File "sqlobject_hammer.py", line 29, in whackit
for count, peep in enumerate(peeps):
File "/usr/lib/python2.3/site-packages/sqlobject/main.py", line 1192, in
__iter__
return conn.iterSelect(self)
File "/usr/lib/python2.3/site-packages/sqlobject/dbconnection.py", line
199, in iterSelect
select, keepConnection=False)
File "/usr/lib/python2.3/site-packages/sqlobject/dbconnection.py", line
393, in __init__
self.cursor.execute(self.query)
File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 95, in
execute
return self._execute(query, args)
File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 114, in
_execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.3/site-packages/MySQLdb/connections.py", line 33,
in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; You can't
run this command now")
Traceback (most recent call last):
File "sqlobject_hammer.py", line 49, in ?
main()
File "sqlobject_hammer.py", line 46, in main
pass
--->8--->8--->8------S---N---I---P------8<---8<---8<---
It just locked up after that.
I'm not really sure how to start debugging this. I'm a little scared of
DBConnection.py. :)
= Chris Gahan =============
(ch...@il...)
begin 666 sqlobject_hammer.py
M9G)O;2!S<6QO8FIE8W0@:6UP;W)T("H*7U]C;VYN96-T:6]N7U\@/2 G;7ES
M<6PZ+R]T97-T.G1E<W1 ;&]C86QH;W-T+W1E<W0G"FEM<&]R="!R86YD;VT*
M"FYU;7!E97!S(#T@-3 *;G5M=VAA8VME<G,@/2 T"@IC;&%S<R!0965P<RA3
M44Q/8FIE8W0I.@H@(&YA;64@/2!3=')I;F=#;VPH9&5F875L=#TB2&%N:S @
M3R=-86QL97DB*0H@(&%G92 ]($EN=$-O;"AD969A=6QT/3$P*0H*9&5F(&UA
M:V5P965P<R@I.@H@('!R:6YT(")-86MI;F<@<&5E<',N+BXB"B @<')I;G0@
M)RTG*C4P"B @4&5E<',N9')O<%1A8FQE*&EF17AI<W1S/51R=64I"B @4&5E
M<',N8W)E871E5&%B;&4H*0H@(&9O<B!I(&EN(')A;F=E*# L;G5M<&5E<',I
M.@H@(" @<&5E<" ](%!E97!S*&YA;64](DAA;FLE9"!/)TUA;&QE>2(E<F%N
M9&]M+G)A;F1I;G0H,2PY.3DI+ H@(" @(" @(" @(" @(" @(&%G93UR86YD
M;VTN<F%N9&EN="@Q+#4P,"DI"B @"B @<')I;G0@(B5D('!E97!S(&-R96%T
M960N+BXB("4@:0H*"F1E9B!W:&%C:VET*&ED*3H*(" @('0@/2 P"B @("!W
M:&EL92!T(#P@-3 P.@H@(" @(" @('0@*ST@,0H@(" @(" @('!E97!S(#T@
M4&5E<',N<V5L96-T*"D*(" @(" @("!F;W(@8V]U;G0L('!E97 @:6X@96YU
M;65R871E*'!E97!S*3H*(" @(" @(" @(" @<&%S<PH@(" @(" @('!R:6YT
M(")W:&%C:V5R(",E9"!\(&ET97)A=&EO;CH@)60@*')E860@)60@<F5C;W)D
M<RDB("4@*&ED+"!T+"!C;W5N="D*(" @(" @("!D96P@<&5E<',*(" @(" @
M("!D96P@<&5E< H@(" *9&5F(&UA:6XH*3H*(" @(&EM<&]R="!T:')E860*
M(" @(&UA:V5P965P<R@I"@H@(" @<')I;G0*(" @('!R:6YT(")72$%#2TE.
M1R$A(2(*(" @('!R:6YT("(](BHU, H*(" @(&9O<B!T(&EN(')A;F=E*&YU
M;7=H86-K97)S*3H*(" @(" @("!T:')E860N<W1A<G1?;F5W7W1H<F5A9"AW
M:&%C:VET+" H="PI*0H@(" @=VAI;&4@+3$Z"B @(" @(" @<&%S<PH@(" @
H"FEF(%]?;F%M95]?(#T]("=?7VUA:6Y?7R<Z"B @("!M86EN*"D*"@``
`
end
|
