Thread: [SQLObject] select() causes webware/threaded application to die (part II)
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Scott R. <sc...@to...> - 2004-03-29 10:51:33
|
0.5.2 brings this problem back, and the patch to fix it from 02-Mar-04 no longer applies. I had to roll back to 0.5.1 and apply that fix to use select without the AppServer punting. Am I the only one using select(), or does my use somehow trigger this problem? |
From: Chris G. <ch...@il...> - 2004-03-30 00:00:35
|
Scott Russell <sc...@to...> wrote in news:1080557484.8209.97.camel@glimmer: > 0.5.2 brings this problem back, and the patch to fix it from 02-Mar-04 > no longer applies. I had to roll back to 0.5.1 and apply that fix to > use select without the AppServer punting. > > Am I the only one using select(), or does my use somehow trigger this > problem? I'm having the same problem. You can read the thread entitled "Concurrency issues..." if you wanna see what I've been experiencing. I think the reason other people don't complain about this is that they don't hammer SQLObject as badly as we do. :) The reason my code hammers SQLObject is because I use my database to store image metadata to display a bunch of images on a webware page, and every image on the page gets returned by a webware servlet, which will result in like 10 servlets all getting called at once, each one does an SQLObject query. I temporarily fixed my problem by setting Webware to only use one thread, but that's not "optimal". :) |
From: Scott R. <sc...@to...> - 2004-03-30 01:31:11
|
On Mon, 2004-03-29 at 19:00, Chris Gahan wrote: > > > Am I the only one using select(), or does my use somehow trigger this > > problem? > > I'm having the same problem. You can read the thread entitled > "Concurrency issues..." if you wanna see what I've been experiencing. > > I think the reason other people don't complain about this is that they > don't hammer SQLObject as badly as we do. :) > > The reason my code hammers SQLObject is because I use my database to > store image metadata to display a bunch of images on a webware page, and > every image on the page gets returned by a webware servlet, which will > result in like 10 servlets all getting called at once, each one does an > SQLObject query. > > I temporarily fixed my problem by setting Webware to only use one thread, > but that's not "optimal". :) I've got a public multiuser scenario where I build dynamic calendars from slightly complex queries - that's the hammering that I'm killing it with. Have you tried moving back to 0.5.1 and applying the patch from the date mentioned below? I'll go catch up on your concurrency thread. |
From: Chris G. <ch...@il...> - 2004-03-30 02:00:48
|
Scott Russell <sc...@to...> wrote: > I've got a public multiuser scenario where I build dynamic calendars > from slightly complex queries - that's the hammering that I'm killing > it with. Ah, I see... > Have you tried moving back to 0.5.1 and applying the patch from the > date mentioned below? I'll go catch up on your concurrency thread. Oh my god! That fixes it completely! I can hammer it insanely and it doesn't flinch! This has to be a simple glitch in the new code then, which I don't compeltely understand. :) |
From: Luke O. <lu...@me...> - 2004-03-30 03:18:38
|
I suppose I shall enter my own bit here: We also have a few apps that initiate several webware/XMLRPC calls per web request to a page (XMLRPC from a flash app), would hang webware until we applied this fix. The glitch (as I thought was documented with the patch?) is that the connection is returned by runWithConnection after the first value, so subsequent iteration may use the connection which has been subsequently been given to another request. - Luke Quoting Chris Gahan <ch...@il...>: > Scott Russell <sc...@to...> wrote: >> I've got a public multiuser scenario where I build dynamic calendars >> from slightly complex queries - that's the hammering that I'm killing >> it with. > > Ah, I see... > >> Have you tried moving back to 0.5.1 and applying the patch from the >> date mentioned below? I'll go catch up on your concurrency thread. > > Oh my god! That fixes it completely! I can hammer it insanely and it > doesn't flinch! > > This has to be a simple glitch in the new code then, which I don't > compeltely understand. :) > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IBM Linux Tutorials > Free Linux tutorial presented by Daniel Robbins, President and CEO of > GenToo technologies. Learn everything from fundamentals to system > administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss -- The Pursuit of Counterfactual Histories |
From: Ian B. <ia...@co...> - 2004-03-30 05:14:30
|
On Mar 29, 2004, at 4:51 AM, Scott Russell wrote: > 0.5.2 brings this problem back, and the patch to fix it from 02-Mar-04 > no longer applies. I had to roll back to 0.5.1 and apply that fix to > use select without the AppServer punting. Which patch are we talking about? There's no open patches on the SF patch manager. If there's something out there that fixes this, I'm sure we can get it into the code. -- Ian Bicking | ia...@co... | http://blog.ianbicking.org |
From: Chris G. <ch...@il...> - 2004-03-30 06:11:08
|
On 30 Mar 2004, Ian Bicking said: > Which patch are we talking about? There's no open patches on the SF > patch manager. If there's something out there that fixes this, I'm > sure we can get it into the code. It was a quick fix you posted on the mailing list. Here's the original message you posted: > Hmm... > > Bah, DBConnection.iterSelect is dumb. It shouldn't be written like it > is. Right now it looks like: > > def iterSelect(self, select): > return self._runWithConnection(self._iterSelect, select, self, > False) > > Which means the connection is returned to the pool immediately when the > generator is returned, then reused when it shouldn't be, and worse it > can be returned to the pool another time if the select results are > exhausted. It should look like: > > def iterSelect(self, select): > conn = self.getConnection() > return self._iterSelect(conn, select, self, False) > > This isn't quite right either, as the connection will only be returned > if you exhaust the select. _iterSelect should be broken out into an > object that returns the connection on __del__. Anyway, this iterSelect > should resolve your concurrency problems for the moment (I haven't > tested it, though; report back if it works) > > Ian This only works for 0.5.1, of course, cause you added the Iterator() class in 0.5.2 and 0.6, but the Iterator() class has the same bug as the unpatched 0.5.1. Here's my SQLObject Hammerer again, incase you wanna replicate and track down this baby. :) begin 644 sqlobject_hammer.py M9G)O;2!344Q/8FIE8W0@:6UP;W)T("H*(U]?8V]N;F5C=&EO;E]?(#T@4&]S M=&=R97-#;VYN96-T:6]N*&AO<W0])VQO8V%L:&]S="<L('5S97(])W1E<W0G M+"!P87-S=V0])W1E<W0G+"!D8CTG=&5S="<I"E]?8V]N;F5C=&EO;E]?(#T@ M37E344Q#;VYN96-T:6]N*'5S97(])W1E<W0G+"!P87-S=V0])W1E<W0G+"!D M8CTG=&5S="<I"B-?7V-O;FYE8W1I;VY?7R`]($UY4U%,0V]N;F5C=&EO;BAU M<V5R/2=T97-T)RP@<&%S<W=D/2=T97-T)RP@9&(])W1E<W0G+"!D96)U9STQ M*0II;7!O<G0@<F%N9&]M"@IN=6UP965P<R`](#(*;G5M=VAA8VME<G,@/2`R M,`IW:&%C:W,@/2`U,#`*"F-L87-S(%!E97!S*%-13$]B:F5C="DZ"B`@;F%M M92`](%-T<FEN9T-O;"AD969A=6QT/2)(86YK,"!/)TUA;&QE>2(I"B`@86=E M(#T@26YT0V]L*&1E9F%U;'0],3`I"@ID968@;6%K97!E97!S*"DZ"B`@<')I M;G0@(DUA:VEN9R!P965P<RXN+B(*("!P<FEN="`G+2<J-3`*("!0965P<RYD M<F]P5&%B;&4H:69%>&ES=',]5')U92D*("!0965P<RYC<F5A=&5486)L92@I M"B`@9F]R(&D@:6X@<F%N9V4H,"QN=6UP965P<RDZ"B`@("!P<FEN="`B<&5E M<"(L(&D*("`@('!E97`@/2!0965P<RYN97<H(&YA;64](DAA;FLE9"!/)TUA M;&QE>2(E<F%N9&]M+G)A;F1I;G0H,2PY.3DI+`H@("`@("`@("`@("`@("`@ M("`@("`@86=E/7)A;F1O;2YR86YD:6YT*#$L-3`P*2`I"B`@"B`@<')I;G0@ M(B5D('!E97!S(&-R96%T960N+BXB("4@*&DK,2D*"@ID968@=VAA8VMI="AI M9"DZ"B`@("!T(#T@,`H@("`@=VAI;&4@="`\('=H86-K<SH*("`@("`@("!T M("L](#$*("`@("`@("!P965P<R`](%!E97!S+G-E;&5C="@I"B`@("`@("`@ M9F]R(&-O=6YT+"!P965P(&EN(&5N=6UE<F%T92AP965P<RDZ"B`@("`@("`@ M("`@('!A<W,*("`@("`@("!P<FEN="`B=VAA8VME<B`C)60@?"!I=&5R871I M;VXZ("5D("AR96%D("5D(')E8V]R9',I(B`E("AI9"P@="P@8V]U;G0K,2D* M("`@("`@("!D96P@<&5E<',*("`@("`@("!D96P@<&5E<`H@("`@("`@(`H@ M("`@<')I;G0@(G=H86-K97(@(R5D("T^($9)3DE32$5$(2(@)2!I9`H@("`* M9&5F(&UA:6XH*3H*("`@(&EM<&]R="!T:')E860*("`@(&UA:V5P965P<R@I M"@H@("`@<')I;G0*("`@('!R:6YT(")72$%#2TE.1R$A(2(*("`@('!R:6YT M("(](BHU,`H*("`@(&9O<B!T(&EN(')A;F=E*&YU;7=H86-K97)S*3H*("`@ M("`@("!T:')E860N<W1A<G1?;F5W7W1H<F5A9"AW:&%C:VET+"`H="PI*0H@ M("`@=VAI;&4@+3$Z"B`@("`@("`@<&%S<PH@("`@"FEF(%]?;F%M95]?(#T] 9("=?7VUA:6Y?7R<Z"B`@("!M86EN*"D*"HD! ` end |
From: Ian B. <ia...@co...> - 2004-03-31 06:48:33
|
Okay, the problem is fixed in the repository now (svn://colorstudy.com/branches/SQLObject/0.5). It turned out the connections were being returned to the pool multiple times, once when the iterator was exhausted and a second time when the iterator was deleted. Blech. Anyway, should be fixed now. I'll make another release shortly as long as there aren't any problems. Kind of sucks to make another release to fix the problem the last release was supposed to fix. The 0.6 branch is being updated as well. -- Ian Bicking | ia...@co... | http://blog.ianbicking.org |
From: Chris G. <ch...@il...> - 2004-03-31 16:47:33
|
On 31 Mar 2004, Ian Bicking said: > Okay, the problem is fixed in the repository now > (svn://colorstudy.com/branches/SQLObject/0.5). It turned out the > connections were being returned to the pool multiple times, once when > the iterator was exhausted and a second time when the iterator was > deleted. Blech. Anyway, should be fixed now. I'll make another > release shortly as long as there aren't any problems. Kind of sucks to > make another release to fix the problem the last release was supposed > to fix. > > The 0.6 branch is being updated as well. Awesome! :D Thanks for figuring this out. I was kinda stuck. :) |