Webware for Python

Are you using the latest version of mod_webkit from CVS?  There have been
bugfixes since Webware 0.8.1, and it's possible the bugs could cause lockups
or crashes.

- Geoff

deelan wrote:
> John Goodyear wrote:
> 
>> I get the feeling this is some kind of socket lockup, I've developed
>> multi threaded socket servers in C before and I have seen similar
>> lockups. I may take a look at the Adapter code and see if the problem
>> lies there.
> 
> hi there,
> 
> this happened to me recently too, running suse linux 8
> + apache 1.3, Webware 0.8.1 + mod_webkit.
> 
> at the time of writing my experimental site
> <http://www.netspyke.com/> is down due to webware
> refuse to process the /index.py servlet--please
> note that apache is up and running, just try
> to request the template for the homepage:
> <http://www.netspyke.com/t/site.html>.
> 
> it's happened on last friday also. i had to manually
> kill the process and run ./AppServer again. i'm really
> puzzled since untile things  run very smoothly. this
> thing is hard to track and like already noted it
> appears randomly, and what's worst it doesn't return any error
> information. 
> 
> 
> any hints on what might causing this will
> be greatly appreciated.
> 
> thanks in advance.
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss

Re: [Webware-discuss] Random AppServer Lockups? From: deelan <deelan@in...> - 2004-07-20 12:47 Geoffrey Talvola wrote: > Are you using the latest version of mod_webkit from CVS? There have been > bugfixes since Webware 0.8.1, and it's possible the bugs could cause lockups > or crashes. no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. from what i see mod_webkit for apache 1.3 (unlike mod_webkit2) has not be patched recently: <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/Adapters/> if you suspect that lockups could be caused by python code other than C code in mod_webkit i could try to update my webware to CVS version, but i'm a bit reclutant to do that since i've got some production code running with 0.8.1. cheers, deelan.

RE: [Webware-discuss] Random AppServer Lockups? From: Geoffrey Talvola <gtalvola@na...> - 2004-07-20 14:25 deelan wrote: > Geoffrey Talvola wrote: > >> Are you using the latest version of mod_webkit from CVS? There have >> been bugfixes since Webware 0.8.1, and it's possible the bugs could >> cause lockups or crashes. > > no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. > from what i see mod_webkit for apache 1.3 (unlike > mod_webkit2) has not be patched recently: > > <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/ > Adapters/> You're right, I forgot that the bugfixes were only for Apache 2.0. > if you suspect that lockups could be caused by python > code other than C code in mod_webkit i could try to update my webware > to CVS version, but i'm a bit reclutant to do that since i've > got some production code running with 0.8.1. I don't know of anything in the Python code in 0.8.1 that would cause lockups -- sorry. - Geoff

RE: [Webware-discuss] Random AppServer Lockups? From: Ian Maurer <ian_maurer@ya...> - 2004-07-20 15:02 --- Geoffrey Talvola <gtalvola@...> wrote: > deelan wrote: > > Geoffrey Talvola wrote: > > > >> Are you using the latest version of mod_webkit from CVS? There > have > >> been bugfixes since Webware 0.8.1, and it's possible the bugs > could > >> cause lockups or crashes. > > > > no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. > > from what i see mod_webkit for apache 1.3 (unlike > > mod_webkit2) has not be patched recently: > > > > <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/ > > Adapters/> > > You're right, I forgot that the bugfixes were only for Apache 2.0. > > > if you suspect that lockups could be caused by python > > code other than C code in mod_webkit i could try to update my > webware > > to CVS version, but i'm a bit reclutant to do that since i've > > got some production code running with 0.8.1. > > I don't know of anything in the Python code in 0.8.1 that would > cause > lockups -- sorry. > > - Geoff I am just reporting that I have had the same issue on my laptop which is very low-powered: - RedHat 8 - 267 MHZ Pentium II - Dell Latitude - Webware 0.8.1 - Apache 1.3 - mod_webkit The app server starts up fine but eventually it will end up just sitting there not even showing any requests coming in. To get things working again I would not only have to restart the appserver but apache as well. This leads me to believe it may be a mod_webkit issue but I am not sure. I normally don't use this laptop for anything other than an X client to my desktop server, but I tried setting up the laptop for a travel and I wasn't able to get past this issue. __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail

RE: [Webware-discuss] Random AppServer Lockups? From: Geoffrey Talvola <gtalvola@na...> - 2004-07-20 15:33 Ian Maurer wrote: > --- Geoffrey Talvola <gtalvola@...> wrote: >> deelan wrote: >>> Geoffrey Talvola wrote: >>> >>>> Are you using the latest version of mod_webkit from CVS? There have >>>> been bugfixes since Webware 0.8.1, and it's possible the bugs could >>>> cause lockups or crashes. >>> >>> no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. >>> from what i see mod_webkit for apache 1.3 (unlike >>> mod_webkit2) has not be patched recently: >>> >>> <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/ >>> Adapters/> >> >> You're right, I forgot that the bugfixes were only for Apache 2.0. >> >>> if you suspect that lockups could be caused by python >>> code other than C code in mod_webkit i could try to update my >>> webware to CVS version, but i'm a bit reclutant to do that since >>> i've got some production code running with 0.8.1. >> >> I don't know of anything in the Python code in 0.8.1 that would cause >> lockups -- sorry. >> >> - Geoff > > I am just reporting that I have had the same issue on my laptop which > > is very low-powered: > > - RedHat 8 > - 267 MHZ Pentium II - Dell Latitude > - Webware 0.8.1 > - Apache 1.3 > - mod_webkit > > The app server starts up fine but eventually it will end up just > sitting there not even showing any requests coming in. To get things > working again I would not only have to restart the appserver but > apache as well. This leads me to believe it may be a mod_webkit issue > but I am not sure. > > I normally don't use this laptop for anything other than an X client > to my desktop server, but I tried setting up the laptop for a travel > and I wasn't able to get past this issue. It would be good to determine if individual threads in the thread pool are getting locked up, which would eventually reduce your threads from its initial pool of 10 threads (or whatever) to 0 threads -- at that point the application will be locked up. I would recommend placing a call like this: print '***** starting request on thread', threading.currentThread().getName() at the beginning of RequestHandler.handleRequest in ThreadedAppServer.py, and print '***** done with request on thread', threading.currentThread().getName() at the end of that method. Then you will be able to examine your logfile to see if threads are getting wedged in the middle of a request. - Geoff

RE: [Webware-discuss] Random AppServer Lockups? From: Aaron Switzer <aaron@ne...> - 2004-07-20 15:39 I have no idea if this is relevent, but there is an issue with lock-ups on Redhat 9 (and 8 too?) that has to do with how it handles threads. It's a known bug with the stock kernel and can be fixed by updating the kernel, or more simply by typing this at the command line before executing Webware: export LD_ASSUME_KERNEL=2.4.19 I got the tip off this list and I don't know why/how it works, but it does. If it works you can add the line to your .profile. Hope it helps, - Aaron On Tue, 2004-07-20 at 11:02, Ian Maurer wrote: > --- Geoffrey Talvola <gtalvola@...> wrote: > > deelan wrote: > > > Geoffrey Talvola wrote: > > > > > >> Are you using the latest version of mod_webkit from CVS? There > > have > > >> been bugfixes since Webware 0.8.1, and it's possible the bugs > > could > > >> cause lockups or crashes. > > > > > > no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. > > > from what i see mod_webkit for apache 1.3 (unlike > > > mod_webkit2) has not be patched recently: > > > > > > <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/ > > > Adapters/> > > > > You're right, I forgot that the bugfixes were only for Apache 2.0. > > > > > if you suspect that lockups could be caused by python > > > code other than C code in mod_webkit i could try to update my > > webware > > > to CVS version, but i'm a bit reclutant to do that since i've > > > got some production code running with 0.8.1. > > > > I don't know of anything in the Python code in 0.8.1 that would > > cause > > lockups -- sorry. > > > > - Geoff > > I am just reporting that I have had the same issue on my laptop which > > is very low-powered: > > - RedHat 8 > - 267 MHZ Pentium II - Dell Latitude > - Webware 0.8.1 > - Apache 1.3 > - mod_webkit > > The app server starts up fine but eventually it will end up just > sitting there not even showing any requests coming in. To get things > working again I would not only have to restart the appserver but > apache as well. This leads me to believe it may be a mod_webkit issue > but I am not sure. > > I normally don't use this laptop for anything other than an X client > to my desktop server, but I tried setting up the laptop for a travel > and I wasn't able to get past this issue. > > > > > __________________________________ > Do you Yahoo!? > New and Improved Yahoo! Mail - Send 10MB messages! > http://promotions.yahoo.com/new_mail > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click > _______________________________________________ > Webware-discuss mailing list > Webware-discuss@... > https://lists.sourceforge.net/lists/listinfo/webware-discuss

RE: [Webware-discuss] Random AppServer Lockups? From: Ian Maurer <ian_maurer@ya...> - 2004-07-21 00:18 Aaron, It might be premature, since I haven't done exhaustive testing, but I think this solved the problem. My laptop indeed was RedHat 9 (not 8 like I said) and this solution seems to work. Thanks for helping me... and hopefully this helps deelan as well. -ian --- Aaron Switzer <aaron@...> wrote: > I have no idea if this is relevent, but there is an issue with > lock-ups > on Redhat 9 (and 8 too?) that has to do with how it handles > threads. > It's a known bug with the stock kernel and can be fixed by updating > the > kernel, or more simply by typing this at the command line before > executing Webware: > > export LD_ASSUME_KERNEL=2.4.19 > > I got the tip off this list and I don't know why/how it works, but > it > does. If it works you can add the line to your .profile. > > Hope it helps, > > - Aaron > > > On Tue, 2004-07-20 at 11:02, Ian Maurer wrote: > > --- Geoffrey Talvola <gtalvola@...> wrote: > > > deelan wrote: > > > > Geoffrey Talvola wrote: > > > > > > > >> Are you using the latest version of mod_webkit from CVS? > There > > > have > > > >> been bugfixes since Webware 0.8.1, and it's possible the > bugs > > > could > > > >> cause lockups or crashes. > > > > > > > > no, i'm using apache 1.3 + mod_webkit from webware 0.8.1. > > > > from what i see mod_webkit for apache 1.3 (unlike > > > > mod_webkit2) has not be patched recently: > > > > > > > > > <http://cvs.sourceforge.net/viewcvs.py/webware/Webware/WebKit/ > > > > Adapters/> > > > > > > You're right, I forgot that the bugfixes were only for Apache > 2.0. > > > > > > > if you suspect that lockups could be caused by python > > > > code other than C code in mod_webkit i could try to update my > > > webware > > > > to CVS version, but i'm a bit reclutant to do that since > i've > > > > got some production code running with 0.8.1. > > > > > > I don't know of anything in the Python code in 0.8.1 that would > > > cause > > > lockups -- sorry. > > > > > > - Geoff > > > > I am just reporting that I have had the same issue on my laptop > which > > > > is very low-powered: > > > > - RedHat 8 > > - 267 MHZ Pentium II - Dell Latitude > > - Webware 0.8.1 > > - Apache 1.3 > > - mod_webkit > > > > The app server starts up fine but eventually it will end up just > > sitting there not even showing any requests coming in. To get > things > > working again I would not only have to restart the appserver but > > apache as well. This leads me to believe it may be a mod_webkit > issue > > but I am not sure. > > > > I normally don't use this laptop for anything other than an X > client > > to my desktop server, but I tried setting up the laptop for a > travel > > and I wasn't able to get past this issue. > > > > > > > > > > __________________________________ > > Do you Yahoo!? > > New and Improved Yahoo! Mail - Send 10MB messages! > > http://promotions.yahoo.com/new_mail > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by BEA Weblogic Workshop > > FREE Java Enterprise J2EE developer tools! > > Get your free copy of BEA WebLogic Workshop 8.1 today. > > http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click > > _______________________________________________ > > Webware-discuss mailing list > > Webware-discuss@... > > https://lists.sourceforge.net/lists/listinfo/webware-discuss > > __________________________________ Do you Yahoo!? Yahoo! Mail - 50x more storage than other providers! http://promotions.yahoo.com/new_mail

Re: [Webware-discuss] Random AppServer Lockups? From: Andrea <deelan@in...> - 2004-09-07 14:30 Geoffrey Talvola wrote: > It would be good to determine if individual threads in the thread pool are > getting locked up, which would eventually reduce your threads from its > initial pool of 10 threads (or whatever) to 0 threads -- at that point the > application will be locked up. I would recommend placing a call like this: > > print '***** starting request on thread', > threading.currentThread().getName() > > at the beginning of RequestHandler.handleRequest in ThreadedAppServer.py, > and > > print '***** done with request on thread', > threading.currentThread().getName() > > at the end of that method. Then you will be able to examine your logfile to > see if threads are getting wedged in the middle of a request. finally it seems i've traced something interesting, the folling is the output of nohup.out file: *** Starting request on thread Thread-9 341 2004-09-07 14:59:18 /products/ 341 0.07 secs /products/ *** Done with request on thread Thread-9 *** Starting request on thread Thread-11 342 2004-09-07 15:00:41 /dealerarea/ 342 0.02 secs /dealerarea/ *** Done with request on thread Thread-11 <--- ok until here *** Starting request on thread Thread-10 343 2004-09-07 15:00:48 /news/ *** Starting request on thread Thread-12 344 2004-09-07 15:04:16 / *** Starting request on thread Thread-13 345 2004-09-07 15:05:15 /news/ *** Starting request on thread Thread-9 346 2004-09-07 15:05:37 /it/index.py 346 0.04 secs /it/index.py *** Done with requ./AppServer: line 9: 31074 Killed note how Thread-10, 12 and 13 got stuck. i'm running SUSE linux. i'm wondering if this can be in some way related to my use of a NOT threadsafe mysqldb module. i plan to move soon the app to a new linux server (it's SUSE too) with mysqldb compiled againt threadsafe mysqlclient library. to my understanding to check if mysqldb is using a threadsafe client lib at the python prompt one has to type: import _mysql print _mysql.thread_safe() a zero value stands for no, otherwise yes. also, i'm not experiencing any lock-up on my win2k box, where i develop most of the code (threadsafe mysqldb module ). on both installations i'm using DbConnectionPool module, by Written by Warren A. Smith to allows to use a pool of mysql connection in a more reliable way then standard webware dbpool (it resumes a "closed" connection, useful when you have a nasty firewal between webware and mysql). i always return the connection to the pool since i've this code on my SitePage class: def awake(self, transaction): Page.awake(self, transaction) # acquire DB connection from pool self._cnn = cm.connection def sleep(self, transaction): Page.sleep(self, transaction) # release DB connection to pool self._cnn.close() i'm really puzzled since the lock-up seems act randomly both in time and in the place (different servlets). next step is to replace DbConnectionPool with webware with DbPool, since there isn't any firewall between linux server and mysql (they are on the same server). any idea will be greatly appreciated. cheers, deelan.

Re: [Webware-discuss] Random AppServer Lockups? From: Warren Smith <warren@wa...> - 2004-09-07 16:03 Andrea said: > on both installations i'm using DbConnectionPool module, > by Written by Warren A. Smith to allows to use a pool of mysql > connection in a more reliable way then standard webware dbpool > (it resumes a "closed" connection, useful when you have > a nasty firewal between webware and mysql). > > i always return the connection to the pool since i've this > code on my SitePage class: > > def awake(self, transaction): > Page.awake(self, transaction) > > # acquire DB connection from pool > self._cnn = cm.connection > > def sleep(self, transaction): > Page.sleep(self, transaction) > > # release DB connection to pool > self._cnn.close() > > i'm really puzzled since the lock-up seems act randomly both > in time and in the place (different servlets). next step is to > replace DbConnectionPool with webware with DbPool, since there isn't > any firewall between linux server and mysql (they are on the same > server). > > any idea will be greatly appreciated. > I haven't done any Webware development for quite some time and I never fully tested DbConnectionPool. However, this past weekend I was working on a Webware/Cheetah/PostgresSQL project and had some similar problems to yours on my development server, where Apache, Webware, and PostgresSQL are all running on the same machine (no firewall anywhere). Suspecting the database connection as the culprit, I tried adjusting the maxIdle and maxAge parameters of DbConnectionPool.startExpiration() to lower values than the defaults. So far, my problems have vanished. Perhaps I was running into a default timeout on the PostgresSQL connection itself? Just curious. Have you had any problems with DbConnectionPool? I knew a few people gave it a try back when I wrote it, but this is the first time that I have heard of someone still using it. Feedback, both positive and negative are still appreciated. I have some ideas to increase the efficiency of the expiration logic and I have also considered adding a maxUsage parameter that limits the number of times a connection can be acquired. I also need to write a better test suite using Python's unittest framework. The existing test code, while exercising the module pretty well, does not really accomplish the task of testing it. Now that I am working on a project myself that make use of DbConnectionPool, perhaps I can work in some of these changes. -- Warren Smith warren@...

Re: [Webware-discuss] Random AppServer Lockups? From: Warren Smith <warren@wa...> - 2004-09-07 18:34 Andrea said: > on both installations i'm using DbConnectionPool module, > by Written by Warren A. Smith to allows to use a pool of mysql > connection in a more reliable way then standard webware dbpool > (it resumes a "closed" connection, useful when you have > a nasty firewal between webware and mysql). > You may already realize this, but DbConnectionPool has no way of knowing if the connection that it has an internal reference to is really usable, so I'm not sure what you mean by 'resumes a "closed" connection'. My intent in writing DbConnectionPool was to only keep references to relatively "fresh" connections in the hopes that they really ARE usable. Anything that causes a connection to become unusable can still cause problems while that connection is still in the pool. As long as the "bad" connection is being handed out and "used" (attribute proxying), it will never expire because of "idleness", even though the connection is unusable for whatever reason. The maxAge expiration parameter was designed to help minimize the impact (and negative exposure) of this situation. A new maxUsage (name?) expiration parameter would also accomplish the same thing, but would be based on the number of times a (potentially "bad") connection has been acquired instead of the amount of time a (potentially "bad") connection has been in the pool. This would make the degraded service time window smaller for a more heavily loaded site, while sacrificing some performance benefits of having the pool in the first place. Please let me know if this is something that would be usefull for you. I think it would be fairly trivial to implement. Later. -- Warren Smith warren@...

Re: [Webware-discuss] DB Connection pooling (Random AppServer Lockups? ) From: deelan <deelan@in...> - 2004-09-08 09:47 hi warren, thanks for you interest on the topic. my reply is below. Warren Smith wrote: > I haven't done any Webware development for quite some time and I never > fully tested DbConnectionPool. However, this past weekend I was working > on a Webware/Cheetah/PostgresSQL project and had some similar problems to > yours on my development server, where Apache, Webware, and PostgresSQL are > all running on the same machine (no firewall anywhere). Suspecting the > database connection as the culprit, I tried adjusting the maxIdle and > maxAge parameters of DbConnectionPool.startExpiration() to lower values > than the defaults. So far, my problems have vanished. Perhaps I was > running into a default timeout on the PostgresSQL connection itself? so "bad" connections into DbConnectionPool can potentially block the threads? this seems my problem too. after reviewing again my log file i've spotted a request which caused a 404 response to be generated by webware: *** Starting request on thread Thread-9 346 2004-09-07 15:05:37 /it/index.py <--- this file does not exist 346 0.04 secs /it/index.py *** Done with requ[est on thread Thread-9] but note how Thread-9 didn't get stuck, probably because webware didn't requested any connection from the pool (no SitePage awake/sleep cycle), but simply redirected user to its default 404 page (handleBadURL method). > > Just curious. Have you had any problems with DbConnectionPool? I knew a > few people gave it a try back when I wrote it, but this is the first time > that I have heard of someone still using it. Feedback, both positive and > negative are still appreciated. i've started using DbConnectionPool cause i have a firewall between my dev win2k box and mysql running un linux, i was experiencing frequent "lost connection to mysql" exceptions (not python-releated, i got those error messages with MysqlCC GUI tool too). so i replaced MiscUtils.DbPool with DbConnectionPool and problems went away. but on my win2k i've no chance to test properly long-running processes and such appserver lock-ups really manifest on the deploy machine. > You may already realize this, but DbConnectionPool has no way of knowing > if the connection that it has an internal reference to is really usable, > so I'm not sure what you mean by 'resumes a "closed" connection'. yes, my apologies, the phrase 'resumes a "closed" connection' does not really describe what DbConnectionPool do. i've not completely ruled out that the problems might be caused by the fact i'm using a not threadsafe mysqlclient library. i'll hopefully move the whole thing to a new linux server and see if it fixes the problem, but at this time i'm pretty sure it's a database-related problem. thanks for your time. cheers, deelan.

1 message has been excluded from this view by a project administrator.