Webware for Python

Why not parse the python code fragments before executing them and insert a
check for a stop event in the beginning of each loop.  It could even force a
loop counter of some type that would allow the main thread to force the
worker thread to stop processing through a stop event once a threshold has
been reached.

-----Original Message-----
> [Webware-discuss] Dead threads: detection and recovery Oliver Bock 
> Mon, 19 Jun 2006 00:50:23 -0700
>
> My system allows users to write their own python code fragments for 
> some tasks.  Unfortunately the users sometimes write infinite loops, 
> which gradually lock up threads until none remain.  A few minutes 
> later a monitor notices that the system is unresponsive and kills and 
> restarts AppServer.  There must be a better way.
>
> 1. Can I find out how long each busy thread has been working on its 
> current request?
> 2. Can I kill threads without irritating w4py?
>
> I guess I can roll my own solution for detecting busy threads, but I'm 
> hoping someone else has a solution for this, or that I've failed to 
> notice something obvious that already exists.
>
>
>    Oliver

_______________________________________________
Webware-discuss mailing list
Webware-discuss@...
https://lists.sourceforge.net/lists/listinfo/webware-discuss

Re: [Webware-discuss] Dead threads: detection and recovery From: Oliver Bock <oliver@g7...> - 2006-06-21 23:32 This is a good idea, but a lot of work. (It messes with my head whenever I get into the parse tree stuff.) Oliver -- Wheeler, Richard wrote: > Why not parse the python code fragments before executing them and insert a > check for a stop event in the beginning of each loop. It could even force a > loop counter of some type that would allow the main thread to force the > worker thread to stop processing through a stop event once a threshold has > been reached. >

Re: [Webware-discuss] Dead threads: detection and recovery Oliver Bock From: Ian Bicking <ianb@co...> - 2006-06-21 23:55 Wheeler, Richard wrote: > Why not parse the python code fragments before executing them and insert a > check for a stop event in the beginning of each loop. It could even force a > loop counter of some type that would allow the main thread to force the > worker thread to stop processing through a stop event once a threshold has > been reached. This recipe might be of use: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496746 I don't believe it is threadsafe, but I'm not sure. I think it needs to run the code in the main thread, then puts an alarm in a subthread that interrupts the main thread if it takes too long. I don't think other threads can be interrupted that way.

Re: [Webware-discuss] Dead threads: detection and recovery From: Richard Wheeler <richard_wheeler@ci...> - 2006-06-22 01:25 Ian Bicking <ianb@...> writes: > This recipe might be of use: > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496746 > > I don't believe it is threadsafe, but I'm not sure. I think it needs to > run the code in the main thread, then puts an alarm in a subthread that > interrupts the main thread if it takes too long. I don't think other > threads can be interrupted that way. That is a very good find Ian. It is very similar to what I am suggesting. In my worker threads we used event object for signaling. The main thread joins the worker thread to set the event which worker thread then detects it is to end. Using that mechanism you would avoid the issue of how to kill the thread since the thread ends it self based on the event object allowing it to break out of the loop. Since the recipe has the parsing logic already in place it shouldn't take much tweaking to scan for loop keywords and insert code. I would log those code fragments so that if the parsing logic misses a loop, you could examine the code fragment and further refine the parsing logic. Eventually it would become bullet proof. Using this mechanism you can enforce the necessary logic to avoid infinite loops. Oliver to solve your issue it is going to take some work given the nature of your system that you have described. With that recipe you have a lot of the work already done. If you are not sure how to setup worker threads and use event objects I could post a code fragment here. For that matter I think ASPN already has recipes for those concepts.

Re: [Webware-discuss] Dead threads: detection and recovery From: Oliver Bock <oliver@g7...> - 2006-06-22 07:42 I've implemented the infinite loop-catching suggestion by instrumenting the Python code. I haven't adopted the techniques from the article Ian found because I'm lazy, and because my users are silly, not evil. It took a bit of buggering about but the result is fairly simple. Read on if you're interested. 1. I figure that the only Python construct that can easily loop infinitely is the while statement. (for statements could also blow out, and I'll fix that if it becomes a problem.) 2. I can convert the code to be evaluated into a parse tree as follows: asList = parser.suite(code).tolist(True) # We want original line numbers to be preserved 3. I can instrument this code by adding a call to loopCheck() at the beginning of each while loop: (This code assumes you're familiar with the structure of Python parse trees.) # Construct the parse tree for the call to loopCheck() _callLoopCheck = parser.suite('loopCheck()').tolist()[1] def _addLoopChecks(l): """Takes a parse tree and adds calls to loopCheck() to every loop.""" if type(l) is list: for subL in l: _addLoopChecks(subL) if type(l[1]) is list: if symbol.sym_name[l[0]] == 'while_stmt': m = l[4] assert symbol.sym_name[m[0]] == 'suite' i = 1 while type(m[i][1]) is not list: i = i + 1 m.insert(i, _callLoopCheck) 4. Then I can recompile the patched parse tree: compiled = parser.sequence2ast(asList).compile() 5. And finally execute it: class _LoopCheck: def __init__(self): self._loopCount = 0 def loopCheck(self): self._loopCount = self._loopCount + 1 if self._loopCount == 500000: raise Exception, "Code has gone into an infinite loop." exec compiled in { 'loopCheck': _LoopCheck().loopCheck } Thanks for all the help. Oliver -- Richard Wheeler wrote: > Ian Bicking <ianb@...> writes: > >> This recipe might be of use: >> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496746 >> >> I don't believe it is threadsafe, but I'm not sure. I think it needs to >> run the code in the main thread, then puts an alarm in a subthread that >> interrupts the main thread if it takes too long. I don't think other >> threads can be interrupted that way. >> > > That is a very good find Ian. It is very similar to what I am suggesting. In > my worker threads we used event object for signaling. The main thread joins > the worker thread to set the event which worker thread then detects it is to > end. Using that mechanism you would avoid the issue of how to kill the thread > since the thread ends it self based on the event object allowing it to break > out of the loop. > > Since the recipe has the parsing logic already in place it shouldn't take much > tweaking to scan for loop keywords and insert code. I would log those code > fragments so that if the parsing logic misses a loop, you could examine the > code fragment and further refine the parsing logic. Eventually it would > become bullet proof. > > Using this mechanism you can enforce the necessary logic to avoid infinite > loops. > > Oliver to solve your issue it is going to take some work given the nature of > your system that you have described. With that recipe you have a lot of the > work already done. If you are not sure how to setup worker threads and use > event objects I could post a code fragment here. For that matter I think ASPN > already has recipes for those concepts. > > > > > > > > > All the advantages of Linux Managed Hosting--Without the Cost and Risk! > Fully trained technicians. The highest number of Red Hat certifications in > the hosting industry. Fanatical Support. Click to learn more > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642 > _______________________________________________ > Webware-discuss mailing list > Webware-discuss@... > https://lists.sourceforge.net/lists/listinfo/webware-discuss > >

Re: [Webware-discuss] Dead threads: detection and recovery From: Richard Wheeler <richard_wheeler@ci...> - 2006-06-22 10:48 ----- Oliver Bock <oliver@...> writes: > > I've implemented the infinite loop-catching suggestion by instrumenting > the Python code. I haven't adopted the techniques from the article Ian > found because I'm lazy, and because my users are silly, not evil. It > took a bit of buggering about but the result is fairly simple. Read on > if you're interested. > > ................ > > Thanks for all the help. > > Oliver That is what I like about Python many ways to get the job done. Hmmm, I guess Alan Turing must not have been using Python. :)

Re: [Webware-discuss] Dead threads: detection and recovery From: Shayne O'Neill <shayne@pe...> - 2006-06-23 13:54 > Oliver Bock <oliver@...> writes: >> >> I've implemented the infinite loop-catching suggestion by instrumenting >> the Python code. I haven't adopted the techniques from the article Ian >> found because I'm lazy, and because my users are silly, not evil. It >> took a bit of buggering about but the result is fairly simple. Read on >> if you're interested. >> >> ................ >> >> Thanks for all the help. >> >> Oliver > > That is what I like about Python many ways to get the job done. > > Hmmm, I guess Alan Turing must not have been using Python. :) Mr Turing only said you couldnt (reliably) predict it, not that it cant be caught when it happens. That said, I'd *suspect* that you can actually say "It is possible to write a program on a universal machine that will tell if another universal machine will complete before a certain time/iiterations", but to know if it would ever end is not (reliably) possible (Although one could look for certain heuristics that are known to be infinite a-priori). I didnt know the parse tree stuff btw, what a clever little critter a python is. > > > All the advantages of Linux Managed Hosting--Without the Cost and Risk! > Fully trained technicians. The highest number of Red Hat certifications in > the hosting industry. Fanatical Support. Click to learn more > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642 > _______________________________________________ > Webware-discuss mailing list > Webware-discuss@... > https://lists.sourceforge.net/lists/listinfo/webware-discuss >

[Webware-discuss] Forwarded requests and exception reports From: Oliver Bock <oliver@g7...> - 2006-06-29 04:14 To provide more concise URLs to users, I use a servlet called index.py to forward requests. I'm sure I didn't think this up myself, but I cannot find it in the current docs. from WebKit.HTTPServlet import HTTPServlet class index(HTTPServlet): def respond(self, trans): trans.application().forward(trans, '/RealServlet' + trans.request().extraURLPath() ) This has been working well for me except when I override writeExceptionReport(), in which case I see this under the "Session" heading of the exception report: Uncaught exception while asking *servlet* to write report: Traceback (most recent call last): File "/Users/oliver/Webware-0.9.1/WebKit/Transaction.py", line 199, in writeExceptionReport obj.writeExceptionReport(handler) AttributeError: 'index' object has no attribute 'writeExceptionReport' I think the 'index' object here is my redirecting servlet. Am I doing something wrong here? Can anyone suggest a workaround? I tried to follow the code in Transaction.py and Application.py, but I got lost in a maze of little methods, all alike. Oliver

Re: [Webware-discuss] Forwarded requests and exception reports From: Christoph Zwerschke <cito@on...> - 2006-06-29 09:22 Oliver Bock wrote: > To provide more concise URLs to users, I use a servlet called index.py > to forward requests. I'm sure I didn't think this up myself, but I > cannot find it in the current docs. You can also use Main.py, and you can configure it in Application.config, setting "DirectoryFile": http://www.webwareforpython.org/WebKit/Docs/Configuration.html#general-settings > from WebKit.HTTPServlet import HTTPServlet > class index(HTTPServlet): > def respond(self, trans): > trans.application().forward(trans, '/RealServlet' + > trans.request().extraURLPath() ) > > This has been working well for me except when I override > writeExceptionReport(), HTTPServlet does not support this. You should use HTTPContent or Page. See the class hierarchy here: http://www.webwareforpython.org/WebKit/Docs/Source/ClassHierarchy.html Page is usually the best choice as base class for your web pages or for your own master class. -- Chris

Re: [Webware-discuss] Forwarded requests and exception reports From: Oliver Bock <oliver@g7...> - 2006-06-29 23:57 Christoph Zwerschke wrote: >> To provide more concise URLs to users, I use a servlet called index.py >> to forward requests. I'm sure I didn't think this up myself, but I >> cannot find it in the current docs. >> > > You can also use Main.py, and you can configure it in > Application.config, setting "DirectoryFile"... > That works perfectly and is a better way to do it. Thanks. >> This has been working well for me except when I override >> writeExceptionReport(), >> > > HTTPServlet does not support this. You should use HTTPContent or Page. > While your suggestion gets rid of the extra exception in the error report, it doesn't solve the problem, which is that forwarding a servlet fails to forward the call to writeExceptionReport(). (Your above solution fixes my problem; I'm continuing this thread solely to point out a bug in WebKit.) Oliver

[Webware-discuss] thread._processing/processing From: Oliver Bock <oliver@g7...> - 2006-06-30 04:43 In ThreadedAppServer.py (both CVS HEAD and 0.9.1), whether a thread is doing work is indicated via a boolean attribute called ".processing" in some situations, and "._processing" in others. A consequence of this mix up is that activeThreadCount() always returns zero because it looks in ._processing whereas threadloop() sets .processing. This means that ThreadedAppServer will _always_ run only the minimum number of threads. Have I got this right? If so, should I just submit a bug report on sourceforge? Oliver

Re: [Webware-discuss] thread._processing/processing From: Ian Bicking <ianb@co...> - 2006-06-30 21:30 On Jun 29, 2006, at 11:43 PM, Oliver Bock wrote: > In ThreadedAppServer.py (both CVS HEAD and 0.9.1), whether a thread is > doing work is indicated via a boolean attribute called ".processing" in > some situations, and "._processing" in others. A consequence of this > mix up is that activeThreadCount() always returns zero because it looks > in ._processing whereas threadloop() sets .processing. This means that > ThreadedAppServer will _always_ run only the minimum number of threads. > > Have I got this right? If so, should I just submit a bug report on > sourceforge? I don't think the thread management code was every really filled out or used for anything, so it would not be surprising if there was dead code in there. -- Ian Bicking | ianb@... | http://blog.ianbicking.org

Re: [Webware-discuss] Forwarded requests and exception reports From: Christoph Zwerschke <cito@on...> - 2006-07-05 21:07 Oliver Bock wrote: > While your suggestion gets rid of the extra exception in the error > report, it doesn't solve the problem, which is that forwarding a servlet > fails to forward the call to writeExceptionReport(). (Your above > solution fixes my problem; I'm continuing this thread solely to point > out a bug in WebKit.) Ok, I have now fixed this in the SVN trunk. -- Christoph

Re: [Webware-discuss] thread._processing/processing From: Christoph Zwerschke <cito@on...> - 2006-07-05 21:36 Oliver Bock wrote: > In ThreadedAppServer.py (both CVS HEAD and 0.9.1), whether a thread is > doing work is indicated via a boolean attribute called ".processing" in > some situations, and "._processing" in others. A consequence of this > mix up is that activeThreadCount() always returns zero because it looks > in ._processing whereas threadloop() sets .processing. This means that > ThreadedAppServer will _always_ run only the minimum number of threads. I have fixed this in the trunk as well. -- Christoph

1 message has been excluded from this view by a project administrator.