Thread: [Pydev-code] PyDev Debugger: Monkeypatch thread.start_new_thread to debug threads?
Brought to you by:
fabioz
From: Leo S. M. <leo...@gm...> - 2010-02-21 17:45:28
|
Hi all, When running the Django server under the PyDev debugger there is a problem: Under Jython the server spawns a worker thread which do all the actual work while the main thread just keeps monitoring the filesystem for changes. But, AFAICS, the pydev debugger only monitors the main thread. I'm playing with monkey-patching thread.start_new_thread to set sys.settrace to the internal settrace function used with PyDev when executing the function in the new thread. Any obvious reason I'm missing why that may be a bad idea? -- Leo Soto M. http://blog.leosoto.com |
From: Fabio Z. <fa...@es...> - 2010-02-21 18:55:14
|
On Sun, Feb 21, 2010 at 2:45 PM, Leo Soto M. <leo...@gm...> wrote: > Hi all, > > When running the Django server under the PyDev debugger there is a > problem: Under Jython the server spawns a worker thread which do all > the actual work while the main thread just keeps monitoring the > filesystem for changes. But, AFAICS, the pydev debugger only monitors > the main thread. > > I'm playing with monkey-patching thread.start_new_thread to set > sys.settrace to the internal settrace function used with PyDev when > executing the function in the new thread. > > Any obvious reason I'm missing why that may be a bad idea? Actually, Pydev should already trigger debugging in any thread (it sets the threading.settrace to do that). I believe django creates a separate process to do that, which is the reason the remote debugger must used for debugging django (a version without the reload feature should already work for all threads). What could be done is that the remote debugger is started and each new process calls pydevd.settrace(suspend=False, traceAllThreads=True), where traceAllThreads is a flag that is still not implemented, which calls threading.settrace(debugger.trace_dispatch) for all new threads -- not only the pydevd_tracing.SetTrace(debugger.trace_dispatch) -- and maybe pass the sys._current_frames to set the tracing for existing threads. If you think that'd be useful, I can create that option... Cheers, Fabio |
From: Leo S. M. <leo...@gm...> - 2010-02-21 19:10:01
|
On Sun, Feb 21, 2010 at 3:55 PM, Fabio Zadrozny <fa...@es...> wrote: > > Actually, Pydev should already trigger debugging in any thread (it > sets the threading.settrace to do that). Yeah, but some code uses thread.start_new_thread directly, which is something not handled by > I believe django creates a separate process to do that, which is the > reason the remote debugger must used for debugging django (a version > without the reload feature should already work for all threads). True, but that's the behavior of Django under CPython. Under Jython it doesn't start a new process, but a new thread. The thread is started using thread.start_new_thread though, which makes it untraceable. > What could be done is that the remote debugger is started and each new > process calls pydevd.settrace(suspend=False, traceAllThreads=True), > where traceAllThreads is a flag that is still not implemented, which > calls threading.settrace(debugger.trace_dispatch) for all new threads > -- not only the pydevd_tracing.SetTrace(debugger.trace_dispatch) -- > and maybe pass the sys._current_frames to set the tracing for existing > threads. > > If you think that'd be useful, I can create that option... Actually, I discovered that the whole thing won't be useful for my use case (debug the Django dev server under Jython with autoreloading enabled). Unfortunately the debugger connection will be lost when the reloading takes place (as it reloads the whole interpreter). However, here is a use case: 1. You have the following program in PyDev: import thread def hello(name): print "Hello", name thread.start_new_thread(hello, ("World",)) 2. You put a debugger marker on the print line. 3. You debug the program, but it never stops. Nor on CPython nor Jython. The fix/hack consisted on the following code (with self.monkeyPatchThread() being invoked after threading.settrace(), in PyDB.run()): def monkeyPatchThread(self): import types try: import thread except: import _thread as thread # Python 3 original_start_new_thread = thread.start_new_thread def patched_start_new_thread(func, *args, **kwargs): def patched_func(*args, **kwargs): import sys pydevd_tracing.SetTrace(self.trace_dispatch) func(*args, **kwargs) original_start_new_thread(patched_func, *args, **kwargs) if self.isJythonClassModule(thread): thread.start_new_thread = staticmethod(patched_start_new_thread) else: thread.start_new_thread = patched_start_new_thread def isJythonClassModule(self, module): try: from java.lang import Class except: return False else: return isinstance(module, Class) But it only works correctly with Jython. With CPython it hits the debugger but then somehow the connection is lost. I'm probably not pushing this forward (since it won't work for the Django server use case), but anyway, that's the code if anyone is interested. -- Leo Soto M. http://blog.leosoto.com |
From: Leo S. M. <leo...@gm...> - 2010-02-21 19:12:13
|
On Sun, Feb 21, 2010 at 4:09 PM, Leo Soto M. <leo...@gm...> wrote: [...] > Under Jython it doesn't start a new process, but a new thread. The > thread is started using thread.start_new_thread though, which makes it > untraceable. For the sake of correctness I must say I shouldn't have used "untraceable" here. There is nothing preventing the thread code to be traced. It's just that the threading.settrace() hook only works for threads started using the threading module, but not for the ones started using the low-level thread module. -- Leo Soto M. http://blog.leosoto.com |
From: Fabio Z. <fa...@es...> - 2010-02-22 19:07:13
|
> On Sun, Feb 21, 2010 at 4:09 PM, Leo Soto M. <leo...@gm...> wrote: > [...] >> Under Jython it doesn't start a new process, but a new thread. The >> thread is started using thread.start_new_thread though, which makes it >> untraceable. > > For the sake of correctness I must say I shouldn't have used > "untraceable" here. There is nothing preventing the thread code to be > traced. It's just that the threading.settrace() hook only works for > threads started using the threading module, but not for the ones > started using the low-level thread module. > An idea might be asking the Django guys to make it more friendly to debuggers so that this can be easier -- maybe even passing a patch to them -- this is one of the things I have in my checklist after the django integration is in place. Best Regards, Fabio |