#535 [INVESTIGATE] Better way for debugging Jython

[paulj]

System: Win XP
Eclipse: 3.3
PyDev: 1.3.8
Eclipse location: C:\ide\eclipse-for-pydev
Project Location: C:\automation\scripts

---

I have seen this problem with various combinations of Jython (2.1, 2.2rc1, 2.2rc2) and PyDev (1.3.5, 1.3.8).

1. Set a breakpoint in some "main" code in a module.
2. Launch the Jython debugger.
3. The console prints "pydev debugger"
4. A "red square" appears to show that the script is running.
5. The breakpoint is not hit.
6. The pause button is greyed-out.
7. The only available option is to terminate the run.

I believe that the problem is actually with Jython sys.settrace() not working the same as in CPython however I don't know if this is 100% correct and I would welcome a PyDev solution/workaround and at least a second opinion.

I created a PyDev patch that makes the debugger work (i.e. not hang). They way it is currently coded, I wouldn't recommend it to anyone, but it should provide food for thought.

Thanks,

..PJ

[paulj]

A patch to demonstrate a theory re:Jython debugging

[Fabio Zadrozny]

Logged In: YES
user_id=617340
Originator: NO

Ok, I've applied your patch (just did a minor refactoring to put that portion of code in the superclass of PyDBDaemonThread). Still, I'd like to know a bit about the theory behind it... It appears that for some reason jython is deadlocking when using the debugger and creating a different PySystemState for each thread solves that (probably a jython bug?)...

Can you elaborate a bit on it? (its mostly out of curiosity, as I don't know enough about jython internals and why that makes it work).

Also, just a separate note: I've been able to run the debugger as it was on jython 2.1 (but to run on 2.2rc2 your patch was needed).

That patch will be available for pydev 1.3.9

[paulj]

Logged In: YES
user_id=1849169
Originator: YES

Ok, here are some details. I 'm not an expert in Jython so my understanding on this is not 100% either.

The PySystemState implements the Python 'sys' module and perhaps a few other things. It also contains a 'tracefunc' function reference. The tracefunc reference is null by default and it can be set using sys.settrace. If tracefunc is not null then it represents a function that is called to trace program execution.

Static fields of PySystemState (args, path, etc) are initialized when Jython starts up and a default instance of PySystemState is created and associated with the main thread. Each thread is assocated with a PySystemState.

In practice, a new thread inherits the PySystemState of the parent thread. This is a not a copy of the PySystemState but a reference to the same (effectively global) PySystemState .

SUMMARY: It appears that the current version Jython does not allow per-thread control of tracing as expected (by the PyDev debugger).

It might be that if the debugger threads call the traceback function that they deadlock somehow - I'm not sure. I noticed that the debugger threads were not progressing then I noticed that they were calling the traceback function.

The patch that I made forces each of the three spawned debugger threads to have a fresh copy of PySystemState (and separate tracefunc references) and allows tracing to be disabled on the debugger threads. I don't think that this is a fix or even a particularly good work-around but it is what I could do to test my theory without recompiling Jython.

The patch seems to unblock the debugger, but there may be unknown side-effects of creating multiple copies of PySystemState. For example, the new instances have defaults for some fields.

If you deploy the patch in an upcoming release, I recommend that you leave this bug open to track the need for a properly understood fix.

[Oti Humbel]

Logged In: YES
user_id=105844
Originator: NO

Would it be an option to make the tracefunc and profilefunc instances of PySystemState ThreadLocal ?
I think we need some stuff on PySystemState really global.
At least the modules are designed to be, as far as I learned.
But still I might be wrong.

[Jake]

Logged In: YES
user_id=1745484
Originator: NO

I had this issue with PyDev 1.3.8, Jython 2.2rc3. I switched back to Jython 2.1 and the issue went away.

Jake

[Fabio Zadrozny]

Logged In: YES
user_id=617340
Originator: NO

Changed title for investigate (as the latest release should work with the fixes).

[Fabio Zadrozny]

Logged In: YES
user_id=617340
Originator: NO

This was fixed at 1.3.11.