#535 [INVESTIGATE] Better way for debugging Jython

1.3.9
closed-fixed
nobody
debugger (210)
5
2008-02-05
2007-07-19
paulj
No

System: Win XP
Eclipse: 3.3
PyDev: 1.3.8
Eclipse location: C:\ide\eclipse-for-pydev
Project Location: C:\automation\scripts

---

I have seen this problem with various combinations of Jython (2.1, 2.2rc1, 2.2rc2) and PyDev (1.3.5, 1.3.8).

1. Set a breakpoint in some "main" code in a module.
2. Launch the Jython debugger.
3. The console prints "pydev debugger"
4. A "red square" appears to show that the script is running.
5. The breakpoint is not hit.
6. The pause button is greyed-out.
7. The only available option is to terminate the run.

I believe that the problem is actually with Jython sys.settrace() not working the same as in CPython however I don't know if this is 100% correct and I would welcome a PyDev solution/workaround and at least a second opinion.

I created a PyDev patch that makes the debugger work (i.e. not hang). They way it is currently coded, I wouldn't recommend it to anyone, but it should provide food for thought.

Thanks,

..PJ

Discussion

  • paulj

    paulj - 2007-07-19

    A patch to demonstrate a theory re:Jython debugging

     
  • paulj

    paulj - 2007-07-19
    • labels: --> debugger
    • milestone: --> 738525
     
  • Fabio Zadrozny

    Fabio Zadrozny - 2007-07-21

    Logged In: YES
    user_id=617340
    Originator: NO

    Ok, I've applied your patch (just did a minor refactoring to put that portion of code in the superclass of PyDBDaemonThread). Still, I'd like to know a bit about the theory behind it... It appears that for some reason jython is deadlocking when using the debugger and creating a different PySystemState for each thread solves that (probably a jython bug?)...

    Can you elaborate a bit on it? (its mostly out of curiosity, as I don't know enough about jython internals and why that makes it work).

    Also, just a separate note: I've been able to run the debugger as it was on jython 2.1 (but to run on 2.2rc2 your patch was needed).

    That patch will be available for pydev 1.3.9

     
  • paulj

    paulj - 2007-07-23

    Logged In: YES
    user_id=1849169
    Originator: YES

    Ok, here are some details. I 'm not an expert in Jython so my understanding on this is not 100% either.

    The PySystemState implements the Python 'sys' module and perhaps a few other things. It also contains a 'tracefunc' function reference. The tracefunc reference is null by default and it can be set using sys.settrace. If tracefunc is not null then it represents a function that is called to trace program execution.

    Static fields of PySystemState (args, path, etc) are initialized when Jython starts up and a default instance of PySystemState is created and associated with the main thread. Each thread is assocated with a PySystemState.

    In practice, a new thread inherits the PySystemState of the parent thread. This is a not a copy of the PySystemState but a reference to the same (effectively global) PySystemState .

    SUMMARY: It appears that the current version Jython does not allow per-thread control of tracing as expected (by the PyDev debugger).

    It might be that if the debugger threads call the traceback function that they deadlock somehow - I'm not sure. I noticed that the debugger threads were not progressing then I noticed that they were calling the traceback function.

    The patch that I made forces each of the three spawned debugger threads to have a fresh copy of PySystemState (and separate tracefunc references) and allows tracing to be disabled on the debugger threads. I don't think that this is a fix or even a particularly good work-around but it is what I could do to test my theory without recompiling Jython.

    The patch seems to unblock the debugger, but there may be unknown side-effects of creating multiple copies of PySystemState. For example, the new instances have defaults for some fields.

    If you deploy the patch in an upcoming release, I recommend that you leave this bug open to track the need for a properly understood fix.

     
  • Oti Humbel

    Oti Humbel - 2007-08-14

    Logged In: YES
    user_id=105844
    Originator: NO

    Would it be an option to make the tracefunc and profilefunc instances of PySystemState ThreadLocal ?
    I think we need some stuff on PySystemState really global.
    At least the modules are designed to be, as far as I learned.
    But still I might be wrong.

     
  • Jake

    Jake - 2007-08-19

    Logged In: YES
    user_id=1745484
    Originator: NO

    I had this issue with PyDev 1.3.8, Jython 2.2rc3. I switched back to Jython 2.1 and the issue went away.

    Jake

     
  • Fabio Zadrozny

    Fabio Zadrozny - 2007-09-10

    Logged In: YES
    user_id=617340
    Originator: NO

    Changed title for investigate (as the latest release should work with the fixes).

     
  • Fabio Zadrozny

    Fabio Zadrozny - 2007-09-10
    • milestone: 738525 --> 1.3.9
    • summary: PyDev debugger for Jython hangs --> [INVESTIGATE] Better way for debugging Jython
     
  • Fabio Zadrozny

    Fabio Zadrozny - 2008-02-05
    • status: open --> closed-fixed
     
  • Fabio Zadrozny

    Fabio Zadrozny - 2008-02-05

    Logged In: YES
    user_id=617340
    Originator: NO

    This was fixed at 1.3.11.