#1114 Uninitialization during Py_EndInterpreter causes later fail

open
nobody
python (259)
5
2010-12-10
2010-11-26
Jim
No

This was accidentally added originally as a "Feature Request." Please delete Feature Request 3119739.

This is complicated and has taken me several weeks and many hours to track down to a SWIG issue. Attached is a tar.gz file with commented code that reproduces the problem and a detailed README.txt file.

The issue is related to running Python in an embedded manner using multiple threads with multiple calls to Py_NewInterpreter/Py_EndInterpreter. The first time a call to Py_EndInterpreter is executed the SWIG modules in other interpreters become completely invalid. This seems to even be the case for the *.py generated proxies. I've been baffled by what's going on but I managed to do the following things:

1) Reproduce the problem with a test case with minimal code (attached).
2) Verify it is not a Python bug by running this test case compiled as either a SWIG project OR with a hand coded python bridge. The handcoded python bridge works. The SWIG generated code does not.

Please see the README.txt file in the attached for more details.

This sample is meant to run on Linux and will run against SWIG 2.0.1. I have not tested it against an earlier release.

Discussion

  • Jim

    Jim - 2010-11-26

    Source code, build scripts, run script, to reprodice the problem

     
  • Jim

    Jim - 2010-11-26

    One more note here. The SWIG compiled version will occasionally work so there's clearly a race condition somewhere. The HandCoded version always works.

     
  • Jim

    Jim - 2010-11-27

    I tried a few more things. If you:

    1) Add a couple more short run threads to main.cpp
    2) Compile against the debug python (this requires compiling with -DPy_DEBUG)

    ... you can create an exception in which Python detects extra dereferences from SWIG (the reference count on some SWIG objects goes to -1).

     
  • Olly Betts

    Olly Betts - 2010-12-10
    • summary: Uninitialization durring Py_EndInterpreter causes later fail --> Uninitialization during Py_EndInterpreter causes later fail
    • labels: --> python
     
  • Jim

    Jim - 2011-01-14

    I understand you guys are probably busy, and I'm willing to put more time into this trying to figure it out, but I was hoping to get some feedback from someone that knows the python generation system that might give me some insight or suggestions for avenues of investigation.

    If someone has time and wants to work with me on this, please let me know. I attached well documented code that will reproduce the problem and if you have other requirements, let me know and I'll do what I can.

    Thanks
    Jim

     
  • William Fulton

    William Fulton - 2011-02-04

    Jim, there isn't anyone really knowledgeable on Python who is monitoring these bug reports. All I can suggest you do is analyse the SWIG generated code to see how it differs with your working hand written version. if you can modify the generated code to fix the problem, it will be a lot easier for another SWIG developer to modify SWIG to get the generated code to put in any fixes.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks