#730 segfault with iterators in two modules

open
python (259)
5
2012-12-21
2006-08-17
No

when I import two modules with iterators I get segfault
simple example w/ runme.sh contains testcase

swig is from CVS

$ swig -version

SWIG Version 1.3.30

$ g++ -v
Reading specs from
/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/specs
Configured with:
/var/tmp/portage/gcc-3.4.6-r1/work/gcc-3.4.6/configure
--prefix=/usr
--bindir=/usr/i686-pc-linux-gnu/gcc-bin/3.4.6
--includedir=/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/include
--datadir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6
--mandir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/man
--infodir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/info
--with-gxx-include-dir=/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/include/g++-v3
--host=i686-pc-linux-gnu --build=i686-pc-linux-gnu
--disable-altivec --enable-nls
--without-included-gettext --with-system-zlib
--disable-checking --disable-werror
--disable-libunwind-exceptions --disable-multilib
--disable-libgcj --enable-languages=c,c++
--enable-shared --enable-threads=posix
--enable-__cxa_atexit --enable-clocale=gnu
Thread model: posix
gcc version 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0,
pie-8.7.9)

$ python2.4 -V
Python 2.4.3

works on Debian Sarge also

Discussion

  • Joseph Winston

    Joseph Winston - 2006-08-18

    Logged In: YES
    user_id=398495

    From:     josephwinston@mac.com
    Subject:    std\_map.i and RTLD\_GLOBAL for python modules
    Date:   August 2, 2006 9:46:27 AM CDT
    To:       swig-user@lists.sourceforge.net
    

    I'm having problems finding the correct options that are
    needed to
    compile and link a python module on linux that throws an
    exception in
    one dynamically loaded module which is caught in another
    dynamically
    loaded module. I've seen this issue on two different linux
    distributions: gentoo, gcc 3.4.6, python 2.4.3 or ubuntu dapper,
    python 2.4.3, gcc 4.0.3 which both are using the CVS version
    of SWIG.
    This problem can be fixed with setting the RTLD_GLOBAL flag
    on dlopen
    but I'd like to avoid this solution.

    This type of problem occurs when one is using std_map.i
    Consider the
    following interface:

    %module foo

    %{
    #include "foo.h"
    %}

    %include std_string.i
    %include std_pair.i
    %include std_map.i

    %naturalvar FooMap;
    %template() std::pair< std::string, Foo >;
    %template(FooMap) std::map< std::string, Foo >;

    %include "foo.h"

    with Foo defined as:

    class Foo
    {
    public:
    Foo() {}
    ~Foo() {}
    };

    And another interface defined as:

    %module bar

    %{
    #include "bar.h"
    %}

    %include std_string.i
    %include std_pair.i
    %include std_map.i

    %include "bar.h"

    with bar.h containing

    class Bar
    {
    public:
    Bar() {}
    ~Bar() {}
    };

    Compiled as follows:

    $ swig -c++ -python foo.i
    $ g++ -c -fpic foo_wrap.cxx
    -I/home/jody/Tools/i686-unknown-linux/include/python2.4
    -I/home/jody/Tools/i686-unknown-linux/lib/python2.4/config
    $ gcc -shared -Xlinker -export-dynamic foo_wrap.o -o _foo.so
    $ swig -c++ -python bar.i
    $ g++ -c -fpic bar_wrap.cxx
    -I/home/jody/Tools/i686-unknown-linux/include/python2.4
    -I/home/jody/Tools/i686-unknown-linux/lib/python2.4/config
    $ gcc -shared -Xlinker -export-dynamic bar_wrap.o -o _bar.so

    The problem is seen in the following python example:

    import foo
    import bar

    m = foo.FooMap()
    m['cat'] = foo.Foo()
    m['dog'] = foo.Foo()

    for i, k in m.iteritems():
    print i, k

    The std_map implementation throws a swig::stop_iteration when it
    reaches the end of the std::map and this is caught not in
    foo's code
    but in bar's.

    Here's the traceback from gdb:

    #0 0xffffe410 in __kernel_vsyscall ()
    #1 0xb7d4c3b1 in raise () from /lib/tls/libc.so.6
    #2 0xb7d4ddcd in abort () from /lib/tls/libc.so.6
    #3 0xb7f20614 in __gnu_cxx::__verbose_terminate_handler () at
    vterminate.cc:97
    #4 0xb7f1e005 in __cxxabiv1::__terminate (handler=0xb7f204c0
    <__gnu_cxx::__verbose_terminate_handler()>) at
    eh_terminate.cc:43
    #5 0xb7f1e042 in std::terminate () at eh_terminate.cc:53
    #6 0xb7f1e17a in __cxa_throw (obj=0x81b0a00,
    tinfo=0xb7c09754, dest=0)
    at eh_throw.cc:77
    #7 0xb7bfe1ac in
    swig::PySwigIteratorClosed_T<std::_Rb_tree_iterator<std::pair<std::string
    const, Foo> >, std::pair<std::string const, Foo>,
    swig::from_oper<std::pair<std::string const, Foo> > >::value
    () from ./_foo.so
    #8 0xb7bd5404 in swig::PySwigIterator::next () from ./_bar.so
    #9 0xb7bd306a in _wrap_PySwigIterator_next () from ./_bar.so
    #10 0x0805cd07 in PyObject_Call (func=0xda6, arg=0x0, kw=0x0) at
    abstract.c:1795
    #11 0x080bab51 in PyEval_EvalFrame (f=0x816bdec) at ceval.c:3840
    #12 0x080be28b in PyEval_EvalCodeEx (co=0xb7c9e3a0, globals=0x0,
    locals=0x6, args=0x0, argcount=1, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0)
    at ceval.c:2736
    #13 0x0810a125 in function_call (func=0xb7ca156c,
    arg=0xb7c8ffcc,
    kw=0x0) at funcobject.c:548
    #14 0x0805cd07 in PyObject_Call (func=0xda6, arg=0x0, kw=0x0) at
    abstract.c:1795
    #15 0x08065b1e in instancemethod_call (func=0x0,
    arg=0xb7c8ffcc, kw=0x0)
    at classobject.c:2447
    #16 0x0805cd07 in PyObject_Call (func=0xda6, arg=0x0, kw=0x0) at
    abstract.c:1795
    #17 0x0808ddfe in call_method (o=0xb7ca25ac, name=0x0,
    nameobj=0x81498b8, format=0x8118457 "()") at typeobject.c:923
    #18 0x0809528c in slot_tp_iternext (self=0x0) at
    typeobject.c:4702
    #19 0x080b7a11 in PyEval_EvalFrame (f=0x818973c) at ceval.c:2121
    #20 0x080be28b in PyEval_EvalCodeEx (co=0xb7c88e20, globals=0x0,
    locals=0x6, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0)
    at ceval.c:2736
    #21 0x080be4e9 in PyEval_EvalCode (co=0x0, globals=0x0,
    locals=0x0) at
    ceval.c:484
    #22 0x080d8b6d in PyImport_ExecCodeModuleEx (name=0xbfe94ce0
    "runme",
    co=0xb7c88e20, pathname=0xbfe94860 "runme.py") at import.c:631
    #23 0x080d8f7b in load_source_module (name=0xbfe94ce0 "runme",
    pathname=0xbfe94860 "runme.py", fp=0x815d090) at import.c:909
    #24 0x080da17a in load_module (name=0xbfe94ce0 "runme", fp=0x6,
    buf=0xbfe94860 "runme.py", type=1, loader=0xda6) at
    import.c:1672
    #25 0x080da5c7 in import_submodule (mod=0x8126be0,
    subname=0xbfe94ce0
    "runme", fullname=0xbfe94ce0 "runme") at import.c:2266
    #26 0x080da7c4 in load_next (mod=0x8126be0, altmod=0x8126be0,
    p_name=0xbfe94ce0, buf=0xbfe94ce0 "runme",
    p_buflen=0xbfe94cdc) at
    import.c:2086
    #27 0x080dacac in import_module_ex (name=0x0,
    globals=0xbfe94cd8,
    locals=0xb7cfc824, fromlist=0x8126be0) at import.c:1921
    #28 0x080dafb4 in PyImport_ImportModuleEx (name=0xb7c8fd34
    "runme",
    globals=0xb7cfc824, locals=0x0, fromlist=0x0) at import.c:1962
    #29 0x080ae861 in builtin___import__ (self=0x0, args=0x0) at
    bltinmodule.c:45
    #30 0x0805cd07 in PyObject_Call (func=0xda6, arg=0x0, kw=0x0) at
    abstract.c:1795
    #31 0x080b661b in PyEval_CallObjectWithKeywords (func=0x0,
    arg=0xb7d02b44, kw=0xb7cebd6c) at ceval.c:3430
    #32 0x080b8495 in PyEval_EvalFrame (f=0x817eb2c) at ceval.c:2020
    #33 0x080be28b in PyEval_EvalCodeEx (co=0xb7c88c60, globals=0x0,
    locals=0x6, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0)
    at ceval.c:2736
    #34 0x080be4e9 in PyEval_EvalCode (co=0x0, globals=0x0,
    locals=0x0) at
    ceval.c:484
    #35 0x080e465c in PyRun_InteractiveOneFlags (fp=0x0,
    filename=0x8110008
    "<stdin>", flags=0xbfe95578) at pythonrun.c:1265
    #36 0x080e47d0 in PyRun_InteractiveLoopFlags (fp=0xb7e38760,
    filename=0x8110008 "<stdin>", flags=0xbfe95578) at
    pythonrun.c:695
    #37 0x080e4d32 in PyRun_AnyFileExFlags (fp=0xb7e38760,
    filename=0x8110008 "<stdin>", closeit=0, flags=0xbfe95578) at
    pythonrun.c:658
    #38 0x08055b93 in Py_Main (argc=0, argv=0xbfe956a4) at
    main.c:493
    #39 0x080550be in main (argc=0, argv=0x0) at python.c:23

    This problem can be fixed by changing the dlopen flags as
    follows:

    import sys
    import dl
    sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL)

    import foo
    import bar

    m = foo.FooMap()
    m['cat'] = foo.Foo()
    m['dog'] = foo.Foo()

    for i, k in m.iteritems():
    print i, k

    If this is the suggested way to resolve this issue, perhaps
    this should
    be in the documentation.

     
  • Maks N. Polunin

    Maks N. Polunin - 2006-08-19

    Logged In: YES
    user_id=705410

    RTLD_GLOBAL helps because stop_iteration related symbols now
    are shared between modules, but it still is thrown from one
    module and catched in another. I suppose there can be bad
    side effects w/ this flag set.

    The problem w/ iterators occurs because of the way swig
    registers symbols - if symbols is already registered in some
    loaded module, swig just overwrites its client data w/ new one.

    I think updating this behaviour and registering interal
    symbols (like PySwigIterator) in the same module (instead of
    overwriting in loaded one) could be the solution.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks