#1071 [python] Strange behaviour of temporary objects

[onko]

Hi,
maybe this is me misunderstanding some deep memory allocation issues, but I noticed a quite strange behaviour with SWIG and temporary objects. If this is the case, I'd like to have the matching pointer to the documentation or ask you to consider this as a documentation bug :-)
See attached reduced test case. When calling make.sh, I expect to have the same result (1.0) twice, but I get:
./make.sh
A: 1.0
B: 0.0

I am using debian, python version 2.5, and swig 1.3.40, g++ 4.3.4, though this is not the only configuration showing this behaviour (Kubuntu 9.10 with default packages does the same).

Best regards,

Onno

[onko]

Boiled-down test case for access problem/bug

[William Fulton]

Modifying your test.i file to:

///////////////////////////////////////////////////////////
%module test

%{
#include "test.hh"
%}

%feature("exceptvar") {
printf("%p In variable wrapper ... $symname\n", arg1);
$action;
}
%exception {
printf("%p In function... $symname\n", arg1);
$action;
}

%exception v::v {
$action;
printf("%p In constructor... $symname\n", result);
}
%exception comp::comp {
$action;
printf("%p In constructor... $symname\n", result);
}

%exception v::~v() {
printf("%p In destructor... $symname\n", arg1);
$action;
}
%exception comp::~comp() {
printf("%p In destructor... $symname\n", arg1);
$action;
}
%include "test.hh"
///////////////////////////////////////////////////////////

With just the one line of python code:

print "B:", comp(v(1,1,1), v(2,2,2)).a.x

The output is:

B:0x45ae698 In constructor... new_v
0x45af200 In constructor... new_v
0x45af470 In constructor... new_comp
0x45af200 In destructor... delete_v
0x45ae698 In destructor... delete_v
0x45af470 In variable wrapper ... comp_a_get
0x45af470 In destructor... delete_comp
0x45af470 In variable wrapper ... v_x_get
1.0

So v_x_get uses deleted memory (after delete_comp is called). Note the address of comp is the same as comp::x as x is at the first member variable. Valgrind confirms the problem:

==26786== Invalid read of size 8
==26786== at 0x4036746: _wrap_v_x_get (in /home/william/swig/trunk/Examples/william/temp_access/_test.so)
==26786== by 0x8065B83: PyObject_CallFunction (in /usr/bin/python2.6)
==26786== by 0x80940C6: PyObject_GenericGetAttr (in /usr/bin/python2.6)
==26786== by 0x80B280E: ??? (in /usr/bin/python2.6)
==26786== by 0x80DDD1F: PyEval_EvalFrameEx (in /usr/bin/python2.6)
==26786== by 0x80E2806: PyEval_EvalCodeEx (in /usr/bin/python2.6)
==26786== by 0x80E2906: PyEval_EvalCode (in /usr/bin/python2.6)
==26786== by 0x81005AC: PyRun_FileExFlags (in /usr/bin/python2.6)
==26786== by 0x8100811: PyRun_SimpleFileExFlags (in /usr/bin/python2.6)
==26786== by 0x805DE5B: Py_Main (in /usr/bin/python2.6)
==26786== by 0x805D03A: main (in /usr/bin/python2.6)

I don't have the solution nor the expertise about the python reference counting, so all I can do is confirm the behaviour will provide undefined results, even though I see the correct result of 1.0. However the equivalent code in Java is known to be problematic and the solution is to enforce a reference in the 'comp' instance to 'a' so that it is not garbage collected early.

[Olly Betts]

Still present in git master.

All-in-one testcase test.i:

%module test

%feature("exceptvar") {
printf("%p In variable wrapper ... $symname\n", arg1);
$action;
}
%exception {
printf("%p In function... $symname\n", arg1);
$action;
}

%exception v::v {
$action;
printf("%p In constructor... $symname\n", result);
}
%exception comp::comp {
$action;
printf("%p In constructor... $symname\n", result);
}

%exception v::~v() {
printf("%p In destructor... $symname\n", arg1);
$action;
}
%exception comp::~comp() {
printf("%p In destructor... $symname\n", arg1);
$action;
}
%inline %{
struct v {
    double x,y,z;
    v(double x_, double y_, double z_) : x(x_), y(y_), z(z_) {}
};
struct comp {
    v a, b;
    comp(v a_, v b_) : a(a_), b(b_) {}
};
%}

And runme:

#!/bin/sh -ex
../preinst-swig -c++ -python test.i
g++ -fPIC -shared `python3-config --cflags` test_wrap.cxx -o _test.so
python3 -c 'from test import *;print("B:", comp(v(1,1,1),v(2,2,2)).a.x)'

The output shows two delete_v before comp_a_get and a bogus value:

0x1f4bf40 In constructor... new_v
0x1f4b300 In constructor... new_v
0x20372a0 In constructor... new_comp
0x1f4b300 In destructor... delete_v
0x1f4bf40 In destructor... delete_v
0x20372a0 In variable wrapper ... comp_a_get
0x20372a0 In destructor... delete_comp
0x20372a0 In variable wrapper ... v_x_get
B: 4.0746e-320

[Olly Betts]

Still reproducible with git master shortly after 4.2.0.

Looking at this afresh, the two delete_v calls are OK as those are for the temporary v(1.1.1) and v(2,2,2) objects passed to the comp constructor.

The essential problem here is that comp_a_get returns a Python v object which wraps a pointer to the C++ v a inside the temporary comp object, but then delete_comp is called before we call v_x_get on that v.

For this to work it seems either a reference to comp needs to be kept by the v returned by comp_a_get, or else the C++ v needs to be copied. Copying would be reasonable in this case, but in general it's not ideal - struct v could be very large or not support copying.