From: Andrew W. <and...@gm...> - 2011-11-30 12:30:26
|
Hello, Given something like: class X; std::vector<X> * GetXs (); %template(XVector) std::vector<X>; %newobject GetXs; If the following is done in Python: def GetX(): return GetXs().front() GetX().foo() The X returned by GetX is destroyed before foo is called causing a crash because it is owned by the XVector. Changing the example to use pop instead of front would not crash. My question is, how do I tell SWIG to return a copy of X any time an item is obtained from a wrapped std::vector? My objects are very lightweight, so copying is no problem. Regards, Andrew Ward. |
From: Stefan Z. <sz...@gm...> - 2011-11-30 15:47:08
|
I think you can accomplish this by explicitly using the copy constructor for X: def GetX(): xvec = GetXs() return X(xvec.front()) I added the variable xvec to ensure that the result of GetXs() stays in scope. Hope that helps. Stefan On Wed, Nov 30, 2011 at 4:14 AM, Andrew Ward <and...@gm...> wrote: > Hello, > > Given something like: > > class X; > std::vector<X> * GetXs (); > > %template(XVector) std::vector<X>; > %newobject GetXs; > > If the following is done in Python: > > def GetX(): > return GetXs().front() > > GetX().foo() > > The X returned by GetX is destroyed before foo is called causing a crash > because it is owned by the XVector. Changing the example to use pop instead > of front would not crash. > > My question is, how do I tell SWIG to return a copy of X any time an item is > obtained from a wrapped std::vector? My objects are very lightweight, so > copying is no problem. > > > Regards, > > > Andrew Ward. > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure > contains a definitive record of customers, application performance, > security threats, fraudulent activity, and more. Splunk takes this > data and makes sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-novd2d > _______________________________________________ > Swig-user mailing list > Swi...@li... > https://lists.sourceforge.net/lists/listinfo/swig-user |
From: Bob H. <bh...@co...> - 2011-11-30 15:54:50
|
On 11/30/2011 5:14 AM, Andrew Ward wrote: > My question is, how do I tell SWIG to return a copy of X any time an item is > obtained from a wrapped std::vector? My objects are very lightweight, so > copying is no problem. I'm probably doing something wrong, but given your example, it appears to work for me using SWIG 2.0.4 and Python 2.7.2 (debug build): Python 2.7.2 (default, Jun 20 2011, 09:53:03) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import swigtest [44766 refs] >>> from swigtest import GetXs [44768 refs] >>> def GetX(): ... return GetXs().front() ... [44789 refs] >>> GetX().foo() 5 [44884 refs] >>> My 'X' class and GetXs() are defined as: class X { public: X() {} virtual ~X() {} int foo() { return 5; } }; std::vector<X> * GetXs () { return new std::vector<X>(5); } My interface file is: %module (naturalvar="1") swigtest %begin %{ #include <vector> #include "test.h" %} %include python.swg %include <typemaps.i> %include <std_vector.i> %include "test.h" %template(XVector) std::vector<X>; %newobject GetXs; Render me gone, ||| Bob ^(===)^ ---------------------------------oOO--(_)--OOo--------------------------------- "One of the penalties for refusing to participate in politics is that you end up being governed by your inferiors." -- /Plato/ |
From: Andrew W. <and...@gm...> - 2011-11-30 18:49:09
|
Bob Hood <bhood2 <at> comcast.net> writes: > > > On 11/30/2011 5:14 AM, Andrew Ward wrote: > > My question is, how do I tell SWIG to return a copy of X any time an item is > obtained from a wrapped std::vector? My objects are very lightweight, so > copying is no problem. > > > > I'm probably doing something wrong, but given your example, it Hello Bob, Thanks for your response. I think you'll find if you access a member of X from foo() that it will crash, or at least print garbage. For example: class X { public: X() : m_i (123) {} virtual ~X() {} int foo() { return this->m_i; } int m_i; }; Regards, Andrew |
From: Bob H. <bh...@co...> - 2011-11-30 19:10:05
|
> Hello Bob, > > Thanks for your response. I think you'll find if you access a member of X > from foo() that it will crash, or at least print garbage. For example: > > class X > { > public: > X() : m_i (123) {} > virtual ~X() {} > > int foo() { return this->m_i; } > int m_i; > }; Again, it's probably me. I updated my class to match yours: class X { int m_i; public: X() : m_i(123) {} virtual ~X() {} int foo() { return this->m_i; } }; recompiled and tried again: Python 2.7.2 (default, Jun 20 2011, 09:53:03) [MSC v.1500 64 bit (AMD64)] on win 32 Type "help", "copyright", "credits" or "license" for more information. >>> import swigtest [44766 refs] >>> def GetX(): ... return swigtest.GetXs().front() ... [44788 refs] >>> GetX().foo() 123 [44883 refs] >>> The debug version must not be doing something that the release builds are. At any rate, I would have suggested the same thing Stephan did, just that you assign the return value so its reference count is not lowered before you access it. Sorry I couldn't help. :) Render me gone, ||| Bob ^(===)^ ---------------------------------oOO--(_)--OOo--------------------------------- "One of the penalties for refusing to participate in politics is that you end up being governed by your inferiors." -- /Plato/ |
From: Josh C. <jc...@nc...> - 2011-11-30 19:33:51
|
On Wed, 30 Nov 2011, Andrew Ward wrote: > My question is, how do I tell SWIG to return a copy of X any time an > item is obtained from a wrapped std::vector? My objects are very > lightweight, so copying is no problem. I can see a couple of approaches. Wrapping of vectors of built-in types, such as std::vector<int>, does something like this; the wrapped vector does not return references or pointers to its items, but copies in the form of Python integers. You could looke at how SWIG does this, perhaps with the aid of the -E command-line option. Alternatively, it might work to provide or %apply an "out" typemap that makes a copy to the relevant return types (X&, const X&, and perhaps pointers). You could make this apply to all functions returning X&, or, with some care, just to vector methods. However, I am not absolutely certain that this will work with the current implementation of std::vector wrapping. Josh |
From: Andrew W. <and...@gm...> - 2011-12-01 11:39:08
|
Josh Cherry <jcherry <at> ncbi.nlm.nih.gov> writes: > Alternatively, it might work to provide or %apply an "out" typemap that Hi Josh, I have managed to get this working, here is the solution: %define RETURN_COPY_FROM_VECTOR(TYPE) %typemap(out) TYPE & front { $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, SWIG_POINTER_OWN | %newpointer_flags); } %typemap(out) TYPE & back { $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, SWIG_POINTER_OWN | %newpointer_flags); } %typemap(out) TYPE & __getitem__ { $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, SWIG_POINTER_OWN | %newpointer_flags); } %enddef RETURN_COPY_FROM_VECTOR(X) In the process I have found a bug in the Python std::vector implementation. The __getitem__ method has two versions, one takes an index, the other takes a slice object. The version that takes a slice object returns a new std::vector but does not use SWIG_POINTER_OWN and therefore leaks memory. I cannot see a way to patch this up as the typemap: %typemap(out) TYPE & __getitem__ is only applied to the version of __getitem__ that takes an index, so there seems no way to modify the other version through typemaps. Also, if anyone is interested, here is the solution for C# and Java: // For some unknown reason SWIGEXCODE causes a syntax error if used with // RETURN_COPY_FROM_VECTOR So define our own version called SWIGEXCODE3 %define SWIGEXCODE3 "if ($modulePINVOKE.SWIGPendingException.Pending) throw $modulePINVOKE.SWIGPendingException.Retrieve();" %enddef %define RETURN_COPY_FROM_VECTOR(TYPE) %typemap(out) std::vector<TYPE>::const_reference getitem { jresult = (void *) new TYPE (*result); } %typemap(csout, excode=SWIGEXCODE3) std::vector<TYPE>::const_reference getitem { TYPE ret = new TYPE ($imcall, true); $excode return ret; } %enddef RETURN_COPY_FROM_VECTOR(X) %define RETURN_COPY_FROM_VECTOR(TYPE) %typemap(out) std::vector<TYPE>::const_reference get { *(std::vector< TYPE >::value_type **)&jresult = new TYPE (*result); } %typemap(javaout) std::vector<TYPE>::const_reference get { return new TYPE ($jnicall, true); } %enddef RETURN_COPY_FROM_VECTOR(X) |
From: Stefan Z. <sz...@gm...> - 2011-12-01 21:01:04
|
On Thu, Dec 1, 2011 at 3:38 AM, Andrew Ward <and...@gm...> wrote: > In the process I have found a bug in the Python std::vector implementation. > The __getitem__ method has two versions, one takes an index, the other takes > a slice object. The version that takes a slice object returns a new std::vector > but does not use SWIG_POINTER_OWN and therefore leaks memory. Thanks for catching this. I have filed a bug on it, and committed a fix which will go into the next SWIG release. https://sourceforge.net/tracker/?func=detail&aid=3447426&group_id=1645&atid=101645 Stefan |
From: William S F. <ws...@fu...> - 2012-02-03 06:55:01
|
On 01/12/11 11:38, Andrew Ward wrote: > Josh Cherry<jcherry<at> ncbi.nlm.nih.gov> writes: > >> Alternatively, it might work to provide or %apply an "out" typemap that > > Hi Josh, > > I have managed to get this working, here is the solution: > > %define RETURN_COPY_FROM_VECTOR(TYPE) > %typemap(out) TYPE& front { > $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, > SWIG_POINTER_OWN | %newpointer_flags); > } > %typemap(out) TYPE& back { > $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, > SWIG_POINTER_OWN | %newpointer_flags); > } > %typemap(out) TYPE& __getitem__ { > $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, > SWIG_POINTER_OWN | %newpointer_flags); > } > %enddef > > RETURN_COPY_FROM_VECTOR(X) > > > In the process I have found a bug in the Python std::vector implementation. > The __getitem__ method has two versions, one takes an index, the other takes > a slice object. The version that takes a slice object returns a new std::vector > but does not use SWIG_POINTER_OWN and therefore leaks memory. Which version of SWIG are you using because I can't see this problem. This overload of __getitem__: std::vector<_Tp,_Alloc >* __getitem__(PySliceObject *slice); has a %feature("new") for ensuring SWIG_POINTER_OWN gets generated to take ownership of the memory. And indeed the generated code does have SWIG_POINTER_OWN in the wrappers, so there is no memory leak. William |
From: William S F. <ws...@fu...> - 2012-02-03 06:59:55
|
On 03/02/12 06:54, William S Fulton wrote: > On 01/12/11 11:38, Andrew Ward wrote: >> Josh Cherry<jcherry<at> ncbi.nlm.nih.gov> writes: >> >>> Alternatively, it might work to provide or %apply an "out" typemap that >> >> Hi Josh, >> >> I have managed to get this working, here is the solution: >> >> %define RETURN_COPY_FROM_VECTOR(TYPE) >> %typemap(out) TYPE& front { >> $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, >> SWIG_POINTER_OWN | %newpointer_flags); >> } >> %typemap(out) TYPE& back { >> $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, >> SWIG_POINTER_OWN | %newpointer_flags); >> } >> %typemap(out) TYPE& __getitem__ { >> $result = SWIG_NewPointerObj(%new_copy(*$1, $*ltype), $descriptor, >> SWIG_POINTER_OWN | %newpointer_flags); >> } >> %enddef >> >> RETURN_COPY_FROM_VECTOR(X) >> >> >> In the process I have found a bug in the Python std::vector >> implementation. >> The __getitem__ method has two versions, one takes an index, the other >> takes >> a slice object. The version that takes a slice object returns a new >> std::vector >> but does not use SWIG_POINTER_OWN and therefore leaks memory. > Which version of SWIG are you using because I can't see this problem. > This overload of __getitem__: > > std::vector<_Tp,_Alloc >* __getitem__(PySliceObject *slice); > > has a %feature("new") for ensuring SWIG_POINTER_OWN gets generated to > take ownership of the memory. And indeed the generated code does have > SWIG_POINTER_OWN in the wrappers, so there is no memory leak. Not to worry, I found the bug report and I am looking at the version which includes the already committed fix. William |
From: Josh C. <jc...@nc...> - 2011-11-30 19:38:35
|
On Wed, 30 Nov 2011, Bob Hood wrote: > I'm probably doing something wrong, but given your example, it appears > to work for me using SWIG 2.0.4 and Python 2.7.2 (debug build): ... > My interface file is: > %module (naturalvar="1") swigtest > > %begin %{ > #include <vector> > #include "test.h" > %} > > %include python.swg > %include <typemaps.i> > %include <std_vector.i> > > %include "test.h" > > %template(XVector) std::vector<X>; > %newobject GetXs; Does this %newobject come too late to have an effect, replacing the original problem with a memory leak? Josh |
From: Bob H. <bh...@co...> - 2011-11-30 21:52:55
|
On 11/30/2011 12:38 PM, Josh Cherry wrote: > > On Wed, 30 Nov 2011, Bob Hood wrote: > >> I'm probably doing something wrong, but given your example, it appears to >> work for me using SWIG 2.0.4 and Python 2.7.2 (debug build): > ... >> My interface file is: >> %module (naturalvar="1") swigtest >> >> %begin %{ >> #include <vector> >> #include "test.h" >> %} >> >> %include python.swg >> %include <typemaps.i> >> %include <std_vector.i> >> >> %include "test.h" >> >> %template(XVector) std::vector<X>; >> %newobject GetXs; > > Does this %newobject come too late to have an effect, replacing the original > problem with a memory leak? Yes, thanks. That's what I was doing wrong. :) Render me gone, ||| Bob ^(===)^ ---------------------------------oOO--(_)--OOo--------------------------------- "One of the penalties for refusing to participate in politics is that you end up being governed by your inferiors." -- /Plato/ |
From: Andrew W. <and...@gm...> - 2011-12-01 09:07:08
|
Stefan Zager <szager <at> gmail.com> writes: > > I think you can accomplish this by explicitly using the copy constructor for X: > Stefan > Hi Stefan, Thanks for your suggestion. Unfortunately I am designing an API for others to use, and cannot trust them to remember to do this. Regards, Andrew |