Re: Memory leak

[Fernando L. <fer...@gm...>]

Please see my answer below yours.

>
> I'll comment inline
>>
>> The base.hxx file is
>> """
>> #ifndef __BASE__
>> #define __BASE__
>> #include "CXX/Objects.hxx"
>> #include "CXX/Extensions.hxx"
>> #include "iostream"
>> #include "string"
>
> Its odd to use "header" and not <header> why do you do this?
>

Yes, it is a typo ...

>>
>> #define ENDL "\n";
>
> Use std::endl as its in the standard.

For some odd reason, the extension produced with mingw32 as compiler
crashes when std::endl is used. I found that "\n" doesn't, but I don't
know why. If I find some time to review this I will post a comment to
mingw32.

>
>>
>> class Block: public Py::PythonClass< Block >
>> {
>>     Py::Object parent;
>>     Py::String name;
>>     Py::Int number;
>> public:
>>     Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds)
>>     : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>>     {
>>         args.verify_length(3);
>>         name   = args[0];
>>         number = args[1];
>>         parent = args[2];
>>     }
>>
>>     virtual ~Block()
>>     {
>>     }
>>
>>     static void init_type(void)
>>     {
>>         behaviors().name( "Block" );
>>         behaviors().doc( "Clase base para los bloques de simulacion" );
>>         //behaviors().supportSetattr();
>>         behaviors().supportGetattro();
>>         behaviors().supportSetattro();
>>
>>
>>         PYCXX_ADD_NOARGS_METHOD( getFullName, getFullName, "obtener el
>> nombre completo del bloque" );
>>
>>
>>         // Call to make the type ready for use
>>         behaviors().readyType();
>>     }
>>
>>     //static PyObject* tp_new(PyTypeObject *subtype, PyObject *args,
>> PyObject *kwds) {
>>     //    return static_cast<PyObject*>(new Block(subtype,args,kwds));
>>     //}
>>     //static void tp_dealloc(PyObject *obj) {
>>     //    Block* k = static_cast<Block*>(obj);
>>     //    Py::_XDECREF(*(k->parent));
>>     //    Py::_XDECREF(*(k->name));
>>     //    Py::_XDECREF(*(k->number));
>>     //    //delete k;
>>     //    extension_object_deallocator(obj);
>>     //}
>
> PyCXX writes this tp_new and the other necessary functions for you.

Yes, I've commented those lines (it was a test based on some google
search, but it doesn't work).

>
>>
>>     //Py::Object getFullName( const Py::Tuple& args )
>>     Py::Object getFullName( void )
>>     {
>>         std::string r = "";
>>         if( *parent != Py_None )
>
> You can use parent->isNone().

Ok

>
>>             r +=
>> Py::String(PyObject_CallMethod(*parent,"getFullName","")).as_std_string(
>> "utf-8" ) + ".";
>
> You are freely mixing Pyhon C API and PyCXX. This means you have to worry
> about reference counting.
>
> Use the newly added callMemberFunction.
>
> 	Py::String( parent->callMemberFunction( "getFullName" ) )

Ok

>
>> #endif // __BASE__
>> """
>>
>> I'm using trunk version of the CXX files.
>> The software works, creates the blocks, chain them, etc, but creating and
>> destroying 1000000 base.Block objects finish with a 44MB memory usage
>> (when 4MB is expected) (the sizeof(Block) is 40 bytes times 1000000 is
>> 40MB !!!)
>
>
>>
>> I've tryed some variations creating Block as a PythonExtension
>> successfully, an the memory doesn't grow with my 1000000 creation/deletion
>> test process (no memory leak).
>>
>> This leads me to thinks that something in the deallocator for
>> new_style_classes isn't working well.
>> What do you think?
>
>
>
>>
>> ----
>>
>> Also, I've made the test with the simple.cxx module from Demo directory,
>> and the same thing happens (as the object is ligthier you have to
>> create/destroy 10000000, or so, times to see the memory growing).
>> Someone have any clue?
>
> Can you post the python code you used to test simple.cxx which to show the
> memory leak please?

Ok, see below

>
> On what python version and operating system are you doing this work?

Python 2.5.2, Windows XP professional, Mingw32 (gcc 3.4.5 mingw-vista
special r3)

>
> Barry
>
>

Attached is the code that shows a memory increase in simple.cxx (with
minor changes to avoid printing a lot when creating and destroying
objects, and with the addition of a Py::String to the old_style_class,
to make it the same content than new_style_class).

When I run the mytestsimple.py, the python process memory usage starts
at 4072kB, changes to 4080kB after the OLD style class test and then
grows to 28.208kB after the NEW style test (these values are taken
from the Windows Task Manager).

I also run it with the "MemoryValidator" software, and it shows a
HotSpot before the end (after new style test)
Size        Count   HotSpot
 271.872  107  1.06%, 271.872 bytes in 107 allocations.
  271.872   107   PyWrapper_New : [{FUNC}PyWrapper_New Line 0]

 25.165.824  96  98.16%, 25.165.824 bytes in 96 allocations.
  25.165.824   96   PyObject_Dir : [{FUNC}PyObject_Dir Line 0]

>From that I understand that the memory increase is located or related
to a PyObject_Dir function. I only found one at cxxsupport.cxx,
related to the dir() method of Py::Object, but I can't find where it
is used in the application.
-- 
Fernando Libonati
fernando.libonati at gmail.com