cxx-users — Discuss the use and improvement of PyCXX

Re: Derived class getattro

[Barry S. <ba...@ba...>]

On 10 Apr 2010, at 19:34, Fernando Libonati wrote:

> In cxx_extensions.cxx
> 
> extern "C" PyObject *getattro_handler( PyObject *self, PyObject *name )
> {
>     try
>     {
>         PythonExtensionBase *p = getPythonExtensionBase( self );
>         return new_reference_to( p->getattro( Py::String( name ) ) );
>     }
>     catch( Py::Exception &)
>     {
>         return NULL;    // indicate error
>     }
> }
> 
> In a derived class this always call the getattro of the base class, and never get the derived class attributes visible.
> How can I access a derived class attro?
> 
> I've attached the test system, base.h and base.cxx.
> When compiled, it is possible to create a g = base.Gain("b",1,None,2.43) object, and access name,number and parent attributes, but doesn't reach the gain attribute.
> Python says
> """Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AttributeError: 'Gain' object has no attribute 'gain'
> """
> 
> It would be possible to call first the derived class method, and inside it call the base class method (under the user responsibility)?

You cannot use C++ derivation to create a set of Python types.

When Block is setup there are a lot of python data structures created and initialise to make Block available to Python.

Gain has none of these python data structures.

If you want  a hierarchy of classes then only the leafs can be turned into Python classes.

Barry

Fwd: Derived class getattro

[Fernando L. <fer...@gm...>]

A few minutes after this post, I've realized that Block::init_type() must be
called in module initialization, and not in Gain::init_type(), but the same
thing happens.

Fernando

---------- Forwarded message ----------
From: Fernando Libonati <fer...@gm...>
Date: 2010/4/10
Subject: Derived class getattro
To: Discuss PyCXX use and improvement <cxx...@li...>


In cxx_extensions.cxx

....

In a derived class this always call the getattro of the base class, and
never get the derived class attributes visible.
How can I access a derived class attro?

I've attached the test system, base.h and base.cxx.
When compiled, it is possible to create a g = base.Gain("b",1,None,2.43)
object, and access name,number and parent attributes, but doesn't reach the
gain attribute.
Python says
"""Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Gain' object has no attribute 'gain'
"""

It would be possible to call first the derived class method, and inside it
call the base class method (under the user responsibility)?

-- 
Fernando Libonati
fernando.libonati at gmail.com



-- 
Fernando Libonati
fernando.libonati at gmail.com

Re: Memory leak

[Fernando L. <fer...@gm...>]

Excellent !

I've updated my trunk copy to r232 and now the test with simple.cxx works
fine without any leak.
(I changed the PYCXX_ADD_..._METHOD sentences to the new syntax).

The changes in ExtensionType.hxx are very subtle, and I can imagine the
difficulty to find them.

Again, thank you for the fast response.

Fernando


2010/4/10 Barry Scott <ba...@ba...>

>
> On 9 Apr 2010, at 22:07, Barry Scott wrote:
>
> I have reproduced the mem leak on the mac and fedora.
>
> I'm looking into the problem.
>
> Barry
>
>
> This was hard to debug using any build of python with
> its own memory allocator enabled. This prevents malloc
> debug tools from finding the  real location of the leak.
>
> I build python with --without-pymalloc which tells python
> to use malloc and free.
>
> valgrind then found the memory leak and a coding error.
>
> This is fixed in r232. Please test in your setup and confirm
> the memory leak is fixed.
>
> Barry
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> CXX-Users mailing list
> CXX...@li...
> https://lists.sourceforge.net/lists/listinfo/cxx-users
>
>


-- 
Fernando Libonati
fernando.libonati at gmail.com

Re: Memory leak

[Barry S. <ba...@ba...>]

On 9 Apr 2010, at 22:07, Barry Scott wrote:

> I have reproduced the mem leak on the mac and fedora.
> 
> I'm looking into the problem.
> 
> Barry

This was hard to debug using any build of python with
its own memory allocator enabled. This prevents malloc
debug tools from finding the  real location of the leak.

I build python with --without-pymalloc which tells python
to use malloc and free.

valgrind then found the memory leak and a coding error.

This is fixed in r232. Please test in your setup and confirm
the memory leak is fixed.

Barry

Re: Memory leak

[Barry S. <ba...@ba...>]

I have reproduced the mem leak on the mac and fedora.

I'm looking into the problem.

Barry


On 7 Apr 2010, at 16:17, Fernando Libonati wrote:

> Please see my answer below yours.
> 
>> 
>> I'll comment inline
>>> 
>>> The base.hxx file is
>>> """
>>> #ifndef __BASE__
>>> #define __BASE__
>>> #include "CXX/Objects.hxx"
>>> #include "CXX/Extensions.hxx"
>>> #include "iostream"
>>> #include "string"
>> 
>> Its odd to use "header" and not <header> why do you do this?
>> 
> 
> Yes, it is a typo ...
> 
>>> 
>>> #define ENDL "\n";
>> 
>> Use std::endl as its in the standard.
> 
> For some odd reason, the extension produced with mingw32 as compiler
> crashes when std::endl is used. I found that "\n" doesn't, but I don't
> know why. If I find some time to review this I will post a comment to
> mingw32.
> 
>> 
>>> 
>>> class Block: public Py::PythonClass< Block >
>>> {
>>>    Py::Object parent;
>>>    Py::String name;
>>>    Py::Int number;
>>> public:
>>>    Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds)
>>>    : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>>>    {
>>>        args.verify_length(3);
>>>        name   = args[0];
>>>        number = args[1];
>>>        parent = args[2];
>>>    }
>>> 
>>>    virtual ~Block()
>>>    {
>>>    }
>>> 
>>>    static void init_type(void)
>>>    {
>>>        behaviors().name( "Block" );
>>>        behaviors().doc( "Clase base para los bloques de simulacion" );
>>>        //behaviors().supportSetattr();
>>>        behaviors().supportGetattro();
>>>        behaviors().supportSetattro();
>>> 
>>> 
>>>        PYCXX_ADD_NOARGS_METHOD( getFullName, getFullName, "obtener el
>>> nombre completo del bloque" );
>>> 
>>> 
>>>        // Call to make the type ready for use
>>>        behaviors().readyType();
>>>    }
>>> 
>>>    //static PyObject* tp_new(PyTypeObject *subtype, PyObject *args,
>>> PyObject *kwds) {
>>>    //    return static_cast<PyObject*>(new Block(subtype,args,kwds));
>>>    //}
>>>    //static void tp_dealloc(PyObject *obj) {
>>>    //    Block* k = static_cast<Block*>(obj);
>>>    //    Py::_XDECREF(*(k->parent));
>>>    //    Py::_XDECREF(*(k->name));
>>>    //    Py::_XDECREF(*(k->number));
>>>    //    //delete k;
>>>    //    extension_object_deallocator(obj);
>>>    //}
>> 
>> PyCXX writes this tp_new and the other necessary functions for you.
> 
> Yes, I've commented those lines (it was a test based on some google
> search, but it doesn't work).
> 
>> 
>>> 
>>>    //Py::Object getFullName( const Py::Tuple& args )
>>>    Py::Object getFullName( void )
>>>    {
>>>        std::string r = "";
>>>        if( *parent != Py_None )
>> 
>> You can use parent->isNone().
> 
> Ok
> 
>> 
>>>            r +=
>>> Py::String(PyObject_CallMethod(*parent,"getFullName","")).as_std_string(
>>> "utf-8" ) + ".";
>> 
>> You are freely mixing Pyhon C API and PyCXX. This means you have to worry
>> about reference counting.
>> 
>> Use the newly added callMemberFunction.
>> 
>> 	Py::String( parent->callMemberFunction( "getFullName" ) )
> 
> Ok
> 
>> 
>>> #endif // __BASE__
>>> """
>>> 
>>> I'm using trunk version of the CXX files.
>>> The software works, creates the blocks, chain them, etc, but creating and
>>> destroying 1000000 base.Block objects finish with a 44MB memory usage
>>> (when 4MB is expected) (the sizeof(Block) is 40 bytes times 1000000 is
>>> 40MB !!!)
>> 
>> 
>>> 
>>> I've tryed some variations creating Block as a PythonExtension
>>> successfully, an the memory doesn't grow with my 1000000 creation/deletion
>>> test process (no memory leak).
>>> 
>>> This leads me to thinks that something in the deallocator for
>>> new_style_classes isn't working well.
>>> What do you think?
>> 
>> 
>> 
>>> 
>>> ----
>>> 
>>> Also, I've made the test with the simple.cxx module from Demo directory,
>>> and the same thing happens (as the object is ligthier you have to
>>> create/destroy 10000000, or so, times to see the memory growing).
>>> Someone have any clue?
>> 
>> Can you post the python code you used to test simple.cxx which to show the
>> memory leak please?
> 
> Ok, see below
> 
>> 
>> On what python version and operating system are you doing this work?
> 
> Python 2.5.2, Windows XP professional, Mingw32 (gcc 3.4.5 mingw-vista
> special r3)
> 
>> 
>> Barry
>> 
>> 
> 
> Attached is the code that shows a memory increase in simple.cxx (with
> minor changes to avoid printing a lot when creating and destroying
> objects, and with the addition of a Py::String to the old_style_class,
> to make it the same content than new_style_class).
> 
> When I run the mytestsimple.py, the python process memory usage starts
> at 4072kB, changes to 4080kB after the OLD style class test and then
> grows to 28.208kB after the NEW style test (these values are taken
> from the Windows Task Manager).
> 
> I also run it with the "MemoryValidator" software, and it shows a
> HotSpot before the end (after new style test)
> Size        Count   HotSpot
> 271.872  107  1.06%, 271.872 bytes in 107 allocations.
>  271.872   107   PyWrapper_New : [{FUNC}PyWrapper_New Line 0]
> 
> 25.165.824  96  98.16%, 25.165.824 bytes in 96 allocations.
>  25.165.824   96   PyObject_Dir : [{FUNC}PyObject_Dir Line 0]
> 
>> From that I understand that the memory increase is located or related
> to a PyObject_Dir function. I only found one at cxxsupport.cxx,
> related to the dir() method of Py::Object, but I can't find where it
> is used in the application.
> -- 
> Fernando Libonati
> fernando.libonati at gmail.com
> <simple.cxx><mytestsimple.py>------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev_______________________________________________
> CXX-Users mailing list
> CXX...@li...
> https://lists.sourceforge.net/lists/listinfo/cxx-users

Re: Call new type method

[Fernando L. <fer...@gm...>]

Excellent explanation !

I promise to start a deeper study of the code this weekend in order to
minimize bothering you (but not stop, ha ha).

Fernando

2010/4/9 Barry Scott <ba...@ba...>

>
> On 9 Apr 2010, at 15:27, Fernando Libonati wrote:
>
> Two questions ...
> a) How can I pass a pointer to a new type object?
>
>
> Can you get a pointer from a Py::Object? Yes
>
> b) Can I access methods from this pointer?
>
>
> yes. But it depends on the method. If its
> a C++ method then its trivia. If its a python method
> on the object that may have been overridden in
> a derived class then you need to use a PyCXX helper.
>
> The code below depends on PyCXX trunk that I will release
> as 6.2.0 soon.
>
>
>
> Like in
>
> class Block : Public PythonClass<Block> {
>      Block *parent;
>      Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds):
>      : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>     {
>         args.verify_length(1);
>        // Is this possible ??? or something like this
>         parent  = (Block*) args[0];
>
>
> This does not work on lots of levels.
> a) Py::Object just holds a pointer to the PyObject.
> b) You cannot cast a PyObject as its not part of the C++ class.
>      It contains a pointer to the C++ class.
>
> Use this to make sure you really have a Block:
>
> Py::PythonClassObject<Block> py_block( args[0] );
>
> Now you can get the C++ object pointer:
>
> Block *p_block = py_block.getCxxObject();
>
> To call getFullName you have a couple of choices:
>
> Py::String s( p_block->getFullName() );
>
> or
>
> Py::TupleN args;
> Py::String py_block.callMemberFunction( "getFullName", args );
>
>
>     }
> ...
> ...
>     Py::Object getFullName( void )
>     {
>         std::string r = "";
>         if( ! parent->isNone() )
>             r += Py::String( parent->getFullName()).as_std_string( "utf-8"
> ) + ".";
>
> /* I want to access parent's methods directly to avoid the overload of
> calling
>    callMemberFunction("getFullName") and many others that I need.
>    Is it possible?
>    Is callMemberFunction restricted to the exposed ones?
> */
>         r += name.as_std_string( "utf-8" );
>         return Py::String(r);
>     }
> ...
> ...
> };
>
>
> Barry
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> CXX-Users mailing list
> CXX...@li...
> https://lists.sourceforge.net/lists/listinfo/cxx-users
>
>


-- 
Fernando Libonati
fernando.libonati at gmail.com

Re: Call new type method

[Barry S. <ba...@ba...>]

On 9 Apr 2010, at 15:27, Fernando Libonati wrote:

> Two questions ...
> a) How can I pass a pointer to a new type object?

Can you get a pointer from a Py::Object? Yes

> b) Can I access methods from this pointer?

yes. But it depends on the method. If its
a C++ method then its trivia. If its a python method
on the object that may have been overridden in
a derived class then you need to use a PyCXX helper.

The code below depends on PyCXX trunk that I will release
as 6.2.0 soon.


> 
> Like in
> 
> class Block : Public PythonClass<Block> {
>      Block *parent;
>      Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds):
>      : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>     {
>         args.verify_length(1);
>        // Is this possible ??? or something like this
>         parent  = (Block*) args[0]; 

This does not work on lots of levels.
a) Py::Object just holds a pointer to the PyObject.
b) You cannot cast a PyObject as its not part of the C++ class.
     It contains a pointer to the C++ class.

Use this to make sure you really have a Block:

	Py::PythonClassObject<Block> py_block( args[0] );

Now you can get the C++ object pointer:

	Block *p_block = py_block.getCxxObject();

To call getFullName you have a couple of choices:

	Py::String s( p_block->getFullName() );

or

	Py::TupleN args;
	Py::String py_block.callMemberFunction( "getFullName", args );


>     }
> ...
> ...
>     Py::Object getFullName( void )
>     {
>         std::string r = "";
>         if( ! parent->isNone() )
>             r += Py::String( parent->getFullName()).as_std_string( "utf-8" ) + ".";
> 
> /* I want to access parent's methods directly to avoid the overload of calling
>    callMemberFunction("getFullName") and many others that I need.
>    Is it possible? 
>    Is callMemberFunction restricted to the exposed ones?
> */
>         r += name.as_std_string( "utf-8" );
>         return Py::String(r);
>     }
> ...
> ...
> };
> 

Barry

Call new type method

[Fernando L. <fer...@gm...>]

Two questions ...
a) How can I pass a pointer to a new type object?
b) Can I access methods from this pointer?

Like in

class Block : Public PythonClass<Block> {
     Block *parent;
     Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds):
     : Py::PythonClass< Block >::PythonClass( self, args, kwds )
    {
        args.verify_length(1);
       // Is this possible ??? or something like this
        parent  = (Block*) args[0];
    }
...
...
    Py::Object getFullName( void )
    {
        std::string r = "";
        if( ! parent->isNone() )
            r += Py::String( parent->getFullName()).as_std_string( "utf-8" )
+ ".";

/* I want to access parent's methods directly to avoid the overload of
calling
   callMemberFunction("getFullName") and many others that I need.
   Is it possible?
   Is callMemberFunction restricted to the exposed ones?
*/
        r += name.as_std_string( "utf-8" );
        return Py::String(r);
    }
...
...
};

-- 
Fernando Libonati
fernando.libonati at gmail.com

Re: Memory leak

[Fernando L. <fer...@gm...>]

Please see my answer below yours.

>
> I'll comment inline
>>
>> The base.hxx file is
>> """
>> #ifndef __BASE__
>> #define __BASE__
>> #include "CXX/Objects.hxx"
>> #include "CXX/Extensions.hxx"
>> #include "iostream"
>> #include "string"
>
> Its odd to use "header" and not <header> why do you do this?
>

Yes, it is a typo ...

>>
>> #define ENDL "\n";
>
> Use std::endl as its in the standard.

For some odd reason, the extension produced with mingw32 as compiler
crashes when std::endl is used. I found that "\n" doesn't, but I don't
know why. If I find some time to review this I will post a comment to
mingw32.

>
>>
>> class Block: public Py::PythonClass< Block >
>> {
>>     Py::Object parent;
>>     Py::String name;
>>     Py::Int number;
>> public:
>>     Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds)
>>     : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>>     {
>>         args.verify_length(3);
>>         name   = args[0];
>>         number = args[1];
>>         parent = args[2];
>>     }
>>
>>     virtual ~Block()
>>     {
>>     }
>>
>>     static void init_type(void)
>>     {
>>         behaviors().name( "Block" );
>>         behaviors().doc( "Clase base para los bloques de simulacion" );
>>         //behaviors().supportSetattr();
>>         behaviors().supportGetattro();
>>         behaviors().supportSetattro();
>>
>>
>>         PYCXX_ADD_NOARGS_METHOD( getFullName, getFullName, "obtener el
>> nombre completo del bloque" );
>>
>>
>>         // Call to make the type ready for use
>>         behaviors().readyType();
>>     }
>>
>>     //static PyObject* tp_new(PyTypeObject *subtype, PyObject *args,
>> PyObject *kwds) {
>>     //    return static_cast<PyObject*>(new Block(subtype,args,kwds));
>>     //}
>>     //static void tp_dealloc(PyObject *obj) {
>>     //    Block* k = static_cast<Block*>(obj);
>>     //    Py::_XDECREF(*(k->parent));
>>     //    Py::_XDECREF(*(k->name));
>>     //    Py::_XDECREF(*(k->number));
>>     //    //delete k;
>>     //    extension_object_deallocator(obj);
>>     //}
>
> PyCXX writes this tp_new and the other necessary functions for you.

Yes, I've commented those lines (it was a test based on some google
search, but it doesn't work).

>
>>
>>     //Py::Object getFullName( const Py::Tuple& args )
>>     Py::Object getFullName( void )
>>     {
>>         std::string r = "";
>>         if( *parent != Py_None )
>
> You can use parent->isNone().

Ok

>
>>             r +=
>> Py::String(PyObject_CallMethod(*parent,"getFullName","")).as_std_string(
>> "utf-8" ) + ".";
>
> You are freely mixing Pyhon C API and PyCXX. This means you have to worry
> about reference counting.
>
> Use the newly added callMemberFunction.
>
> 	Py::String( parent->callMemberFunction( "getFullName" ) )

Ok

>
>> #endif // __BASE__
>> """
>>
>> I'm using trunk version of the CXX files.
>> The software works, creates the blocks, chain them, etc, but creating and
>> destroying 1000000 base.Block objects finish with a 44MB memory usage
>> (when 4MB is expected) (the sizeof(Block) is 40 bytes times 1000000 is
>> 40MB !!!)
>
>
>>
>> I've tryed some variations creating Block as a PythonExtension
>> successfully, an the memory doesn't grow with my 1000000 creation/deletion
>> test process (no memory leak).
>>
>> This leads me to thinks that something in the deallocator for
>> new_style_classes isn't working well.
>> What do you think?
>
>
>
>>
>> ----
>>
>> Also, I've made the test with the simple.cxx module from Demo directory,
>> and the same thing happens (as the object is ligthier you have to
>> create/destroy 10000000, or so, times to see the memory growing).
>> Someone have any clue?
>
> Can you post the python code you used to test simple.cxx which to show the
> memory leak please?

Ok, see below

>
> On what python version and operating system are you doing this work?

Python 2.5.2, Windows XP professional, Mingw32 (gcc 3.4.5 mingw-vista
special r3)

>
> Barry
>
>

Attached is the code that shows a memory increase in simple.cxx (with
minor changes to avoid printing a lot when creating and destroying
objects, and with the addition of a Py::String to the old_style_class,
to make it the same content than new_style_class).

When I run the mytestsimple.py, the python process memory usage starts
at 4072kB, changes to 4080kB after the OLD style class test and then
grows to 28.208kB after the NEW style test (these values are taken
from the Windows Task Manager).

I also run it with the "MemoryValidator" software, and it shows a
HotSpot before the end (after new style test)
Size        Count   HotSpot
 271.872  107  1.06%, 271.872 bytes in 107 allocations.
  271.872   107   PyWrapper_New : [{FUNC}PyWrapper_New Line 0]

 25.165.824  96  98.16%, 25.165.824 bytes in 96 allocations.
  25.165.824   96   PyObject_Dir : [{FUNC}PyObject_Dir Line 0]

>From that I understand that the memory increase is located or related
to a PyObject_Dir function. I only found one at cxxsupport.cxx,
related to the dir() method of Py::Object, but I can't find where it
is used in the application.
-- 
Fernando Libonati
fernando.libonati at gmail.com

Re: Memory leak

[Barry S. <ba...@ba...>]

On 5 Apr 2010, at 19:07, Fernando Libonati wrote:

> Dear all,
> 
> I'm new to the list, and want to soy Hello! posting some questions.
> 
> ------
> 
> I'm working on a project related to a block diagram simulator and we have to move the actual implementation (C++ library + SWIG interface) to PyCXX.
> During the first approach to CXX, I've developed a simple class Block that accepts a parent (another Block) a name and a number ...
> The two files used until now are based on simple.cxx:

I'll comment inline
> 
> The base.hxx file is
> """
> #ifndef __BASE__
> #define __BASE__
> #include "CXX/Objects.hxx"
> #include "CXX/Extensions.hxx"
> #include "iostream"
> #include "string"

Its odd to use "header" and not <header> why do you do this?

> 
> #define ENDL "\n";

Use std::endl as its in the standard.

> 
> //class Block: public Py::PythonExtension< Block >
> class Block: public Py::PythonClass< Block >
> {
>     Py::Object parent;
>     Py::String name;
>     Py::Int number;
> public:
>     //Block( Py::String name_, Py::Int number_, Py::Object parent_ )
>     Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds)
>     : Py::PythonClass< Block >::PythonClass( self, args, kwds )
>     {
>         args.verify_length(3);
>         name   = args[0];
>         number = args[1];
>         parent = args[2];
>         //std::cout << "sizeof() = " << sizeof(Block) << ENDL;
>     }
> 
>     virtual ~Block()
>     {
>         //std::cout << "~Block "<<name<<" (auuughh)." << ENDL;
>     }
> 
>     static void init_type(void)
>     {
>         behaviors().name( "Block" );
>         behaviors().doc( "Clase base para los bloques de simulacion" );
>         //behaviors().supportGetattr()
> ;
>         //behaviors().supportSetattr();
>         behaviors().supportGetattro();
>         behaviors().supportSetattro();
> 
> 
>         PYCXX_ADD_NOARGS_METHOD( getFullName, getFullName, "obtener el nombre completo del bloque" );
>         //add_varargs_method("getFullName",&Block::getFullName,"getFullName()");
> 
>         // Call to make the type ready for use
>         behaviors().readyType();
>     }
> 
>     //static PyObject* tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds) {
>     //    return static_cast<PyObject*>(new Block(subtype,args,kwds));
>     //}
>     //static void tp_dealloc(PyObject *obj) {
>     //    Block* k = static_cast<Block*>(obj);
>     //    Py::_XDECREF(*(k->parent));
>     //    Py::_XDECREF(*(k->name));
>     //    Py::_XDECREF(*(k->number));
>     //    //delete k;
>     //    extension_object_deallocator(obj);
>     //}

PyCXX writes this tp_new and the other necessary functions for you.

> 
>     //Py::Object getFullName( const Py::Tuple& args )
>     Py::Object getFullName( void )
>     {
>         std::string r = "";
>         if( *parent != Py_None )

You can use parent->isNone().

>             r += Py::String(PyObject_CallMethod(*parent,"getFullName","")).as_std_string( "utf-8" ) + ".";

You are freely mixing Pyhon C API and PyCXX. This means you have to worry about
reference counting.

Use the newly added callMemberFunction.

	Py::String( parent->callMemberFunction( "getFullName" ) )


> 
>         //if( parent != Py_None ) r = r.concat(p.name).concat(Py::String("."));
>         r += name.as_std_string( "utf-8" );
>         return Py::String(r);
>     }
>     PYCXX_NOARGS_METHOD_DECL( Block, getFullName )
> 
>     //Py::Object getattr( const char * name_ )
>     Py::Object getattro( const Py::String &name_ )
>     {
>         //std::string aname( name_ );
>         std::string aname( name_.as_std_string( "utf-8" ) );
> 
>         if( aname == "name" )   return name;
>         if( aname == "number" ) return number;
>         if( aname == "parent" ) return parent;
>         //else                    return getattr_methods( name_ );;
>         return genericGetAttro( name_ );
>     }
> 
>     int setattro( const Py::String &name_, const Py::Object &value )
>     {
>         std::string aname( name_.as_std_string( "utf-8" ) );
> 
>         if( aname == "name" ) {
>             name = Py::String(value);
>             return 0;
>         } if( aname == "number" ) {
>             number = Py::Int(value);
>             return 0;
>         } if( aname == "parent" ) {
>             parent = value;
>             return 0;
>         } else {
>             return genericSetAttro( name_, value );
>         }
>     }
> 
> };
> 
> #endif // __BASE__
> """
> 
> The base.cxx file is:
> """
> #include <assert.h>
> #include "base.h"
> 
> class base_module : public Py::ExtensionModule<base_module>
> {
> public:
>     base_module()
>     : Py::ExtensionModule<base_module>( "base" ) // this must be name of the file on disk e.g. simple.so or simple.pyd
>     {
>         Block::init_type();
> 
>         //add_varargs_method("Block", &base_module::factory_Block, "Block()");
>         // after initialize the moduleDictionary will exist
> 
>         initialize( "documentation for the simple module" );
>         //Py::Dict d( moduleDictionary() );
>         //d["var"] = Py::String( "var value" );
>         Py::Object x( Block::type() );
>         d["Block"] = x;
>     }
> 
>     virtual ~base_module()
>     {}
> private:
> /*    Py::Object new_Block(const Py::Tuple& args)
>     {
>         if(args.length() != 3) throw Py::RuntimeError("Incorrect # of args to Block(name,number,parent).");
>         //return Py::asObject(new Block(Py::String(args[0]),Py::Int(args[1]),Py::Object(args[2])));
>         return Py::asObject(new Block(args));
>     }
>     */
> };
> 
> #if defined( PY3 )
> static base_module *base;
> 
> extern "C" PyObject *PyInit_base()
> {
> #if defined(PY_WIN32_DELAYLOAD_PYTHON_DLL)
>     Py::InitialisePythonIndirectPy::Interface();
> #endif
> 
>     base = new base_module;
>     return base->module().ptr();
> }
> 
> // symbol required for the debug version
> extern "C" PyObject *PyInit_base_d()
> {
>     return PyInit_base();
> }
> 
> #else
> 
> static base_module *base;
> 
> extern "C" void initbase()
> {
> #if defined(PY_WIN32_DELAYLOAD_PYTHON_DLL)
>     Py::InitialisePythonIndirectPy::Interface();
> #endif
> 
>     base = new base_module;
> }
> 
> // symbol required for the debug version
> extern "C" void initbase_d()
> {
>     initbase();
> }
> #endif
> """
> 
> I'm using trunk version of the CXX files.
> The software works, creates the blocks, chain them, etc, but creating and destroying 1000000 base.Block objects finish with a 44MB memory usage (when 4MB is expected) (the sizeof(Block) is 40 bytes times 1000000 is 40MB !!!)


> 
> I've tryed some variations creating Block as a PythonExtension successfully, an the memory doesn't grow with my 1000000 creation/deletion test process (no memory leak).
> 
> This leads me to thinks that something in the deallocator for new_style_classes isn't working well. 
> What do you think?



> 
> ----
> 
> Also, I've made the test with the simple.cxx module from Demo directory, and the same thing happens (as the object is ligthier you have to create/destroy 10000000, or so, times to see the memory growing).
> Someone have any clue?

Can you post the python code you used to test simple.cxx which to show the memory leak please?

On what python version and operating system are you doing this work?

Barry

Memory leak

[Fernando L. <fer...@gm...>]

Dear all,

I'm new to the list, and want to soy Hello! posting some questions.

------

I'm working on a project related to a block diagram simulator and we have to
move the actual implementation (C++ library + SWIG interface) to PyCXX.
During the first approach to CXX, I've developed a simple class Block that
accepts a parent (another Block) a name and a number ...
The two files used until now are based on simple.cxx:

The base.hxx file is
"""
#ifndef __BASE__
#define __BASE__
#include "CXX/Objects.hxx"
#include "CXX/Extensions.hxx"
#include "iostream"
#include "string"

#define ENDL "\n";

//class Block: public Py::PythonExtension< Block >
class Block: public Py::PythonClass< Block >
{
    Py::Object parent;
    Py::String name;
    Py::Int number;
public:
    //Block( Py::String name_, Py::Int number_, Py::Object parent_ )
    Block(Py::PythonClassInstance *self, Py::Tuple &args, Py::Dict &kwds)
    : Py::PythonClass< Block >::PythonClass( self, args, kwds )
    {
        args.verify_length(3);
        name   = args[0];
        number = args[1];
        parent = args[2];
        //std::cout << "sizeof() = " << sizeof(Block) << ENDL;
    }

    virtual ~Block()
    {
        //std::cout << "~Block "<<name<<" (auuughh)." << ENDL;
    }

    static void init_type(void)
    {
        behaviors().name( "Block" );
        behaviors().doc( "Clase base para los bloques de simulacion" );
        //behaviors().supportGetattr()
;
        //behaviors().supportSetattr();
        behaviors().supportGetattro();
        behaviors().supportSetattro();


        PYCXX_ADD_NOARGS_METHOD( getFullName, getFullName, "obtener el
nombre completo del bloque" );

//add_varargs_method("getFullName",&Block::getFullName,"getFullName()");

        // Call to make the type ready for use
        behaviors().readyType();
    }

    //static PyObject* tp_new(PyTypeObject *subtype, PyObject *args,
PyObject *kwds) {
    //    return static_cast<PyObject*>(new Block(subtype,args,kwds));
    //}
    //static void tp_dealloc(PyObject *obj) {
    //    Block* k = static_cast<Block*>(obj);
    //    Py::_XDECREF(*(k->parent));
    //    Py::_XDECREF(*(k->name));
    //    Py::_XDECREF(*(k->number));
    //    //delete k;
    //    extension_object_deallocator(obj);
    //}

    //Py::Object getFullName( const Py::Tuple& args )
    Py::Object getFullName( void )
    {
        std::string r = "";
        if( *parent != Py_None )
            r +=
Py::String(PyObject_CallMethod(*parent,"getFullName","")).as_std_string(
"utf-8" ) + ".";

        //if( parent != Py_None ) r = r.concat(p.name
).concat(Py::String("."));
        r += name.as_std_string( "utf-8" );
        return Py::String(r);
    }
    PYCXX_NOARGS_METHOD_DECL( Block, getFullName )

    //Py::Object getattr( const char * name_ )
    Py::Object getattro( const Py::String &name_ )
    {
        //std::string aname( name_ );
        std::string aname( name_.as_std_string( "utf-8" ) );

        if( aname == "name" )   return name;
        if( aname == "number" ) return number;
        if( aname == "parent" ) return parent;
        //else                    return getattr_methods( name_ );;
        return genericGetAttro( name_ );
    }

    int setattro( const Py::String &name_, const Py::Object &value )
    {
        std::string aname( name_.as_std_string( "utf-8" ) );

        if( aname == "name" ) {
            name = Py::String(value);
            return 0;
        } if( aname == "number" ) {
            number = Py::Int(value);
            return 0;
        } if( aname == "parent" ) {
            parent = value;
            return 0;
        } else {
            return genericSetAttro( name_, value );
        }
    }

};

#endif // __BASE__
"""

The base.cxx file is:
"""
#include <assert.h>
#include "base.h"

class base_module : public Py::ExtensionModule<base_module>
{
public:
    base_module()
    : Py::ExtensionModule<base_module>( "base" ) // this must be name of the
file on disk e.g. simple.so or simple.pyd
    {
        Block::init_type();

        //add_varargs_method("Block", &base_module::factory_Block,
"Block()");
        // after initialize the moduleDictionary will exist

        initialize( "documentation for the simple module" );
        //Py::Dict d( moduleDictionary() );
        //d["var"] = Py::String( "var value" );
        Py::Object x( Block::type() );
        d["Block"] = x;
    }

    virtual ~base_module()
    {}
private:
/*    Py::Object new_Block(const Py::Tuple& args)
    {
        if(args.length() != 3) throw Py::RuntimeError("Incorrect # of args
to Block(name,number,parent).");
        //return Py::asObject(new
Block(Py::String(args[0]),Py::Int(args[1]),Py::Object(args[2])));
        return Py::asObject(new Block(args));
    }
    */
};

#if defined( PY3 )
static base_module *base;

extern "C" PyObject *PyInit_base()
{
#if defined(PY_WIN32_DELAYLOAD_PYTHON_DLL)
    Py::InitialisePythonIndirectPy::Interface();
#endif

    base = new base_module;
    return base->module().ptr();
}

// symbol required for the debug version
extern "C" PyObject *PyInit_base_d()
{
    return PyInit_base();
}

#else

static base_module *base;

extern "C" void initbase()
{
#if defined(PY_WIN32_DELAYLOAD_PYTHON_DLL)
    Py::InitialisePythonIndirectPy::Interface();
#endif

    base = new base_module;
}

// symbol required for the debug version
extern "C" void initbase_d()
{
    initbase();
}
#endif
"""

I'm using trunk version of the CXX files.
The software works, creates the blocks, chain them, etc, but creating and
destroying 1000000 base.Block objects finish with a 44MB memory usage (when
4MB is expected) (the sizeof(Block) is 40 bytes times 1000000 is 40MB !!!)

I've tryed some variations creating Block as a PythonExtension successfully,
an the memory doesn't grow with my 1000000 creation/deletion test process
(no memory leak).

This leads me to thinks that something in the deallocator for
new_style_classes isn't working well.
What do you think?

----

Also, I've made the test with the simple.cxx module from Demo directory, and
the same thing happens (as the object is ligthier you have to create/destroy
10000000, or so, times to see the memory growing).
Someone have any clue?

Best regards,
Fernando Libonati

Re: The difference between PythonExtension and ExtensionObject

[Barry S. <ba...@ba...>]

On 1 Apr 2010, at 07:52, B A wrote:

> Hi, guys,
> PyCXX is great. :) 
> After reading the code, I am not clear about the relationship between PythonExtension and ExtensionObject.
> Some usage in the sample code was 
> 
> Py::ExtensionObject<range> rheap( new range(1, 10, 2) );
> 
> and 
> 
> range is based on PythonExtension.
> 
> 
> How to use ExtensionObject?

ExtensionObject<type> is an object that will only accept a Py::Object that is of <type>.
Use this to make sure you have been passed one of your objects.

If I wanted to make sure a Py::Object was a String I could write:

	Py::String s( args[2] );

If args[2] is not a string then an exception is thrown.

Now if I want to make sure I get a MyType then I'd code:

	ExtensionObject<MyType> s( args[2] );

If args[2] is not a MyType then an exception is thrown.

Barry

The difference between PythonExtension and ExtensionObject

[B A <se...@gm...>]

Hi, guys,
PyCXX is great. :)
After reading the code, I am not clear about the relationship between
PythonExtension and ExtensionObject.
Some usage in the sample code was

Py::ExtensionObject<range> rheap( new range(1, 10, 2) );

and

range is based on PythonExtension.


How to use ExtensionObject?

Thanks!

Re: Exception handling in PythonClass

[Barry S. <ba...@ba...>]

On 5 Mar 2010, at 13:19, cxu...@po... wrote:

> Hi,
> 
> The list seems not to have much traffic, but I hope someone will be able to help me. Here's my problem:
> 
> I tried to make a class to join Python2.6 and PyCXX 6.1.1, using Py:PythonClass. I managed to successfully create one basing on simple.cxx, with a vararg method in it. It would all be okay, but any exception thrown inside it causes Python to abort:
> 
>>>> a.write()
> terminate called after throwing an instance of 'Py::IndexError'
> Aborted
> 
> I'm not proficient in C++ and I can't find the missing try-catch clause.
> Throwing exceptions in initializer works, but I can't quite figure out either ;)
> 
> Thanks in advance!

This is bug that has been fixed on trunk. Checkout:

https://cxx.svn.sourceforge.net/svnroot/cxx/trunk/CXX

Barry

Exception handling in PythonClass

[<cxu...@po...>]

Hi,

The list seems not to have much traffic, but I hope someone will be able to help me. Here's my problem:

I tried to make a class to join Python2.6 and PyCXX 6.1.1, using Py:PythonClass. I managed to successfully create one basing on simple.cxx, with a vararg method in it. It would all be okay, but any exception thrown inside it causes Python to abort:

>>> a.write()
terminate called after throwing an instance of 'Py::IndexError'
Aborted

I'm not proficient in C++ and I can't find the missing try-catch clause.
Throwing exceptions in initializer works, but I can't quite figure out either ;)

Thanks in advance!

Cheers,
rhn

Announcement: PyCXX 6.1.1 released

[Barry S. <ba...@ba...>]

PyCXX Version 6.1.1 (26-Sep-2009)

* Supports Python 3 starting at Python 3.1 and Python 2.
* Code clean up to fix compiler warnings reported by gcc 4.2.1 on Mac  
OS X when building for Python 3.

https://sourceforge.net/projects/cxx/

RE: Unicode to std::wstring

[William N. <fir...@ho...>]

Barry Scott <barry@ba...> -	2009-02-21 12:20
> On 18 Feb 2009, at 19:32, William Newbery wrote:
> 
> >
> >
> > > On 10 Feb 2009, at 17:15, William Newbery wrote:
> > >
> > > > I want to start using uncicode strings.
> > > >
> > > > Looking at Py::string I see the method as_unicodestring, however
> > > > this returns a std::basic_string<Py_UNICODE> and provides no  
> > option
> > > > of encodeing...
> > > >
> > > > Another method is the encode method, this lets me provide the
> > > > encoding but just returns another Py::String...
> > > >
> > > >
> > > > What exactly do I need to do to go between a python unicode string
> > > > and a std::wstring (where sizeof(wchar_t)==2)in UTF-16 encoding?
> > >
> > > I take it that Py_UNICODE is 4 on your platform.
> > >
> > > You could try encode('utf-16') to get a Py::String that is in  
> > utf-16.
> > > Then use as_std_string() to get a std:string, use c_str() to get a
> > > pointer to the contents and cast it to wchar_t.
> > >
> > > Adding a as_std_wstring would be a reasonable thing to add to PyCXX
> > > to make this convenient.
> > > as_std_wstring could look inside the Py_Object and avoid a number of
> > > conversion steps.
> > >
> > > Barry
> >
> > The problem is thats basicly a hack and results in several bugs  
> > since your stuffing a double byte string into a std::string.
> > -Any utf-16 charecter that has 00 for the first byte will break it.  
> > I dont know if there are any such charecters in little endian  
> > encoding, but for big endian quite alot will
> 
> >
> > -std::string only terminates with a single \0 but utf-16 needs \0\0.  
> > This means casting the c_str() to a wchar_t wont work because the  
> > charecter after the first \0 it is outside the string, and thus  
> > could be anything. So you end up having to make yet another copy by  
> > allocating a block of which is size()+2 and making sure both of the  
> > last two bytes are 0...
> 
> std::string does not use NUL to terminate strings. Use c_str() to get  
> to the data and use size() to find out the length().
> 
> >
> >
> > "Adding a as_std_wstring would be a reasonable thing to add to PyCXX  
> > to make this convenient." wstring could be say ucs-2 or someother  
> > wide format as easily as utf_16, and then people may also want  
> > ucs-4, etc.
> >
> > Something that can support all the diffrent formats would be good.
> >
> > eg mayby:
> > int Py::String::c_encode(const char *format, char *buffer, int  
> > buffersize);
> > where if *buffer is null it just returns the number of bytes needed  
> > to encode in the given format. The user can then allocate the needed  
> > buffer and get the string encoded correctly in whatever format,  
> > ending with something that is safe to cast to wchar_t or unsigned  
> > int or whatever is correct for that format. buffersize should again  
> > be in bytes to avoid confusion.
> 
> Could you create a patch for this?
> 
> Barry

>From my limited knowleged of the c-api I was able to put this together. Theres a few things I would like to do better but am not aware howto, namly:
-A way to calc the required buffer without actauly encodeing a bytes object
-A way to encode directly into a buffer rather than a python created bytes object which then must be copied

Also I'm not sure how your checking for and throwing exceptions that origenate from python code so Ive left that out.

        int as_c_string(char *buffer, Py_ssize_t bufferBytes, const char *encoding, const char *error="strict")const
        {
            unsigned nullCnt=4;//worst case
            PyObject* bytes = PyUnicode_AsEncodedString(ptr(), encoding, errors);//newref
            
            Py_ssize_t sourceBytes;
            const char *bytesData;
            PyBytes_AsStringAndSize(bytes, (char**)&bytesData, &sourceBytes);
            
            if(!buffer)
            {
                Py_DECREF(bytes);
                return sourceBytes+nullCnt;
            }
            
            if(sourceBytes+nullCnt > bufferBytes)
                throw RuntimeError("buffer to small for string.");
            memcpy((void*)buffer, (const void*)bytesData, sourceBytes);
            memset((void*)buffer+sourceBytes, 0, nullCnt);//null terminate
            Py_DECREF(bytes);
        }

Share your photos with Windows Live Photos - Free Try it Now!
_________________________________________________________________

Hotmail, Messenger, Photos  and more - all with the new Windows Live. Get started! 
http://www.download.live.com/

Re: unresolved symbal

[Barry S. <ba...@ba...>]

On 19 Feb 2009, at 13:16, William Newbery wrote:

>
>
> So I was trying to upgrade to python 2.6.1 (from 2.5.2), and pycxx  
> 5.5.0 (from 5.4.2) but ive run into some unresolved external symbals  
> that were not there before...
>
> "public: virtual class Py::Object __thiscall  
> Py::PythonExtensionBase::getattro(class Py::Object const &)"
> "public: virtual int __thiscall  
> Py::PythonExtensionBase::setattro(class Py::Object const &,class  
> Py::Object const &)"

These symbols are in cxx_extensions.cxx. Sounds like you have not  
compiled the 5.5.0 version.
Did you "make clean" and build again?

Barry

Re: Unicode to std::wstring

[Barry S. <ba...@ba...>]

On 18 Feb 2009, at 19:32, William Newbery wrote:

>
>
> > On 10 Feb 2009, at 17:15, William Newbery wrote:
> >
> > > I want to start using uncicode strings.
> > >
> > > Looking at Py::string I see the method as_unicodestring, however
> > > this returns a std::basic_string<Py_UNICODE> and provides no  
> option
> > > of encodeing...
> > >
> > > Another method is the encode method, this lets me provide the
> > > encoding but just returns another Py::String...
> > >
> > >
> > > What exactly do I need to do to go between a python unicode string
> > > and a std::wstring (where sizeof(wchar_t)==2)in UTF-16 encoding?
> >
> > I take it that Py_UNICODE is 4 on your platform.
> >
> > You could try encode('utf-16') to get a Py::String that is in  
> utf-16.
> > Then use as_std_string() to get a std:string, use c_str() to get a
> > pointer to the contents and cast it to wchar_t.
> >
> > Adding a as_std_wstring would be a reasonable thing to add to PyCXX
> > to make this convenient.
> > as_std_wstring could look inside the Py_Object and avoid a number of
> > conversion steps.
> >
> > Barry
>
> The problem is thats basicly a hack and results in several bugs  
> since your stuffing a double byte string into a std::string.
> -Any utf-16 charecter that has 00 for the first byte will break it.  
> I dont know if there are any such charecters in little endian  
> encoding, but for big endian quite alot will

>
> -std::string only terminates with a single \0 but utf-16 needs \0\0.  
> This means casting the c_str() to a wchar_t wont work because the  
> charecter after the first \0 it is outside the string, and thus  
> could be anything. So you end up having to make yet another copy by  
> allocating a block of which is size()+2 and making sure both of the  
> last two bytes are 0...

std::string does not use NUL to terminate strings. Use c_str() to get  
to the data and use size() to find out the length().

>
>
> "Adding a as_std_wstring would be a reasonable thing to add to PyCXX  
> to make this convenient." wstring could be say ucs-2 or someother  
> wide format as easily as utf_16, and then people may also want  
> ucs-4, etc.
>
> Something that can support all the diffrent formats would be good.
>
> eg mayby:
> int Py::String::c_encode(const char *format, char *buffer, int  
> buffersize);
> where if *buffer is null it just returns the number of bytes needed  
> to encode in the given format. The user can then allocate the needed  
> buffer and get the string encoded correctly in whatever format,  
> ending with something that is safe to cast to wchar_t or unsigned  
> int or whatever is correct for that format. buffersize should again  
> be in bytes to avoid confusion.

Could you create a patch for this?

Barry

unresolved symbal

[William N. <fir...@ho...>]

So I was trying to upgrade to python 2.6.1 (from 2.5.2), and pycxx 5.5.0 (from 5.4.2) but ive run into some unresolved external symbals that were not there before...

"public: virtual class Py::Object __thiscall Py::PythonExtensionBase::getattro(class Py::Object const &)" 
"public: virtual int __thiscall Py::PythonExtensionBase::setattro(class Py::Object const &,class Py::Object const &)"

Compiling the "simple" demo for 5.5.0 does not create this error (having created a vs project for it, the download seems to be missing it), but looking at the code I cant see how mine differs and thus seems to suddenly decided it needs those two methods...


Since vs aborts after an object file causes these errors Im not sure if all of them will fail, however having commented some classes out etc at least 4 of them do (and always been the first it tried to link)

one of the smaller classes causing the problem:

class PyPrimitive : public Py::PythonExtension<PyPrimitive>
{
public:
    static void InitType();
    PyPrimitive();
    virtual ~PyPrimitive();
    
    Py::Object CreateCircleVB(const Py::Tuple &args);

    Py::Object DrawLine(const Py::Tuple &args);
};

void PyPrimitive::InitType()
{
    behaviors().name("Primitive");
    behaviors().doc ("Primitive functions and classes");
    behaviors().supportGetattr();
    
    add_varargs_method("CreateCircleVB",    &PyPrimitive::CreateCircleVB,
        "VertexBuffer(Graphics, (x,y), r, col, sides, solid=False)\nVertexBuffer(Graphics, (x,y), r, col, sides, centreCol)");
    add_varargs_method("DrawLine", &PyPrimitive::DrawLine, "void(VertexBuffer, (x1,y1), (x2,y2) [,col1, col2])");
}
PyPrimitive::PyPrimitive()
{
}
PyPrimitive::~PyPrimitive()
{
}
Py::Object PyPrimitive::DrawLine(const Py::Tuple &args)
{
    try
    {
        IVertexBuffer *vb = Py::ExtensionObject<PyVertexBuffer>(args[0]).extensionObject()->GetIVertexBuffer();
        
        Py::Sequence pyp1 = args[1];
        Point2<> p1((float)Py::Float(pyp1[0]), (float)Py::Float(pyp1[1]));
        
        Py::Sequence pyp2 = args[2];
        Point2<> p2((float)Py::Float(pyp2[0]), (float)Py::Float(pyp2[1]));
        
        unsigned c1 = 0xFFFFFFFF;
        if(args.size()>=4)
            c1 = (unsigned)(__int64)Py::LongLong(args[3]);
        
        unsigned c2 = c1;
        if(args.size()>=5)
            c2 = (unsigned)(__int64)Py::LongLong(args[4]);
        
        ::DrawLine(vb, p1, p2, c1, c2);
        
        return Py::Nothing();
    }
    TRANSLATE_EXCEPTIONS//catches and rethrows any fllib::exception::, boost, or std exceptions as simelar python exception types
}
Py::Object PyPrimitive::CreateCircleVB(const Py::Tuple &args)

{

    try

    {

        ...same sort of conversion code...creates an IVertexBuffer*vb

        return Py::asObject(new PyVertexBuffer(vb));

    }

    TRANSLATE_EXCEPTIONS

}



Only a single instance of this class is generally created, and stored as "Py::ExtensionObject<PyPrimitive> primitive;" in the module object.
PyFlLib::PyFlLib()
:Py::ExtensionModule<PyFlLib>("py_fllib"),
primitive(new PyPrimitive), ..init stuff...
{
    ...init stuff
    PyPrimitive::InitType();
    
    ...methods...
    
    initialize("Python Fl Lib wrapper.");
    
    Py::Dict dict = moduleDictionary();
    ...
    dict["primitive"] = primitive;
    ...
}

Try Facebook in Windows Live Messenger! Try it Now!
_________________________________________________________________
Get all your favourite content with the slick new MSN Toolbar - FREE
http://clk.atdmt.com/UKM/go/111354027/direct/01/

Re: Unicode to std::wstring

[William N. <fir...@ho...>]

> On 10 Feb 2009, at 17:15, William Newbery wrote:
> 
> > I want to start using uncicode strings.
> >
> > Looking at Py::string I see the method as_unicodestring, however  
> > this returns a std::basic_string<Py_UNICODE> and provides no option  
> > of encodeing...
> >
> > Another method is the encode method, this lets me provide the  
> > encoding but just returns another Py::String...
> >
> >
> > What exactly do I need to do to go between a python unicode string  
> > and a std::wstring (where sizeof(wchar_t)==2)in UTF-16 encoding?
> 
> I take it that Py_UNICODE is 4 on your platform.
> 
> You could try encode('utf-16') to get a Py::String that is in utf-16.
> Then use as_std_string() to get a std:string, use c_str() to get a  
> pointer to the contents and cast it to wchar_t.
> 
> Adding a as_std_wstring  would be a reasonable thing to add to PyCXX  
> to make this convenient.
> as_std_wstring could look inside the Py_Object and avoid a number of  
> conversion steps.
> 
> Barry

The problem is thats basicly a hack and results in several bugs since your stuffing a double byte string into a std::string.
-Any utf-16 charecter that has 00 for the first byte will break it. I dont know if there are any such charecters in little endian encoding, but for big endian quite alot will
-std::string only terminates with a single \0 but utf-16 needs \0\0. This means casting the c_str() to a wchar_t wont work because the charecter after the first \0 it is outside the string, and thus could be anything. So you end up having to make yet another copy by allocating a block of which is size()+2 and making sure both of the last two bytes are 0...

"Adding a as_std_wstring  would be a reasonable thing to add to PyCXX  to make this convenient." wstring could be say ucs-2 or someother wide format as easily as utf_16, and then people may also want ucs-4, etc.

Something that can support all the diffrent formats would be good.

eg mayby:
int Py::String::c_encode(const char *format, char *buffer, int buffersize);
where if *buffer is null it just returns the number of bytes needed to encode in the given format. The user can then allocate the needed buffer and get the string encoded correctly in whatever format, ending with something that is safe to cast to wchar_t or unsigned int or whatever is correct for that format. buffersize should again be in bytes to avoid confusion.

_________________________________________________________________
Win New York holidays with Kellogg’s & Live Search
http://clk.atdmt.com/UKM/go/111354033/direct/01/

Re: Unicode to std::wstring

[Barry S. <ba...@ba...>]

On 10 Feb 2009, at 17:15, William Newbery wrote:

> I want to start using uncicode strings.
>
> Looking at Py::string I see the method as_unicodestring, however  
> this returns a std::basic_string<Py_UNICODE> and provides no option  
> of encodeing...
>
> Another method is the encode method, this lets me provide the  
> encoding but just returns another Py::String...
>
>
> What exactly do I need to do to go between a python unicode string  
> and a std::wstring (where sizeof(wchar_t)==2)in UTF-16 encoding?

I take it that Py_UNICODE is 4 on your platform.

You could try encode('utf-16') to get a Py::String that is in utf-16.
Then use as_std_string() to get a std:string, use c_str() to get a  
pointer to the contents and cast it to wchar_t.

Adding a as_std_wstring  would be a reasonable thing to add to PyCXX  
to make this convenient.
as_std_wstring could look inside the Py_Object and avoid a number of  
conversion steps.

Barry

Unicode

[William N. <fir...@ho...>]

I want to start using uncicode strings.

Looking at Py::string I see the method as_unicodestring, however this returns a std::basic_string<Py_UNICODE> and provides no option of encodeing...

Another method is the encode method, this lets me provide the encoding but just returns another Py::String...


What exactly do I need to do to go between a python unicode string and a std::wstring (where sizeof(wchar_t)==2)in UTF-16 encoding?



_________________________________________________________________
Twice the fun—Share photos while you chat with Windows Live Messenger. Learn more.
http://www.microsoft.com/uk/windows/windowslive/products/messenger.aspx

Re: Python objects not deleted

[Barry S. <ba...@ba...>]

On 1 Jan 2009, at 23:56, William Newbery wrote:

> > In cases like this I use a pointer:
> >
> > Py::ExtensionObject<Input> *input_ptr;
> >
> > and new a Py::ExtensionObject<Input>( input ) when I have it  
> available.
> >
> > You need to create the the Py::Object to pass it back to python like
> > this:
> >
> > Py::ExtensionObject<Input> an_input_object( Py::asObject( new
> > Input(this, hwnd) ) ) );

You are not use asObject...


>
> >
> > I've been looking at allowing Py::Object to hold NULL for V6.0.0 -  
> but
> > that targeted at Python 3.0.
> >
> >
> > Barry
>
> Ok, I did as you said and used the Py::ExtensionObject, however I  
> seem to be back where I started with object not being deleted :(
>
> The problem seems to be that somehow it thinks it still refrenced  
> when its not, that is when I checked the refrence count before  
> destroying what should be the final refrence it reports a count of  
> 2. Whats more when the object is first created the count seems to  
> start at 2...
>
> Where the object is created
>     //create voice object
>     Py::ExtensionObject<SourceVoice> voice(new  
> SourceVoice(sourceVoice, Py::ExtensionObject<Sound>(this), loop));

You are not using Py::asObject...

>
>     std::cout << voice.reference_count() << " ";//2???

I'd expect it to be 0 given the error with Py::asObject

Make the asObject change and see what you get.

>
>     audio->extensionObject()->AddSourceVoice(voice);//the Audio  
> class maintains a reference (basicly adds to playing set in Audio  
> class)
>     //start playback
>     sourceVoice->Start(0);
>
>     std::cout << voice.reference_count() << std::endl;//3, as  
> exspected since refrence was 2 before
>     return voice;//in my tests this return value was not used
> }
>
> The objects constructor - I see nothing here to create a 2nd  
> refrence to itself, only an extra refrence to the Sound object
> SourceVoice::SourceVoice(IXAudio2SourceVoice *_voice,  
> Py::ExtensionObject<Sound> _sound, bool _isLooping)
> :voice(_voice), isLooping(_isLooping), isPaused(false), sound(_sound)
> {}

If you print the ref count in the c'tor it should be 0.

Barry

RE: Python objects not deleted

[William N. <fir...@ho...>]

> In cases like this I use a pointer:
> 
>   Py::ExtensionObject<Input> *input_ptr;
> 
> and new a Py::ExtensionObject<Input>( input ) when I have it available.
> 
> You need to create the the Py::Object to pass it back to python like  
> this:
> 
> Py::ExtensionObject<Input> an_input_object( Py::asObject( new  
> Input(this, hwnd) ) ) );
> 
> I've been looking at allowing Py::Object to hold NULL for V6.0.0 - but  
> that targeted at Python 3.0.
> 
> 
> Barry

Ok, I did as you said and used the Py::ExtensionObject, however I seem to be back where I started with object not being deleted :(

The problem seems to be that somehow it thinks it still refrenced when its not, that is when I checked the refrence count before destroying what should be the final refrence it reports a count of 2. Whats more when the object is first created the count seems to start at 2...

Where the object is created
    //create voice object
    Py::ExtensionObject<SourceVoice> voice(new SourceVoice(sourceVoice, Py::ExtensionObject<Sound>(this), loop));
    std::cout << voice.reference_count() << " ";//2???
    audio->extensionObject()->AddSourceVoice(voice);//the Audio class maintains a reference (basicly adds to playing set in Audio class)
    //start playback
    sourceVoice->Start(0);

    std::cout << voice.reference_count() << std::endl;//3, as exspected since refrence was 2 before
    return voice;//in my tests this return value was not used
}

The objects constructor - I see nothing here to create a 2nd refrence to itself, only an extra refrence to the Sound object
SourceVoice::SourceVoice(IXAudio2SourceVoice *_voice, Py::ExtensionObject<Sound> _sound, bool _isLooping)
:voice(_voice), isLooping(_isLooping), isPaused(false), sound(_sound)
{}

Where the final refrence counts should be removed:
    for(std::set<Py::ExtensionObject<SourceVoice>>::iterator it=playing->begin();it!=playing->end();++it)
    {
        it->extensionObject()->GetVoice()->GetState(&state);
        if(state.BuffersQueued <= 0)//finished, we can clear this up
        {
            //since erasing invalidates the iterator, we add it to a seperate container to delete later, safly
            finished->push_back(*it);
        }
    }
    //clear up finished sounds
    for(std::vector<Py::ExtensionObject<SourceVoice>>::iterator it=finished->begin();it!=finished->end();++it)
    {
        playing->erase(*it);
    }
    //here the last refrence should be deleted, since the python test code never kept the refrence returned before
    //and nethier the playing or finsihed list now have a refrence to the SourceVoice object
    finished->clear();

_________________________________________________________________
Imagine a life without walls.  See the possibilities. 
http://clk.atdmt.com/UKM/go/122465943/direct/01/