Re: [Numpy-discussion] Reference counting question

[Travis O. <oli...@ie...>]

Sasha wrote:

>I was looking at the code implementing array_new in arrayobject.c and
>for a while I could not convince myself that it handles ref. counts
>correctly.  The cleanup code (at the "fail:" label contains 
>Py_XDECREF(descr), meaning that descr is unreferenced on failure
>unless it is NULL.  This makes sense because descr is created inside
>array_new  by PyArray_DescrConverter,  but  if the failure is detected
>in PyArg_ParseTupleAndKeywords, descr may be NULL.    What was
>puzzling to me,  failures of  PyArray_NewFromDescr are handled by "if
>(ret == NULL) {descr=NULL;goto fail;}" that sets descr to NULL before
>jumping to cleanup.  As I investigated further, I've discovered the
>following helpful comment preceding PyArray_NewFromDescr : /* steals a
>reference to descr (even on failure) */ that explains  why descr=NULL
>is necessary.
>
>I wonder what was the motivation for this design choice.  I don't
>think this is a natural behavior for python C-API functions.  I am not
>proposing to make any changes, just curious about the design.
>  
>
The PyArray_Descr structure never used to be a Python object.   Now it is.

There is the C-API PyArray_DescrFromType that used to just return a 
C-structure but now it returns a reference-counted Python object. 

People are not used to reference counting the PyArray_Descr objects.  
The easiest way to make this work in my mind was to have the functions 
that use the Descr object steal a reference because ultimately the Descr 
objects purpose is to reside in an array.   It is created for the 
purpose of being a member of an array structure which therefore steals 
it's reference. 

As an example, with this design you can write (and there are macros that 
do).

PyArray_NewFromDescr(...., PyArray_DescrFromType(type_num), ....) 

and not create reference-count leaks.

-Travis