Re: [Numpy-discussion] glibc invalid pointer error

[Fernando P. <fpe...@gm...>]

On 10/30/06, Travis Oliphant <oli...@ee...> wrote:
> Fernando Perez wrote:

> >This sounds awfully reminiscent of the bug I recently mentioned:
> >
> >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099
> >
> >
>
> It actually looks very much like it.  I think the problem may be in f2py
> or in one of the C-API calls where-in there is a reference-count problem
> with the built-in data-type objects.
>
> NumPy won't try to free those anymore which will solve the immediate
> problem, but there is still a reference-count problem somewhere.
>
> The reference to the data-type objects is consumed by constructors that
> take PyArray_Descr * arguments.  So, you often need to INCREF before
> passing to those constructors.  It looks like this INCREF is forgotten
> in some extension module (perhaps in f2py or PyMC).  It's possible it's
> in NumPy itself, though I've re-checked the code lots of times looking
> for that specific problem.

As a data point, our code has almost no manual memory management in C,
but lots and lots of f2py-generated wrappers, as well as a lot of
weave.inline-generated code.

We do have hand-written C extensions, but most of them operate on
externally allocated arrays.  The one little snippet where we manually
manage memory is a copy of numpy's innerproduct() which I simplified
and tuned for our purposes; it just does:

    ret = (PyArrayObject *)PyArray_SimpleNew(nd,dimensions,
                                             ap1->descr->type_num);
    if (ret == NULL) goto fail;

 [ do computational loop to fill in ret array, no memory management here ]

    return (PyObject *)ret;

  fail:
    Py_XDECREF(ret);
    return NULL;

That's the full extent of our manual memory management, and I don't
see any problem with it, but maybe there is: I copied this from numpy
months ago and haven't really looked again.

Cheers,

f