Re: [Numpy-discussion] segfault in PyArray_FromDims

[<co...@ph...>]

At some point, Juerg Tschirren <jue...@ui...> wrote:

> I did some experimenting with the NumPy C API. I wrote two functions.
> One for processing a NumPy array in C++ and the other one for
> generating a NumPy array in C++. The processing function work perfectly
> fine. But in the array-generating function I get a segmentation fault
> whenever I call PyArray_FromDims. I used swig for generating the wrapper
> functions.
> 
> The two src-files (numPyExt.i and numPyExt.cc):

[code snipped]

> Compiled with:
>   g++ -c  -I/usr/local/include/python2.0 -I/usr/local/lib/python2.0/site-packages -O2 numPyExt.cc
>   swig -python -c++ numPyExt.i
>   g++ -c -O2 numPyExt_wrap.c -DOS_LINUX -DHAVE_CONFIG_H -I. -I/usr/local/include/python2.0 -I/usr/local/lib/python2.0/site-packages -I/usr/local/lib/python2.0/config
>   g++ -W1,--heap,50000,--stack,100000 -O2 -shared numPyExt.o numPyExt_wrap.o -lstdc++ -o numPyExt.so
> 

This problem was discussed in April on this list, see
http://www.geocrawler.com/mail/thread.php3?subject=%5BNumpy-discussion%5D+Numeric+on+OS+X+-+Anyone+get+it+to+work+%3F&list=1329

I banged my head against this for hours a few weeks ago until I found
the above. The problem is that PyArray_FromDims (and all other
PyArray_*) are not functions -- they're macros, defined like this:

#define PyArray_FromDims \
  (*(PyArray_FromDims_RET (*)PyArray_FromDims_PROTO) \
   PyArray_API[PyArray_FromDims_NUM])

This means that all the PyArray_* functions are done through a lookup
table PyArray_API, which is initialized by import_array(). By default,
PyArray_API is defined in arrayobject.h as 'static void
**PyArray_API', meaning that it is not accessible outside of the
translation unit (i.e. file) that includes it.

The relevant part of Numeric/arrayobject.h is this:

#if defined(PY_ARRAY_UNIQUE_SYMBOL)
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif

/* C API address pointer */ 
#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)
extern void **PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void **PyArray_API;
#else
static void **PyArray_API;
#endif
#endif

So, one way to get the behaviour you want is 
1) in the file where import_array is called, #define
PY_ARRAY_UNIQUE_SYMBOL to be something like Py_Array_API_myext. (Make
it unique because it will be exported as part of the .so file.) before
including Numeric/arrayobject.h
2) in the other files, #define NO_IMPORT_ARRAY before including Numeric/arrayobject.h

Another way is to have your main file (say, main.c) call functions in the other
files, passing the value of PyArray_API defined there, so that the
other files can set theirs to that value.

Hope this helps.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|co...@ph...