Re: [Swig-user] How to output 2D C arrays to Python as NumPy arrays?

[Nickolas F. <fo...@ca...>]

Alan,

I am quite a novice with SWIG, but I believe that colleagues have
successfully used this numpy.i file to cleanly get numpy arrays out of
C routines:

<http://docs.scipy.org/doc/numpy/reference/swig.html>

Perhaps others can comment on whether this is a good solution and its
limitations. Also, if you want to wrap things manually, this is a
great resource for the numpy C API:

<http://docs.scipy.org/doc/numpy/reference/c-api.html>

Take care,
Nick

On Tue, Jan 10, 2012 at 11:05, Alan W. Irwin <ir...@be...> wrote:
> I already have a solution for outputting 2D C arrays to Python as
> NumPy arrays.  That method is to treat the NumPy arrays as input
> arrays from the SWIG perspective, but simply allow the C code to
> overwrite the contents of those preexisting NumPy arrays that appear
> in Python argument lists. Obviously that is a massive hack that
> violates the sanctity of function arguments that Python users expect.
> Therefore, I would like to replace what I have got with something more
> appropriate to Python, but I need some help to do that.
>
> Here is what I do now:
>
> C code can handle 2D arrays in various ways so I thought I should show
> how these char ** and double ** 2D arrays are normally malloced and
> freed with the following routines (which have had all the potential
> error processing removed for brevity).
>
> void
> ephcom_Alloc2dChar( char ***fp, int nx, int ny )
> {
>     int i;
>
>     *fp = (char **) malloc( ( size_t )nx *  sizeof ( char * ) );
>     for ( i = 0; i < nx; i++ )
>         ( *fp )[i] = (char *) malloc( (size_t) ny  * sizeof ( char ) );
> }
>
> void
> ephcom_Free2dChar( char **f, int nx )
> {
>     int i;
>
>     for ( i = 0; i < nx; i++ )
>         free( (void *) f[i] );
>
>     free( (void *) f );
> }
>
> void
> ephcom_Alloc2dDouble( double ***fp, int nx, int ny )
> {
>     int i;
>
>     *fp = (double **) malloc( (size_t) nx * sizeof ( double * ) );
>     for ( i = 0; i < nx; i++ )
>         ( *fp )[i] = (double *) malloc( (size_t) ny * sizeof ( double ) );
> }
>
> void
> ephcom_Free2dDouble( double **f, int nx )
> {
>     int i;
>
>     for ( i = 0; i < nx; i++ )
>         free( (void *) f[i] );
>
>     free( (void *) f );
> }
>
> I currently set up the typemaps _as if_ the 2D NumPy arrays are input,
> e.g.,
>
> // set count for first dimension and check second dimension is exactly 7.
> %typemap( in ) char **OutMatrixCk7( PyArrayObject * tmp )
> {
>     int i;
>     tmp = (PyArrayObject *) PyArray_ContiguousFromObject( $input, NPY_STRING, 1, 1 );
>     if ( tmp == NULL )
>         return NULL;
>
>     if ( tmp->strides[0] != 7 )
>     {
>         PyErr_SetString( PyExc_ValueError, "NumPy string array must have dtype='S7'." );
>         return NULL;
>     }
>     Alen = tmp->dimensions[0];
>     $1   = (char **) malloc( sizeof ( char* ) * Alen );
>     for ( i = 0; i < Alen; i++ )
>     {
>         $1[i] = ( tmp->data + i * tmp->strides[0] );
>     }
> }
> %typemap( freearg ) char **OutMatrixCk7 { Py_DECREF( tmp$argnum ); free( $1 );}
>
> // Check count for first dimension and check second dimension is
> // exactly 6.
> %typemap( in ) double **OutMatrixCkCk6( PyArrayObject * tmp )
> {
>     int i, size;
>     tmp = (PyArrayObject *) PyArray_ContiguousFromObject( $input, PyArray_DOUBLE, 2, 2 );
>     if ( tmp == NULL )
>         return NULL;
>     if ( tmp->dimensions[0] != Alen || tmp->dimensions[1] != 6 )
>     {
>         PyErr_SetString( PyExc_ValueError, "Matrix must have first dimension the same as previous vector and a second dimension of 6." );
>         return NULL;
>     }
>     size = sizeof ( double ) * 6;
>     $1   = (double **) malloc( sizeof ( double * ) * Alen );
>     for ( i = 0; i < Alen; i++ )
>         $1[i] = (double *) ( tmp->data + i * size );
> }
> %typemap( freearg ) double **OutMatrixCkCk6 {
>     Py_DECREF( tmp$argnum );
>     free( $1 );
> }
>
> There is some extra specifics in here for checking on the
> dimensionality of the NumPy arrays for my particular needs. You should
> ignore those specifics but the basic idea should be clear.  The hack
> is that when my C code overwrites the contents of the above C calling
> code's $1 arrays, they simply overwrite the block of memory pointed to
> by the NumPy arrays. This works because the way I have arranged my C
> code 2D arrays happens to conform to the method that NumPy uses (by
> happy accident since I cannot find the appropriate NumPy documentation
> of their C implementation).  But overwriting the contents of preexisting NumPy
> arrays is a massive hack that depends on NumPy implementation details
> and violates Python user assumptions about the sanctity of input arguments.
> I also use similar hacks to copy the contents of 1D int * and double *
> C arrays to the NumPy equivalent.
>
> I would like to replace the above typemaps (and the equivalent 1D ones
> for "int *" and "double *") with typemaps that do the following:
>
> 1. At the C language level create NumPy arrays of the
> correct dimensionality and type (probably using PyArray_NewFromDescr).
>
> 2. Copy the contents of C 2D (typed char** and double**) and 1D (typed
> int * and double *) arrays to those NumPy arrays.  I could use the
> method above to just allow my C code to overwrite the contents of the
> NumPy object created in step 1, but ideally there is a NumPy function
> to do this instead since conceivably NumPy could change their internal
> implementation so the present method no longer works.
>
> 3. Return those generated and filled NumPy arrays as Python objects
> contained inside a normal Python return tuple using the appropriate
> call to Py_BuildValue.
>
> In principle, I should be able to figure out how to do all three of
> these typemap details but I have absolutely no experience with
> PyArray_NewFromDescr, Py_BuildValue, or output typemaps for Python.
> Furthermore, I cannot find any NumPy documentation about the best
> method of copying C arrays to NumPy arrays.  Advice/help on any of
> these topics would be much appreciated.
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Swig-user mailing list
> Swi...@li...
> https://lists.sourceforge.net/lists/listinfo/swig-user
>



-- 
======================
Nickolas Fotopoulos
Postdoctoral Scholar
LIGO Laboratory, Caltech
Phone: 626-395-8740
Fax: 626-304-9834
======================