[Numpy-discussion] Bugfix for arraybase.h

[Giles H. <gh...@re...>]

Hi,

This simple python code snippit used to core dump on my modern AMD 64
Linux servers:

from numarray  import *
a = zeros( 10, Float32 )
b = transpose(a)
dot( b,a )

This crash was caused by a corrupted "libnumarray_API".  The pointer was
initially assigned the correct value, but becomes corrupted almost
immediately after its initial runtime binding.  On my systems, line 725
of _dotblas.c would clobber the "libnumarray_API" pointer while
initializing the "dotFunctions" vector table.  the problem is
"dotFunctions", an array of function pointers, is sized statically by
"PyArray_NTYPES", an enumerated constant that reflects the value of
"tMaxType" defined in arraybase.h like this:

typedef enum
{
  tAny,
  tBool,
  tInt8,
  tUInt8,
  tInt16,
  tUInt16,
  tInt32,
  tUInt32,
  tInt64,
  tUInt64,
  tFloat32,
  tFloat64,
  tComplex32,
  tComplex64,
  tObject,                   /* placeholder... does nothing */
  tDefault = tFloat64,
#if LP64
  tLong = tInt64,
#else
  tLong = tInt32,
#endif
  tMaxType
} NumarrayType;

The problem is that "tMaxType" is not assigned 17, like we would expect,
since it's at the end of the list.  On my machine, "tMaxType" is
assigned the value 9!  The problem is the explicit assignment of "tLong"
and "tDefault".  Explicitly assigned constants in an enumerated list
will modify the running count.  Since the macro "LP64" is defined,
"tLong" is assigned value of "tInt64", which is 8, and "tMaxType" is
assigned the next value, 9.  This means the vector table "dotFunctions"
is shorter then we intend, and overflowing this table can corrupt
important data.  The fix is simple, just define tMaxType before the
explicitly assigned aliases:

typedef enum
{
  tAny,
  tBool,
  tInt8,
  tUInt8,
  tInt16,
  tUInt16,
  tInt32,
  tUInt32,
  tInt64,
  tUInt64,
  tFloat32,
  tFloat64,
  tComplex32,
  tComplex64,
  tObject,                   /* placeholder... does nothing */
  tMaxType,
  tDefault = tFloat64,
#if LP64
  tLong = tInt64,
#else
  tLong = tInt32,
#endif
} NumarrayType;