[Numpy-discussion] Possible inconsisteny in enumerated type mapping

[Francesc A. <fa...@ca...>]

Hi,

I'm sending a message here because discussing about this in the bug tracker=
 is=20
not very comfortable. This my last try before giving up, so don't  be=20
afraid ;-)

In bug #283 (http://projects.scipy.org/scipy/numpy/ticket/283) I complained=
=20
about the fact that a numpy.int32 is being mapped in NumPy to NPY_LONG=20
enumerated type and I think I failed to explain well why I think this is a=
=20
bad thing. Now, I'll try to expose an (real life) example, in the hope that=
=20
things will make clearer.

Realize that you are coding a C extension that receives NumPy arrays for=20
saving them on-disk for a later retrieval. Realize also that an user is usi=
ng=20
your extension on a 32-bit platform. If she pass to this extension an array=
=20
of type 'int32', and the extension tries to read the enumerated type (using=
=20
array.dtype.num), it will get NPY_LONG. So, the extension use this code=20
(NPY_LONG) to save the type (together with data) on-disk. Now, she send thi=
s=20
data file to a teammate that works on a 64-bit machine, and tries to read t=
he=20
data using the same extension. The extension would see that the data is=20
NPY_LONG type and would try to deserialize interpreting data elements as=20
being as 64-bit integer (this is the size of a NPY_LONG in 64-bit platforms=
),=20
and this is clearly wrong.

Besides this, if for making your C extension you are using a C library that=
 is=20
meant to save data in a platform-independent (say, HDF5), then, having a=20
NPY_LONG will not automatically say which C library datatype maps to, becau=
se=20
it only have datatypes that are of a definite size in all platforms. So, th=
is=20
is a second problem.

Of course there are workarounds for this, but my impression is that they ca=
n=20
be avoided with a more sensible mapping between NumPy Python types and NumP=
y=20
enumerated types, like:

numpy.int32 --> NPY_INT
numpy.int64 --> NPY_LONGLONG
numpy.int_  --> NPY_LONG

in all platforms, avoiding the current situation of ambiguous mapping betwe=
en=20
platforms.

Sorry for being so persistent, but I think the issue is worth it.

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"