numpy-discussion

[Numpy-discussion] Array interface updated to Version 3

[Travis O. <oli...@ie...>]

I just updated the array interface page to emphasize we now have version 
3.  NumPy still supports  objects that expose (the C-side) of version 2 
of the array interface, though. 

The new interface is basically the same except (mostly) for asthetics:  
The differences are listed at the bottom of

http://numeric.scipy.org/array_interface.html

There is talk of ctypes supporting the new interface which is a worthy 
development.  Please encourage that if you can.

Please voice concerns now if you have any.

-Travis

Re: [Numpy-discussion] Array interface updated to Version 3

[Fernando P. <fpe...@gm...>]

On 6/16/06, Travis Oliphant <oli...@ie...> wrote:

> There is talk of ctypes supporting the new interface which is a worthy
> development.  Please encourage that if you can.

That would certainly be excellent, esp. given how ctypes is slated to
be officially part of python 2.5.  I think it would greatly improve
the interoperability landscape for python if the out-of-the-box
toolset had proper access to numpy arrays.

Cheers,

f

Re: [Numpy-discussion] Array interface updated to Version 3

[Andrew S. <str...@as...>]

I noticed in your note labeled 'June 16, 2006' that you refer to the 
"desc" field. However, in the struct description above, there is only a 
field named "descr".

Also, I suggest that you update the information in the comments of descr 
field of the structure description to contain the fact that inter.descr 
is a reference to a tuple equal to ("PyArrayInterface Version 
#",new_tuple_with_array_interface). What is currently there seems out of 
date given the new information. Finally, in the comment  section 
describing this field, I strongly suggesting noting that this field is 
only present *if and only if* the ARR_HAS_DESCR flag is present. It will 
be more clear if it's there rather than in the text underneath.

Is the "#" in the string meant to be replaced with "3"? If so, why not 
write 3? Also, in your note, you should explain whether "dummy" (renamed 
from "version") should still be checked as a sanity check or whether it 
should now be ignored. I think we could call the field "two" and keep 
the sanity check for backwards compatibility. I agree it is confusing to 
have two different version numbers in the same struct, so I don't mind 
having the official name of the field being something other than 
"version", but if we keep it as a required sanity check (in which case 
it probably shouldn't be named "dummy"), the whole thing will remain 
backwards compatible with all current code.

Anyhow, I'm very excited about this array interface, and I await the 
outcome of the Summer of Code project on the 'micro-array' 
implementation based on it!

Cheers!
Andrew

Travis Oliphant wrote:

>I just updated the array interface page to emphasize we now have version 
>3.  NumPy still supports  objects that expose (the C-side) of version 2 
>of the array interface, though. 
>
>The new interface is basically the same except (mostly) for asthetics:  
>The differences are listed at the bottom of
>
>http://numeric.scipy.org/array_interface.html
>
>There is talk of ctypes supporting the new interface which is a worthy 
>development.  Please encourage that if you can.
>
>Please voice concerns now if you have any.
>
>-Travis
>
>
>
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>

Re: [Numpy-discussion] Array interface updated to Version 3

[Albert S. <fu...@gm...>]

Hello all

On Fri, 16 Jun 2006, Travis Oliphant wrote:

> I just updated the array interface page to emphasize we now have version 
> 3.  NumPy still supports  objects that expose (the C-side) of version 2 
> of the array interface, though. 

<snip>

> Please voice concerns now if you have any.

In the documentation for the data attribute you say:

"A reference to the object with this attribute must be stored by the 
new object if the memory area is to be secured."

Does that mean a reference to the __array_interface__ or a reference to 
the object containing the __array_interface__?

Regards,

Albert

Re: [Numpy-discussion] Array interface updated to Version 3

[Thomas H. <th...@py...>]

Travis Oliphant schrieb:
> I just updated the array interface page to emphasize we now have version 
> 3.  NumPy still supports  objects that expose (the C-side) of version 2 
> of the array interface, though. 
> 
> The new interface is basically the same except (mostly) for asthetics:  
> The differences are listed at the bottom of
> 
> http://numeric.scipy.org/array_interface.html
> 
> There is talk of ctypes supporting the new interface which is a worthy 
> development.  Please encourage that if you can.
> 
> Please voice concerns now if you have any.

 From http://numeric.scipy.org/array_interface.html:
"""
New since June 16, 2006:
For safety-checking the return object from PyCObject_GetDesc(obj) should 
be a Python Tuple with the first object a Python string containing 
"PyArrayInterface Version 3" and whose second object is a reference to 
the object exposing the array interface (i.e. self).

Older versions of the interface used the "desc" member of the PyCObject 
itself (do not confuse this with the "descr" member of the 
PyArrayInterface structure above --- they are two separate things) to 
hold the pointer to the object exposing the interface, thus you should 
make sure the object returned is a Tuple before assuming it is in a 
sanity check.

In a sanity check it is recommended to only check for "PyArrayInterface 
Version" and not for the actual version number so that later versions 
will still be compatible.  The old sanity check for the integer 2 in the 
first field is no longer necessary (but it is necessary to place the 
number 2 in that field so that objects reading the old version of the 
interface will still understand this one).
"""

I know that you changed that because of my suggestions, but I don't 
think it should stay like this.

The idea was to have the "desc" member of the PyCObject a 'magic value'
which can be used to determine that the PyCObjects "void *cobj" pointer
really points to a PyArrayInterface structure.  I have seen PyCObject 
uses before in this way, but I cannot find them any longer.

If current implementations of the array interface use this pointer for
other things (like keeping a reference to the array object), that's 
fine, and I don't think the specification should change.  I think it is
espscially dangerous to assume that the desc pointer is a PyObject 
pointer,  Python will segfault if it is not.

I suggest that you revert this change.

Thomas

Re: [Numpy-discussion] Array interface updated to Version 3

[Travis O. <oli...@ie...>]

Thomas Heller wrote:
> Travis Oliphant schrieb:
>> I just updated the array interface page to emphasize we now have 
>> version 3.  NumPy still 
>
> If current implementations of the array interface use this pointer for
> other things (like keeping a reference to the array object), that's 
> fine, and I don't think the specification should change.  I think it is
> espscially dangerous to assume that the desc pointer is a PyObject 
> pointer,  Python will segfault if it is not.

You make a good point.   This is not a very safe sanity check and overly 
complicated for not providing safety.  I've reverted it back but left in 
the convention that the 'desc' pointer contain a reference to the object 
exposing the interface as is the practice now.

Thanks for the review.

-Travis

Re: [Numpy-discussion] Array interface updated to Version 3

[David M. C. <co...@ph...>]

On Tue, 20 Jun 2006 21:05:51 +0200
Thomas Heller <th...@py...> wrote:

> Travis Oliphant schrieb:
> > I just updated the array interface page to emphasize we now have version 
> > 3.  NumPy still supports  objects that expose (the C-side) of version 2 
> > of the array interface, though. 
> > 
> > The new interface is basically the same except (mostly) for asthetics:  
> > The differences are listed at the bottom of
> > 
> > http://numeric.scipy.org/array_interface.html
> > 
> > There is talk of ctypes supporting the new interface which is a worthy 
> > development.  Please encourage that if you can.
> > 
> > Please voice concerns now if you have any.
> 
>  From http://numeric.scipy.org/array_interface.html:
> """
> New since June 16, 2006:
> For safety-checking the return object from PyCObject_GetDesc(obj) should 
> be a Python Tuple with the first object a Python string containing 
> "PyArrayInterface Version 3" and whose second object is a reference to 
> the object exposing the array interface (i.e. self).
> 
> Older versions of the interface used the "desc" member of the PyCObject 
> itself (do not confuse this with the "descr" member of the 
> PyArrayInterface structure above --- they are two separate things) to 
> hold the pointer to the object exposing the interface, thus you should 
> make sure the object returned is a Tuple before assuming it is in a 
> sanity check.
> 
> In a sanity check it is recommended to only check for "PyArrayInterface 
> Version" and not for the actual version number so that later versions 
> will still be compatible.  The old sanity check for the integer 2 in the 
> first field is no longer necessary (but it is necessary to place the 
> number 2 in that field so that objects reading the old version of the 
> interface will still understand this one).
> """
> 
> I know that you changed that because of my suggestions, but I don't 
> think it should stay like this.
> 
> The idea was to have the "desc" member of the PyCObject a 'magic value'
> which can be used to determine that the PyCObjects "void *cobj" pointer
> really points to a PyArrayInterface structure.  I have seen PyCObject 
> uses before in this way, but I cannot find them any longer.
> 
> If current implementations of the array interface use this pointer for
> other things (like keeping a reference to the array object), that's 
> fine, and I don't think the specification should change.  I think it is
> espscially dangerous to assume that the desc pointer is a PyObject 
> pointer,  Python will segfault if it is not.

> I suggest that you revert this change.

When I initially proposed the C version of the array interface, I suggested
using a magic number, like 0xDECAF (b/c it's lightweight :-) as the first
member of the CObject. Currenty, we use a version number, but I believe that
small integers would be more common in random CObjects than a magic number. We
could do similiar, using 0xDECAF003 for version 3, for instance.

That would keep most of the benefits of an explicit "this is an array
interface" CObject token, but is lighter to check, and doesn't impose any
constraints on implementers for their desc fields.

One of the design goals for the C interface was speed; doing a check that the
first member of a tuple begins with a certain string slows it down.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...