numpy-discussion

[Numpy-discussion] ctypes, numpy-array

[Lars F. <lfr...@im...>]

Hello,

I would like to work with some data using python/numpy. The data is
generated with C. To bring the data from C to python, I would like to
use ctype. I am using python 2.4.3 on a windows-System.

To accomplish the described task, I have the following plan. Please tell
me, if this is possible, respectively if there is a better way.

1) In C, I write a .dll, that has a function
int foo(PyObject *arg)

2) In python, I generate a numpy-array with the appropriate size. e.g.
a = zeros((3,3), dtype=int)

3) From python, I call my .dll-function with a as an argument:
windll.mydll.foo(a)

4) In the foo-function in the C-.dll I cast the pointer and access the
data-field.
PyArrayObject *myPtr = (PyArrayObject*) arg;
myPtr->data[0] = 1;
return 0;

However, when I do this, I get an "AccessViolationError writing
0x000000000"

What can I do about it? Thank you for every comment

Lars


-- 
Dipl.-Ing. Lars Friedrich
Optical Measurement Technology
Department of Microsystems Engineering -- IMTEK
University of Freiburg
Georges-Köhler-Allee 102
D-79110 Freiburg
Germany

phone: +49-761-203-7531
fax:   +49-761-203-7537
room:  01 088
email: lfr...@im...

Re: [Numpy-discussion] ctypes, numpy-array

[Albert S. <fu...@gm...>]

Hey Lars

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Lars Friedrich
> Sent: 25 July 2006 12:50
> To: num...@li...
> Subject: [Numpy-discussion] ctypes, numpy-array
> 
> Hello,
> 
> I would like to work with some data using python/numpy. The data is
> generated with C. To bring the data from C to python, I would like to
> use ctype. I am using python 2.4.3 on a windows-System.
> 
> To accomplish the described task, I have the following plan. Please tell
> me, if this is possible, respectively if there is a better way.
> 
> 1) In C, I write a .dll, that has a function
> int foo(PyObject *arg)
> 
> 2) In python, I generate a numpy-array with the appropriate size. e.g.
> a = zeros((3,3), dtype=int)
> 
> 3) From python, I call my .dll-function with a as an argument:
> windll.mydll.foo(a)

What's might be happening here is that a.ctypes.data is in fact being passed
to your function via ctypes's from_param magic (check the ctypes tutorial
for details).

In [10]: x = N.array([])

In [11]: x.ctypes.data
Out[11]: c_void_p(15502816)

In [12]: x._as_parameter_
Out[12]: 15502816
 
> 4) In the foo-function in the C-.dll I cast the pointer and access the
> data-field.
> PyArrayObject *myPtr = (PyArrayObject*) arg;
> myPtr->data[0] = 1;
> return 0;
>
> However, when I do this, I get an "AccessViolationError writing
> 0x000000000"

So what's probably happening here is that you already have a pointer to the
array's data which you then cast to a PyArrayObject pointer. Dereferencing
myPtr->data is looking for a pointer inside the array's data, which contains
zeros.

Here's a few things to try:

- look at ctypes's PyDLL option if you want to pass around Python objects

- Write your function as:

int foo(int* x);

Then do something like this:

x = N.array([...], dtype=N.intc)
mydll.foo.restype = None
mydll.foo.argtypes = [POINTER(c_int)]
mydll.foo(x.ctypes.data)

This might also work:

x = N.array([...], dtype=N.intc)
mydll.foo.restype = None
mydll.foo(x)

Cheers,

Albert

Re: [Numpy-discussion] ctypes, numpy-array

[Lars F. <lfr...@im...>]

> What's might be happening here is that a.ctypes.data is in fact being passed
> to your function via ctypes's from_param magic (check the ctypes tutorial
> for details).
> 
> In [10]: x = N.array([])
> 
> In [11]: x.ctypes.data
> Out[11]: c_void_p(15502816)
> 
> In [12]: x._as_parameter_
> Out[12]: 15502816
>  
OK, I did not know about x.ctypes.data...

> So what's probably happening here is that you already have a pointer to the
> array's data which you then cast to a PyArrayObject pointer. Dereferencing
> myPtr->data is looking for a pointer inside the array's data, which contains
> zeros.

I understand. 

> - look at ctypes's PyDLL option if you want to pass around Python objects

???

> - Write your function as:
> 
> int foo(int* x);
> 
> Then do something like this:
> 
> x = N.array([...], dtype=N.intc)
> mydll.foo.restype = None
> mydll.foo.argtypes = [POINTER(c_int)]
> mydll.foo(x.ctypes.data)

I did that, and it worked fine for me. Thank you very much! This is
really great.

Lars

Re: [Numpy-discussion] ctypes, numpy-array

[Albert S. <fu...@gm...>]

Hello all

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Lars Friedrich
> Sent: 25 July 2006 13:55
> To: num...@li...
> Subject: Re: [Numpy-discussion] ctypes, numpy-array
> 
> > What's might be happening here is that a.ctypes.data is in fact being
> passed
> > to your function via ctypes's from_param magic (check the ctypes
> tutorial
> > for details).
> >
> > In [10]: x = N.array([])
> >
> > In [11]: x.ctypes.data
> > Out[11]: c_void_p(15502816)
> >
> > In [12]: x._as_parameter_
> > Out[12]: 15502816
> >
> OK, I did not know about x.ctypes.data...

Travis added this quite recently. Somebody (probably me) still has to update
the wiki to reflect these changes.

> > So what's probably happening here is that you already have a pointer to
> the
> > array's data which you then cast to a PyArrayObject pointer.
> Dereferencing
> > myPtr->data is looking for a pointer inside the array's data, which
> contains
> > zeros.
> 
> I understand.
> 
> > - look at ctypes's PyDLL option if you want to pass around Python
> objects
> 
> ???

You can read about PyDLL here:

http://docs.python.org/dev/lib/ctypes-loading-shared-libraries.html

I think PyDLL might turn out to be an interesting alternative to traditional
extension modules. But for wrapping C code, I think writing functions that
operate on pointers to ints and floats and whatnot works nicely.
 
> > - Write your function as:
> >
> > int foo(int* x);
> >
> > Then do something like this:
> >
> > x = N.array([...], dtype=N.intc)
> > mydll.foo.restype = None

Slight typo on my part. For this example it should be:

mydll.foo.restype = c_int

> > mydll.foo.argtypes = [POINTER(c_int)]
> > mydll.foo(x.ctypes.data)
> 
> I did that, and it worked fine for me. Thank you very much! This is
> really great.

Cool. Enjoy!

Regards,

Albert

Re: [Numpy-discussion] ctypes, numpy-array

[Christopher B. <Chr...@no...>]

Lars Friedrich wrote:
> I would like to work with some data using python/numpy. The data is
> generated with C. To bring the data from C to python, I would like to
> use ctype. I am using python 2.4.3 on a windows-System.
> 
> To accomplish the described task, I have the following plan. Please tell
> me, if this is possible, respectively if there is a better way.
> 
> 1) In C, I write a .dll, that has a function
> int foo(PyObject *arg)

I'm a bit confused here. I thought the primary point of ctypes was to be 
able to access existing, non-python-aware dlls from Python without 
writing C code.

In this case, you're writing a dll that understands PyObjects (or, I 
assume, a particular PyObject -- a numarray). Why not just forget ctypes 
and write a regular old extension?

Or use Pyrex, or Boost, or.....

Maybe ctypes has some real advantages I don't get.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] ctypes, numpy-array

[Mark H. <ma...@mi...>]

Christopher Barker wrote:
> Lars Friedrich wrote:
>   
>> I would like to work with some data using python/numpy. The data is
>> generated with C. To bring the data from C to python, I would like to
>> use ctype. I am using python 2.4.3 on a windows-System.
>>
>> To accomplish the described task, I have the following plan. Please tell
>> me, if this is possible, respectively if there is a better way.
>>
>> 1) In C, I write a .dll, that has a function
>> int foo(PyObject *arg)
>>     
>
> I'm a bit confused here. I thought the primary point of ctypes was to be 
> able to access existing, non-python-aware dlls from Python without 
> writing C code.
>
> In this case, you're writing a dll that understands PyObjects (or, I 
> assume, a particular PyObject -- a numarray). Why not just forget ctypes 
> and write a regular old extension?
>
>   
See Albert's post up thread. Agreed, if he was going to go to the 
trouble of using the Py C API then he didn't really need ctypes. What 
the original poster wanted (Im assuming) was something much simpler like

int foo(int* x)

as Albert suggested, and then use the new numpy ctypes atrributes and 
ctypes 'argtypes' type mapping facility to pass the data contained in 
the py object into a simple routine. That then, is the advantage of ctypes.

Mark

Re: [Numpy-discussion] ctypes, numpy-array

[Christopher B. <Chr...@no...>]

Lars Friedrich wrote:
>> In this case, you're writing a dll that understands PyObjects (or, I 
>> assume, a particular PyObject -- a numarray). Why not just forget ctypes 
>> and write a regular old extension?
> good point. I am relatively new to Python. The first thing I did was
> trying to write a regular extension. The problem is, that I *have to*
> use a windows-machine. And at the moment, only Visual C++ 6.0 is
> available here. The problem is that for a regular extension, Python and
> the extension need to be compiled by the same compiler AFAIK.

That's mostly true. You have three options:

1) re-compile python yourself -- but then you'd also have to re-compile 
all the other extensions you use!

2) You can also use MingGW to compile extensions -- it takes a bit of 
kludging, but it can be done, and works fine once you've got it set up. 
Google will help you figure out how -- it's been a while since I've done it.

I have to say that I find it ironic that you can use MinGW, but not 
other versions of the MS compiler!

3) MS distributes a command line version of their compiler for free that 
can be used. Again, google should help you find out how to do that.


However, as other posters mentioned, you can use ctypes as it was 
intended with numpy -- that may be the way to go

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...