Re: [Numpy-discussion] copying ctypes arrays to numarray?

[rays <ra...@sa...>]

I'm re-posting this here to see if anyone can shed light on numarray.info() 
behavior, and why assignment from a (ctypes.c_long * 2000)() to a numarray 
is so much slower than a memmove().

I was surprised that assignment wasn't faster and that numarray assignment 
was consistently ~.5% faster than Numeric.
The N vs. n memmove() flip-flopped for fastest, with array.array always slower.

After some discussion, I made a better direct comparison of ctypes and 
memmove, map etc., using all the methods suggested so far. I think it's 
fairly valid.(?) Run on a 2.4GHz WinXP, Py2.3, a number of times.

Parsing numarray.info() was a royal pain, it writes directly to stdout!
I.e.:
 >>> inf = src.info()
class: <class 'numarray.numarraycore.NumArray'>
shape: (32,)
strides: (8,)
byteoffset: 0
bytestride: 8
itemsize: 8
aligned: 1
contiguous: 1
data: <memory at 00815950 with size:256 held by object 00815930 aliasing object
00000000>
byteorder: little
byteswap: 0
type: Float64
 >>> inf
 >>> type(inf)
<type 'NoneType'>


Since the ctype does support slicing, I was considering leaving the data in 
the device driver buffer (~1MB circular) and poking into it, but memmove is 
so much faster than slicing the ctype, I'm doing memmove()s to numarray.
I presume that I should check for numarray.iscontiguous( ) or is_c_array( 
)  first to be safe...

Results and code below.

Thanks to all for the help,
Ray



 >python test.py
Array address 11443208
n address 17039424
N address 21102656
for loop  2027.0 us ea
map       1781.0 us ea
slice     1704.0 us ea
assign  N 1244.0 us ea
assign  n 1242.0 us ea
memmove   4.3831 us ea
memmove N 3.4773 us ea
memmove n 3.4803 us ea

_____________________________________________________

## test.py
import array
import numarray
import ctypes
import time
import string
import StringIO
import sys

buf = (ctypes.c_long * 2000)()
Array = array.array("l", [0]*10000)
n = numarray.zeros((1000000), numarray.Int32)
N = numarray.zeros((1000000), numarray.Int32)

#!!!!!!!!!!!!!!! arrrrgggg!  !!!!!!!!!!!!!!!!
# n.info() writes directly to stdout!
stdout = sys.stdout
fileo = StringIO.StringIO()
sys.stdout = fileo
n.info()
ninfo = fileo.getvalue( )
fileo.close()
sys.stdout = stdout

stdout = sys.stdout
fileo = StringIO.StringIO()
sys.stdout = fileo
N.info()
Ninfo = fileo.getvalue( )
fileo.close()
sys.stdout = stdout
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

print 'Array address', Array.buffer_info()[0]

ninfo = string.split(ninfo)
nAddress = int(ninfo[20], 16)
print 'n address', nAddress
Ninfo = string.split(Ninfo)
NAddress = int(Ninfo[20], 16)
print 'N address', NAddress

t0 = time.clock()
for loop in range(1000):
     for i in range(2000):
         n[loop+i] = buf[i]
print 'for loop ', round((time.clock()-t0)*1000), 'us ea'

t0 = time.clock()
for loop in range(1000):
     n[loop:loop+2000] = map(None, buf)
print 'map      ', round((time.clock()-t0)*1000), 'us ea'

t0 = time.clock()
for loop in range(1000):
     n[loop:loop+2000] = buf[0:2000]
print 'slice    ', round((time.clock()-t0)*1000), 'us ea'

t0 = time.clock()
for loop in range(10000):
     N[loop:loop+2000] = buf
print 'assign  N', round((time.clock()-t0)*100), 'us ea'

t0 = time.clock()
for loop in range(10000):
     n[loop:loop+2000] = buf
print 'assign  n', round((time.clock()-t0)*100), 'us ea'

t0 = time.clock()
for loop in range(10000):
     ctypes.memmove(10+Array.buffer_info()[0],
                      buf,
                      2000)
print 'memmove  ', round((time.clock()-t0)*1, 4), 'us ea'

t0 = time.clock()
for loop in range(10000):
     ctypes.memmove(10+NAddress,
                      buf,
                      2000)
print 'memmove N', round((time.clock()-t0)*1, 4), 'us ea'

t0 = time.clock()
for loop in range(10000):
     ctypes.memmove(10+nAddress,
                      buf,
                      2000)
print 'memmove n', round((time.clock()-t0)*1, 4), 'us ea'