Menu
▾
▴
numpy-discussion — Discussion list for all users of Numerical Python
You can subscribe to this list here.
| 2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
| 2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
| 2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
| 2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
| 2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
| 2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
|
From: Todd M. <jm...@st...> - 2005-11-03 18:09:07
|
Chris Fonnesbeck wrote: >Hi, > >Compiling 1.4.1 under OSX with either gcc 3.3 or 4.0 fails (all >previous versions of numarray compiled without error). Here is the >full output: > > > What version of OS-X are your running? Where is the actual error? (There are a lot of warnings now on OS-X, but 1.4.1 builds fine for me.) Todd >Using EXTRA_COMPILE_ARGS = ['-Ddarwin'] >Using external BLAS and LAPACK >running config >Wrote config.h >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Src/_convmodule.c:13: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Src/_sortmodule.c:13: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Src/_bytesmodule.c:13: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > >... several dozen of the above, then ... > >Packages/LinearAlgebra2/Src/lapack_litemodule.c:12: warning: function >declaration isn't a prototype >Packages/LinearAlgebra2/Src/lapack_litemodule.c:17: warning: function >declaration isn't a prototype >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgeev': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:107: warning: implicit >declaration of function 'dgeev_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dsyevd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:199: warning: implicit >declaration of function 'dsyevd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zheevd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:296: warning: implicit >declaration of function 'zheevd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgelsd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:340: warning: implicit >declaration of function 'dgelsd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgesv': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:378: warning: implicit >declaration of function 'dgesv_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgesdd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:418: warning: implicit >declaration of function 'dgesdd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgetrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:449: warning: implicit >declaration of function 'dgetrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zungqr': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:477: warning: implicit >declaration of function 'zungqr_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dorgqr': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:508: warning: implicit >declaration of function 'dorgqr_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgeqrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:539: warning: implicit >declaration of function 'zgeqrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dgeqrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:570: warning: implicit >declaration of function 'dgeqrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_dpotrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:594: warning: implicit >declaration of function 'dpotrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgeev': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:632: warning: implicit >declaration of function 'zgeev_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgelsd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:679: warning: implicit >declaration of function 'zgelsd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgesv': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:714: warning: implicit >declaration of function 'zgesv_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgesdd': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:759: warning: implicit >declaration of function 'zgesdd_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zgetrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:794: warning: implicit >declaration of function 'zgetrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function >'lapack_lite_zpotrf': >Packages/LinearAlgebra2/Src/lapack_litemodule.c:818: warning: implicit >declaration of function 'zpotrf_' >Packages/LinearAlgebra2/Src/lapack_litemodule.c: At top level: >Packages/LinearAlgebra2/Src/lapack_litemodule.c:851: warning: function >declaration isn't a prototype >Packages/LinearAlgebra2/Src/lapack_litemodule.c:848: warning: >'lapack_liteError' defined but not used >powerpc-apple-darwin8-gcc-4.0.0: -framework: linker input file unused >because linking not done >powerpc-apple-darwin8-gcc-4.0.0: vecLib: linker input file unused >because linking not done >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Packages/RandomArray2/Src/ranlib.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Packages/RandomArray2/Src/ranlibmodule.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Packages/RandomArray2/Src/com.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Src/_dotblas.c:12: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from Src/_dotblas.c:15: >/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:665: >warning: function declaration isn't a prototype >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/high_levelmodule.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/element_wisemodule.c:3: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/one_dimensionalmodule.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/numpy_compatmodule.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/numpy_compat2.c:1: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list >In file included from >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, > from Examples/convolve/testlite.c:26: >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: >warning: 'struct winsize' declared inside parameter list >/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: >warning: 'struct winsize' declared inside parameter list > > >Any ideas for a solution to this problem are welcome. > >-- >Chris Fonnesbeck >Atlanta, GA > > >------------------------------------------------------- >SF.Net email is sponsored by: >Tame your development challenges with Apache's Geronimo App Server. Download >it for free - -and be entered to win a 42" plasma tv or your very own >Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php >_______________________________________________ >Numpy-discussion mailing list >Num...@li... >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > |
|
From: Rick W. <rl...@st...> - 2005-11-03 14:00:47
|
On Wed, 2 Nov 2005, Russell E. Owen wrote: > To use documented interfaces (i.e. not arra._byteorder) and to avoid > byteswapping the input array, I think I'm going to be stuck doing > something like: > > import sys > if sys.byteorder == 'big': > isBigendian = not arr.isbyteswapped() > else: > isBigendian = arr.isbyteswapped() A simpler version: isBigendian = arr.isbyteswapped() != numarray.isBigEndian |
|
From: Francesc A. <fa...@ca...> - 2005-11-03 08:32:26
|
El dc 02 de 11 del 2005 a les 17:16 -0500, en/na Todd Miller va
escriure:
> I added support for the newcore "array interface" to the numarray C-API=20
> and this exception is now gone. The fix/enhancement is checked into=20
> CVS. Kick it around some...
Yes. This issue has been solved:
>>> numarray.__version__
'1.4.2'
>>> Numeric.__version__
'24.0'
>>> num=3DNumeric.zeros((3,),'f')
>>> na=3Dnumarray.ones((3,),'f')
>>> na[...] =3D num
>>> na
array([ 0., 0., 0.], type=3DFloat32)
Now, I've taken the opportunity and tested the Numeric<-->numarray
conversion again:
>>> t1_1=3Dtimeit.Timer("num=3DNumeric.array(na,copy=3D1)","import
Numeric;import numarray; na=3Dnumarray.arange(10)")
>>> t1_1.repeat(3,10000)
[0.25493311882019043, 0.25352001190185547, 0.25211095809936523]
and this figures compared with numarray 1.4.1:
>>> t1.repeat(3,10000)
[0.59375977516174316, 0.57908082008361816, 0.56574010848999023]
shows that you have accelerated the conversion numarray-->Numeric=20
more than 2x.
Now, the Numeric-->numarray:
>>> t3_1=3Dtimeit.Timer("na=3Dnumarray.array(num,copy=3D1)","import Numeric=
;import numarray; num=3DNumeric.arange(10)")
>>> t3_1.repeat(3,10000)
[0.40311408042907715, 0.40207314491271973, 0.39954400062561035]
while the 1.4.1 performance was:
>>> t3.repeat(3,10000)
[1.3475611209869385, 1.3277668952941895, 1.3417830467224121]
which is a factor 3x of improvement, and almost the same than using
the buffer interface. Very nice job, Todd!
Now, let's test the conversion for large objects, with data copy and
without data copy.
For numarray-->Numeric:
>>> t2=3Dtimeit.Timer("num=3DNumeric.array(na,copy=3D0)","import
Numeric;import numarray; na=3Dnumarray.arange(100000)")
>>> t2.repeat(3,100)
[0.0025169849395751953, 0.0025219917297363281, 0.0024950504302978516]
>>> t2_copy=3Dtimeit.Timer("num=3DNumeric.array(na,copy=3D1)","import
Numeric;import numarray; na=3Dnumarray.arange(100000)")
>>> t2_copy.repeat(3,100)
[0.50105500221252441, 0.49400091171264648, 0.49266600608825684]
So it seems like if the data copy is taking place.
For Numeric-->numarray:
>>> t4=3Dtimeit.Timer("na=3Dnumarray.array(num,copy=3D0)","import
Numeric;import numarray; num=3DNumeric.arange(100000)")
>>> t4.repeat(3,100)
[0.0054900646209716797, 0.0044760704040527344, 0.0048201084136962891]
>>> t4_copy=3Dtimeit.Timer("na=3Dnumarray.array(num,copy=3D1)","import
Numeric;importnumarray; num=3DNumeric.arange(100000)")
>>> t4_copy.repeat(3,100)
[0.0063569545745849609, 0.004302978515625, 0.0042738914489746094]
Ooops! the times for conversion with copy and without copy are more or
less the same. Perhaps the copy is not done? It seems so:
>>> num=3DNumeric.arange(10)
>>> na=3Dnumarray.array(num,copy=3D0)
>>> na_copy=3Dnumarray.array(num,copy=3D1)
>>> na.info()
...
data pointer: 0x08312280 (DEBUG ONLY)
>>> na_copy.info()
...
data pointer: 0x08312280 (DEBUG ONLY)
i.e. the same data pointer. So it seems that the Numeric-->numarray is
not copying the data even in the case that we are asking to do that.
Other minor things (maybe this is just because you are in the middle of
refactoring):
>>> na._data
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
while I would expect:
>>> na._data
<memory at 0x081e3618 with size:0x00000028 held by object 0x401f06a0
aliasing object 0x00000000>
Also:
>>> na
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line
930, in __repr__
return array_repr(self)
File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line
1660, in array_repr
lst =3D arrayprint.array2string(
File "/usr/lib/python2.4/site-packages/numarray/arrayprint.py", line
188, in array2string
separator, prefix)
File "/usr/lib/python2.4/site-packages/numarray/arrayprint.py", line
140, in _array2string
data =3D _gen.ravel(a)
File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line
918, in copy
c =3D _gen.NDArray.copy(self)
File "/usr/lib/python2.4/site-packages/numarray/generic.py", line 724,
in copy
arr._itemsize)
TypeError: NA_maybeLongsFromIntTuple: must be a sequence of integers.
Cheers,
--=20
>0,0< Francesc Altet http://www.carabos.com/
V V C=E1rabos Coop. V. Enjoy Data
"-"
|
|
From: Russell E. O. <ro...@ce...> - 2005-11-02 22:52:29
|
In article <113...@lo...>, Francesc Altet <fa...@ca...> wrote: > El dl 31 de 10 del 2005 a les 16:53 -0800, en/na Russell E. Owen va > escriure: > > I've got a module that can send array data to ds9. It screws up on > > byteswapped data and I'm trying to fix it. > > I don't exactly know what a ds9 is, but I'm supposing here that it is > kind of an exotic computer. It's an image display program. > > I need to know if the order is bigendian or littleendian, and I can't > > find a documented way to get that, just an undocumented attribute > > byteorder. > > Yes, it's undocumented, but as far as I can tell, this works flawlessly > in numarray. I'm sure it works now. I was hoping to know if it would continue working in future versions. It'd be nice to have a documented way to get at the info. >... > In general, copy() will return you a well-behaved (native-byte ordered, > non-strided and non-offsetted) array. The manual promises that a copy will be contiguous but I didn't see a promise of native byte order. I have submitted a documentation PR on sourceforge. ... Thanks for the explanation of what byteswapped was doing. That was very helpful. To use documented interfaces (i.e. not arra._byteorder) and to avoid byteswapping the input array, I think I'm going to be stuck doing something like: import sys if sys.byteorder == 'big': isBigendian = not arr.isbyteswapped() else: isBigendian = arr.isbyteswapped() I also submitted a request for a documented direct way to get the info on sourceforge. Regards, -- Russell |
|
From: Todd M. <jm...@st...> - 2005-11-02 22:16:35
|
I added support for the newcore "array interface" to the numarray C-API and this exception is now gone. The fix/enhancement is checked into CVS. Kick it around some... Todd Francesc Altet wrote: >Hi, > >I've recently upgraded to Numeric 24.0 and numarray 1.4.1 and I'm >having a problem that did not exist before: > > > >>>>import Numeric >>>>import numarray >>>>Numeric.__version__ >>>> >>>> >'24.0' > > >>>>numarray.__version__ >>>> >>>> >'1.4.1' > > >>>>na=numarray.zeros((3,),'f') >>>>num=Numeric.zeros((3,),'f') >>>>na[...] = num >>>> >>>> >Traceback (most recent call last): > File "<stdin>", line 1, in ? >ValueError: array sizes must be consistent. > >but... > > > >>>>na=numarray.zeros((3,),'i') >>>>num=Numeric.zeros((3,),'i') >>>>na[...] = num >>>>na >>>> >>>> >array([0, 0, 0]) > >????? > >However, using Numeric 23.8 and numarray 1.3.3: > > > >>>>import Numeric >>>>import numarray >>>>Numeric.__version__ >>>> >>>> >'23.8' > > >>>>numarray.__version__ >>>> >>>> >'1.3.3' > > >>>>na=numarray.zeros((3,),'f') >>>>num=Numeric.zeros((3,),'f') >>>>na[...] = num >>>>na >>>> >>>> >array([ 0., 0., 0.], type=Float32) > >I don't know if the problem is coming from Numeric or numarray, but it >only happens when one mix both: > > > >>>>Numeric.__version__ >>>> >>>> >'24.0' > > >>>>numarray.__version__ >>>> >>>> >'1.4.1' > > >>>>na=numarray.zeros((3,),'f') >>>>na2=numarray.zeros((3,),'f') >>>>na2[...] = na >>>>na2 >>>> >>>> >array([ 0., 0., 0.], type=Float32) > > >>>>num=Numeric.zeros((3,),'f') >>>>num2=Numeric.zeros((3,),'f') >>>>num2[...] = num >>>>num >>>> >>>> >array([ 0., 0., 0.],'f') > >Perhaps the new array protocol is involved here? > > > |
|
From: Chris B. <Chr...@no...> - 2005-11-02 17:43:15
|
There was a discussion about this last month: http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2850560 You should get some good ideas from that. -Chris Philou wrote: > Hi list, > I'm a new subsciber to this numpy list. > > I have a multidimensionnal array (>2). > For example: > x = numarray.array([[[1,2,3],[4,5,6]],[[1,2,3],[4,5,6]]]) > > print x > [[[1 2 3] > [4 5 6]] > > [[1 2 3] > [4 5 6]]] > > I need to enumerate both indices and final values (1D) of the x array. > So i need: > (0,0,0) 1 > (0,0,1) 2 > (0,0,2) 2 > (0,1,0) 4 > (0,1,1) 5 > (0,1,2) 6 > (1,0,0) 1 > (1,0,1) 2 > > How can i do without using the x.flat() method? > > Thanks a lot for the answer. > regards, > Philippe Collet > > > > > ------------------------------------------------------- > SF.Net email is sponsored by: > Tame your development challenges with Apache's Geronimo App Server. > Download > it for free - -and be entered to win a 42" plasma tv or your very own > Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
|
From: Philou <Kal...@wa...> - 2005-11-02 15:32:54
|
Hi list, I'm a new subsciber to this numpy list. I have a multidimensionnal array (>2). For example: x = numarray.array([[[1,2,3],[4,5,6]],[[1,2,3],[4,5,6]]]) print x [[[1 2 3] [4 5 6]] [[1 2 3] [4 5 6]]] I need to enumerate both indices and final values (1D) of the x array. So i need: (0,0,0) 1 (0,0,1) 2 (0,0,2) 2 (0,1,0) 4 (0,1,1) 5 (0,1,2) 6 (1,0,0) 1 (1,0,1) 2 How can i do without using the x.flat() method? Thanks a lot for the answer. regards, Philippe Collet |
|
From: Chris F. <fon...@gm...> - 2005-11-02 13:55:33
|
Hi,
Compiling 1.4.1 under OSX with either gcc 3.3 or 4.0 fails (all
previous versions of numarray compiled without error). Here is the
full output:
Using EXTRA_COMPILE_ARGS =3D ['-Ddarwin']
Using external BLAS and LAPACK
running config
Wrote config.h
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Src/_convmodule.c:13:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Src/_sortmodule.c:13:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Src/_bytesmodule.c:13:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
... several dozen of the above, then ...
Packages/LinearAlgebra2/Src/lapack_litemodule.c:12: warning: function
declaration isn't a prototype
Packages/LinearAlgebra2/Src/lapack_litemodule.c:17: warning: function
declaration isn't a prototype
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgeev':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:107: warning: implicit
declaration of function 'dgeev_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dsyevd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:199: warning: implicit
declaration of function 'dsyevd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zheevd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:296: warning: implicit
declaration of function 'zheevd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgelsd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:340: warning: implicit
declaration of function 'dgelsd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgesv':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:378: warning: implicit
declaration of function 'dgesv_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgesdd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:418: warning: implicit
declaration of function 'dgesdd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgetrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:449: warning: implicit
declaration of function 'dgetrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zungqr':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:477: warning: implicit
declaration of function 'zungqr_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dorgqr':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:508: warning: implicit
declaration of function 'dorgqr_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgeqrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:539: warning: implicit
declaration of function 'zgeqrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dgeqrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:570: warning: implicit
declaration of function 'dgeqrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_dpotrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:594: warning: implicit
declaration of function 'dpotrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgeev':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:632: warning: implicit
declaration of function 'zgeev_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgelsd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:679: warning: implicit
declaration of function 'zgelsd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgesv':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:714: warning: implicit
declaration of function 'zgesv_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgesdd':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:759: warning: implicit
declaration of function 'zgesdd_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zgetrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:794: warning: implicit
declaration of function 'zgetrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function
'lapack_lite_zpotrf':
Packages/LinearAlgebra2/Src/lapack_litemodule.c:818: warning: implicit
declaration of function 'zpotrf_'
Packages/LinearAlgebra2/Src/lapack_litemodule.c: At top level:
Packages/LinearAlgebra2/Src/lapack_litemodule.c:851: warning: function
declaration isn't a prototype
Packages/LinearAlgebra2/Src/lapack_litemodule.c:848: warning:
'lapack_liteError' defined but not used
powerpc-apple-darwin8-gcc-4.0.0: -framework: linker input file unused
because linking not done
powerpc-apple-darwin8-gcc-4.0.0: vecLib: linker input file unused
because linking not done
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Packages/RandomArray2/Src/ranlib.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Packages/RandomArray2/Src/ranlibmodule.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Packages/RandomArray2/Src/com.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Src/_dotblas.c:12:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from Src/_dotblas.c:15:
/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:665:
warning: function declaration isn't a prototype
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/high_levelmodule.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/element_wisemodule.c:3:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/one_dimensionalmodule.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/numpy_compatmodule.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/numpy_compat2.c:1:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
In file included from
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.=
h:55,
from Examples/convolve/testlite.c:26:
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:396:
warning: 'struct winsize' declared inside parameter list
/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.=
h:397:
warning: 'struct winsize' declared inside parameter list
Any ideas for a solution to this problem are welcome.
--
Chris Fonnesbeck
Atlanta, GA
|
|
From: Francesc A. <fa...@ca...> - 2005-11-02 13:07:18
|
Hi, I've recently upgraded to Numeric 24.0 and numarray 1.4.1 and I'm having a problem that did not exist before: >>> import Numeric >>> import numarray >>> Numeric.__version__ '24.0' >>> numarray.__version__ '1.4.1' >>> na=3Dnumarray.zeros((3,),'f') >>> num=3DNumeric.zeros((3,),'f') >>> na[...] =3D num Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: array sizes must be consistent. but... >>> na=3Dnumarray.zeros((3,),'i') >>> num=3DNumeric.zeros((3,),'i') >>> na[...] =3D num >>> na array([0, 0, 0]) ????? However, using Numeric 23.8 and numarray 1.3.3: >>> import Numeric >>> import numarray >>> Numeric.__version__ '23.8' >>> numarray.__version__ '1.3.3' >>> na=3Dnumarray.zeros((3,),'f') >>> num=3DNumeric.zeros((3,),'f') >>> na[...] =3D num >>> na array([ 0., 0., 0.], type=3DFloat32) I don't know if the problem is coming from Numeric or numarray, but it only happens when one mix both: >>> Numeric.__version__ '24.0' >>> numarray.__version__ '1.4.1' >>> na=3Dnumarray.zeros((3,),'f') >>> na2=3Dnumarray.zeros((3,),'f') >>> na2[...] =3D na >>> na2 array([ 0., 0., 0.], type=3DFloat32) >>> num=3DNumeric.zeros((3,),'f') >>> num2=3DNumeric.zeros((3,),'f') >>> num2[...] =3D num >>> num array([ 0., 0., 0.],'f') Perhaps the new array protocol is involved here? =2D-=20 >0,0< Francesc Altet =A0 =A0 http://www.carabos.com/ V V C=E1rabos Coop. V. =A0=A0Enjoy Data "-" |
|
From: Francesc A. <fa...@ca...> - 2005-11-02 08:37:19
|
Hi, I'm having problems in using the array protocol in numarray-->Numeric in some situations. I'm using the PyBuffer_FromReadWriteMemory C call in order to create buffers that will be later used to create numarray objects (both NumArray and CharArray) in Python space. The problem is that the array protocol implementation is doing badly this conversion. I was able to create a small example in pure python that reproduces the problem: >>> import numarray >>> from numarray import memory >>> import Numeric >>> na=3Dnumarray.array([1,2]) >>> Numeric.array(na) array([1, 2]) but ... >>> wbuf =3D memory.writeable_buffer(na) >>> nawb =3D numarray.array(wbuf,type=3D"Int32",shape=3D(2,)) >>> nawb array([1, 2]) >>> Numeric.array(nawb) array([ 2, 135510112]) I'm afraid that the problem should be related with the kind of buffer that is attached to the numarray object: >>> na._data <memory at 0x082577f8 with size:0x00000008 held by object 0x401f10a0 aliasing object 0x00000000> >>> nawb._data <read-write buffer for 0x401f10a0, size -1, offset 0 at 0x40572020> Travis, Todd, are you going to support read-write buffers or should I try to move to using the memory module in order to cope with this? For a series of reasons I'd like to keep using regular read-write buffers, but anyway... I think I'll be able to do the change if absolutely necessary. Thanks, --=20 >0,0< Francesc Altet http://www.carabos.com/ V V C=E1rabos Coop. V. Enjoy Data "-" |
|
From: Travis O. <oli...@ee...> - 2005-11-02 00:48:58
|
Chris Barker wrote: > Travis Oliphant wrote: > >> like. We could push to get something like this in Python core, I >> think, so this Array_Interface header was available to everybody. > > > That would be great. Until then, it would still be a tiny header that > others could easily include with their code. David version number > would help keep things sorted out. > I've placed an updated array interface description that includes this struct-based access on http://numeric.scipy.org Somebody will add support for this in scipy soon too. -Travis |
|
From: Chris B. <Chr...@no...> - 2005-11-02 00:03:41
|
Travis Oliphant wrote:
> like. We could push to get something like this in Python core, I think,
> so this Array_Interface header was available to everybody.
That would be great. Until then, it would still be a tiny header that
others could easily include with their code. David version number would
help keep things sorted out.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: <co...@ph...> - 2005-11-01 21:01:28
|
Travis Oliphant <oli...@ee...> writes: > David M. Cooke wrote: > >>"C >> >>I had posted an idea before: >> >>http://thread.gmane.org/gmane.comp.python.numeric.general/2159 >> >>It would still be one attribute lookup, but then everything would be >>C-based from there on. >> >> > Thanks for reminding us of your idea again. This is a very good idea, > that I think we could add. My only question is should Py_LONG_LONG be > used or Py_intptr_t? Since the array has to be in memory anyway there > does not seem any advantage to using Py_LONG_LONG. Ok, I see how that works. I probably wasn't aware of the existence of Py_intptr_t at the time :-) > I also think a flags member of the structure is useful along with a > typestr instead of typecode. The point of a typecode instead of a typestr was to make it easy for code to determine the type. Endianness would be part of the flags, and the number of bytes for the type would be in itemsize. > I would say the C-struct should look > like. We could push to get something like this in Python core, I think, > so this Array_Interface header was available to everybody. > > typedef struct { > int nd; > char typekind; > int itemsize; > int flags; > Py_intptr_t *shape; > Py_intptr_t *strides; > Py_intptr_t offset; > void *data; > } PyArray_Interface If it's in the Python core, then this is fine. If we did it ourself as an informal protocol (like the array interface spec), I'd still add the version member as a sanity check. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |co...@ph... |
|
From: Travis O. <oli...@ee...> - 2005-11-01 20:39:22
|
David M. Cooke wrote: >"C > >I had posted an idea before: > >http://thread.gmane.org/gmane.comp.python.numeric.general/2159 > >It would still be one attribute lookup, but then everything would be >C-based from there on. > > Thanks for reminding us of your idea again. This is a very good idea, that I think we could add. My only question is should Py_LONG_LONG be used or Py_intptr_t? Since the array has to be in memory anyway there does not seem any advantage to using Py_LONG_LONG. I also think a flags member of the structure is useful along with a typestr instead of typecode. I would say the C-struct should look like. We could push to get something like this in Python core, I think, so this Array_Interface header was available to everybody. typedef struct { int nd; char typekind; int itemsize; int flags; Py_intptr_t *shape; Py_intptr_t *strides; Py_intptr_t offset; void *data; } PyArray_Interface |
|
From: <co...@ph...> - 2005-11-01 19:02:07
|
"Chris Barker" <Chr...@no...> writes: > Travis Oliphant wrote: >> So, I guess the answer to your question is that for small arrays it >> is an intrinsic limitation of the use of Python attributes in the >> array protocol. > > IIRC, in the early discussion of the array protocol, we had talked about > defining a C struct, and a set of utilities to query that struct. Now, I > guess it uses Python attributes. Do I recall incorrectly, or has there > been a design change, or is this a prototype implementation? I guess I'd > like to see an array protocol that is as fast as fromstring(), even for > small arrays, though it's probably not a big deal. Also, when writing C > code to work with an array, it might be easier, as well as faster, to > not have to query Python attributes. I had posted an idea before: http://thread.gmane.org/gmane.comp.python.numeric.general/2159 It would still be one attribute lookup, but then everything would be C-based from there on. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |co...@ph... |
|
From: Travis O. <oli...@ee...> - 2005-11-01 18:53:24
|
Chris Barker wrote: > Travis Oliphant wrote: > >> So, I guess the answer to your question is that for small arrays it >> is an intrinsic limitation of the use of Python attributes in the >> array protocol. > > > IIRC, in the early discussion of the array protocol, we had talked > about defining a C struct, and a set of utilities to query that > struct. Now, I guess it uses Python attributes. Do I recall > incorrectly, or has there been a design change, or is this a prototype > implementation? I guess I'd like to see an array protocol that is as > fast as fromstring(), even for small arrays, though it's probably not > a big deal. Also, when writing C code to work with an array, it might > be easier, as well as faster, to not have to query Python attribute You are correct that it would be good to have a C-protocol that did not require attribute lookups (the source of any speed difference with fromstring). Not much progress has been made on a C-version of the protocol, though. I don't know how to do it without adding something to Python itself. At SciPy 2005, I outlined my vision for how we could proceed in that direction. There is a PEP in a subversion repository at http://svn.scipy.org/svn/PEP that explains my view. Basically, I think we should push for a simple (C-based) Nd array object that is nothing more than the current C-structure of the N-d array. Then, arrays would inherit from this base class but all of Python would be able to understand and query it's C-structure. If we could also get an array interface into the typeobject table, it would be a simple thing to populate this structure even with objects that didn't inherit from the base object. I am still interested in other ideas for how to implement the array interface in C, without adding something to the type-object table (we could push for that, but it might take more political effort). -Travis |
|
From: Francesc A. <fa...@ca...> - 2005-11-01 18:34:57
|
El dt 01 de 11 del 2005 a les 13:23 -0500, en/na Todd Miller va escriure: > Yes. I added a check in array() for suppliers of the array-interface=20 > which are not NDArrays. It only works for numerical arrays. Sounds good. I suppose that you will add the same check to asarray() in order to avoid the copy, isn't it? --=20 >0,0< Francesc Altet http://www.carabos.com/ V V C=E1rabos Coop. V. Enjoy Data "-" |
|
From: Todd M. <jm...@st...> - 2005-11-01 18:23:55
|
Francesc Altet wrote: >El dt 01 de 11 del 2005 a les 12:27 -0500, en/na Todd Miller va >escriure: > > > >>numarray's array interface for Numeric-->numarray >>was degenerating to fromlist(); I added array interface "consumption" >>support for numerical arrays by beefing up numarray's array() function. >> >> > >I'm not sure what are you saying here. You mean that in CVS there is >already code for Numeric-->numarray that does not degenerate to >fromlist() anymore? > Yes. I added a check in array() for suppliers of the array-interface which are not NDArrays. It only works for numerical arrays. Todd |
|
From: Francesc A. <fa...@ca...> - 2005-11-01 18:15:29
|
El dt 01 de 11 del 2005 a les 12:27 -0500, en/na Todd Miller va escriure: > I was working on improving this for numarray yesterday so some=20 > improvements are in CVS now. >=20 > I moved some of the Python array interface properties down to C=20 > attributes and the performance ratio for my debug Python is now <2x for=20 > numarray-->Numeric. Good! so for numarray-->Numeric we have really good performance now. > numarray's array interface for Numeric-->numarray=20 > was degenerating to fromlist(); I added array interface "consumption"=20 > support for numerical arrays by beefing up numarray's array() function. I'm not sure what are you saying here. You mean that in CVS there is already code for Numeric-->numarray that does not degenerate to fromlist() anymore? If this is the case, it's great news, of course. I'll try it as soon as I can. Thanks, --=20 >0,0< Francesc Altet http://www.carabos.com/ V V C=E1rabos Coop. V. Enjoy Data "-" |
|
From: Chris B. <Chr...@no...> - 2005-11-01 18:07:21
|
Travis Oliphant wrote:
> So, I guess the answer to your question is that for small arrays it is
> an intrinsic limitation of the use of Python attributes in the array
> protocol.
IIRC, in the early discussion of the array protocol, we had talked about
defining a C struct, and a set of utilities to query that struct. Now, I
guess it uses Python attributes. Do I recall incorrectly, or has there
been a design change, or is this a prototype implementation? I guess I'd
like to see an array protocol that is as fast as fromstring(), even for
small arrays, though it's probably not a big deal. Also, when writing C
code to work with an array, it might be easier, as well as faster, to
not have to query Python attributes.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Todd M. <jm...@st...> - 2005-11-01 17:27:30
|
Francesc Altet wrote:
>Hi,
>
>I'm trying to start using the array protocol for conversion between
>Numeric <--> numarray (and newcore in the future), but I'm a bit
>disappointed because of its performance. For numarray --> Numeric we
>have:
>
>
>
>>>>t1=timeit.Timer("num=Numeric.array(na)", "import numarray; import
>>>>
>>>>
>Numeric; na=numarray.arange(10)")
>
>
>>>>t1.repeat(3,10000)
>>>>
>>>>
>[0.59375977516174316, 0.57908082008361816, 0.56574010848999023]
>
>
>>>>t2=timeit.Timer("num=Numeric.fromstring(na._data,typecode=na.typecode())", "import numarray; import Numeric; na=numarray.arange(10)")
>>>>t2.repeat(3,10000)
>>>>
>>>>
>[0.11653494834899902, 0.1140749454498291, 0.1141819953918457]
>
>i.e. the array protocol seems 5x slower than the fromstring() method.
>
>Conversely, for Numeric --> numarray:
>
>
>
>>>>t3=timeit.Timer("na=numarray.array(num)", "import numarray; import
>>>>
>>>>
>Numeric;num=Numeric.arange(10)")
>
>
>>>>t3.repeat(3,10000)
>>>>
>>>>
>[1.3475611209869385, 1.3277668952941895, 1.3417830467224121]
>
>
>>>>t4=timeit.Timer("na=numarray.array(buffer(num),type=num.typecode(),shape=num.shape)", "import numarray; import Numeric; num=Numeric.arange(10)")
>>>>t4.repeat(3,10000)
>>>>
>>>>
>[0.42027187347412109, 0.41690587997436523, 0.41626906394958496]
>
>in this case, the array protocol is 3x slower than using the buffer
>interface.
>
>I'm wondering whether this relatively poor performance in present
>implementation of the array protocol is surmountable or is an intrinsic
>limitation of it.
>
>Thanks,
>
>
I was working on improving this for numarray yesterday so some
improvements are in CVS now.
I moved some of the Python array interface properties down to C
attributes and the performance ratio for my debug Python is now <2x for
numarray-->Numeric. numarray's array interface for Numeric-->numarray
was degenerating to fromlist(); I added array interface "consumption"
support for numerical arrays by beefing up numarray's array() function.
I tweaked your benchmarks a little to support profiling as well as
timing and attached the result.
% python arrayif_bench.py
numarray-->Numeric array_if: [0.35534501075744629, 0.36865997314453125, 0.36826896667480469]
numarray-->Numeric fromstring: [0.36841487884521484, 0.21085405349731445, 0.20747494697570801]
Numeric-->numarray array_if: [0.73384881019592285, 0.6396629810333252, 0.60234308242797852]
Numeric-->numarray buffer_if: [0.36455512046813965, 0.24601507186889648, 0.23759102821350098]
|
|
From: Travis O. <oli...@ee...> - 2005-11-01 17:22:26
|
Francesc Altet wrote: >BTW, I'm wondering whether a False value for copy should be used as the >default instead of True. IMO, many people would want to make use of the >array protocol just to access easily the data, and making a copy() >behind the scenes just for this might be potentially killer, specially >for large objects. > > Thats exactly what the asarray function is for. It is equivalent to array except its default for copy is False. -Travis |
|
From: Francesc A. <fa...@ca...> - 2005-11-01 17:19:28
|
El dt 01 de 11 del 2005 a les 09:11 -0700, en/na Travis Oliphant va
escriure:
> If you are going to be copying the data anyway, then there may be no=20
> advantage to the array protocol (in fact because it has to look up=20
> several attributes of the input object it can be slower). When you use=20
> Numeric.array(na) it makes a copy of the data by default.
>=20
> The idea is to be able to use the array protocol to not have to make=20
> copies of the data.
Yes, I don't want to do a copy. And, in fact, I want to use moderately
large array conversions (10**4 ~ 10**6 elements).
> Try using num =3D Numeric.array(na,copy=3D0) in your first timing runs =
and=20
> see what that provides.
Good! Using copy=3D0 and larger arrays (10**5 elements) I'm getting now:
>>> t1_2=3Dtimeit.Timer("num=3DNumeric.array(na,copy=3D0)", "import numarra=
y; import Numeric; na=3Dnumarray.arange(100000)")
>>> t1_2.repeat(3,1000)
[0.064317941665649414, 0.060917854309082031, 0.07666015625]
>>> t2_2=3Dtimeit.Timer("num=3DNumeric.fromstring(na._data,typecode=3Dna.ty=
pecode())", "import numarray; import Numeric; na=3Dnumarray.arange(100000)"=
)
>>> t2_2.repeat(3,1000)
[4.8582658767700195, 4.8404099941253662, 4.8652839660644531]
So, the implementation of the array protocol in the numarray --> Numeric wa=
y is
performing ashtonishingly well :-)
For the records, using the array protocol without a copy gives:
>>> t1=3Dtimeit.Timer("num=3DNumeric.array(na)", "import numarray; import N=
umeric; na=3Dnumarray.arange(100000)")
>>> t1.repeat(3,1000)
[5.014805793762207, 4.9959368705749512, 5.0420081615447998]
i.e. almost as fast a the fromstring() method, which is very good as
well!=20
BTW, I'm wondering whether a False value for copy should be used as the
default instead of True. IMO, many people would want to make use of the
array protocol just to access easily the data, and making a copy()
behind the scenes just for this might be potentially killer, specially
for large objects.
>> Conversely, for Numeric --> numarray:
> Again, you are making copies of the data. I'm not sure how numarray=20
> handles the array protocol on consumption of the interface, so I can't=20
> comment further.
Mmmm, I've tried disabling the copy, but unfortunately enough I can't
get the same figures as above:
>>> t3=3Dtimeit.Timer("na=3Dnumarray.array(num,copy=3D0)", "import numarray=
;
import Numeric; num=3DNumeric.arange(100000)")
>>> t3.repeat(3,10)
[1.6356601715087891, 1.6529910564422607, 1.6299269199371338]
>>>t4=3Dtimeit.Timer("na=3Dnumarray.array(buffer(num),type=3Dnum.typecode()=
,shape=3Dnum.shape)", "import numarray; import Numeric; num=3DNumeric.arang=
e(100000)")
>>> t4.repeat(3,1000)
[0.045578956604003906, 0.043890953063964844, 0.043296098709106445]
so, for the Numeric --> numarray way, the slowdown is more than three
orders of magnitude than expected (note the fewer iterations for the
first repeat loop). Maybe Todd can comment more on this.
Thanks!
--=20
>0,0< Francesc Altet http://www.carabos.com/
V V C=E1rabos Coop. V. Enjoy Data
"-"
|
|
From: Travis O. <oli...@ee...> - 2005-11-01 16:18:39
|
Francesc Altet wrote: >Hi, > >I'm trying to start using the array protocol for conversion between >Numeric <--> numarray (and newcore in the future), but I'm a bit >disappointed because of its performance. For numarray --> Numeric we >have: > > The other factor in this discussion is that you are creating and sharing relatively small arrays. The fromstring apporach is not a problem if all you need is a copy of the data for small arrays. The array protocol approach uses attribute lookups on an object. This is going to have overhead that will only be useful for large arrays (or arrays that you want to have share the same data region). So, I guess the answer to your question is that for small arrays it is an intrinsic limitation of the use of Python attributes in the array protocol. -Travis |
|
From: Travis O. <oli...@ee...> - 2005-11-01 16:12:15
|
Francesc Altet wrote:
>Hi,
>
>I'm trying to start using the array protocol for conversion between
>Numeric <--> numarray (and newcore in the future), but I'm a bit
>disappointed because of its performance. For numarray --> Numeric we
>have:
>
>
>
>>>>t1=timeit.Timer("num=Numeric.array(na)", "import numarray; import
>>>>
>>>>
>Numeric; na=numarray.arange(10)")
>
>
>>>>t1.repeat(3,10000)
>>>>
>>>>
>[0.59375977516174316, 0.57908082008361816, 0.56574010848999023]
>
>
>>>>t2=timeit.Timer("num=Numeric.fromstring(na._data,typecode=na.typecode())", "import numarray; import Numeric; na=numarray.arange(10)")
>>>>t2.repeat(3,10000)
>>>>
>>>>
>[0.11653494834899902, 0.1140749454498291, 0.1141819953918457]
>
>i.e. the array protocol seems 5x slower than the fromstring() method.
>
>
>
If you are going to be copying the data anyway, then there may be no
advantage to the array protocol (in fact because it has to look up
several attributes of the input object it can be slower). When you use
Numeric.array(na) it makes a copy of the data by default.
The idea is to be able to use the array protocol to not have to make
copies of the data.
Try using num = Numeric.array(na,copy=0) in your first timing runs and
see what that provides.
>Conversely, for Numeric --> numarray:
>
>
>
>>>>t3=timeit.Timer("na=numarray.array(num)", "import numarray; import
>>>>
>>>>
>Numeric;num=Numeric.arange(10)")
>
>
>>>>t3.repeat(3,10000)
>>>>
>>>>
>[1.3475611209869385, 1.3277668952941895, 1.3417830467224121]
>
>
>>>>t4=timeit.Timer("na=numarray.array(buffer(num),type=num.typecode(),shape=num.shape)", "import numarray; import Numeric; num=Numeric.arange(10)")
>>>>t4.repeat(3,10000)
>>>>
>>>>
>[0.42027187347412109, 0.41690587997436523, 0.41626906394958496]
>
>in this case, the array protocol is 3x slower than using the buffer
>interface.
>
>
Again, you are making copies of the data. I'm not sure how numarray
handles the array protocol on consumption of the interface, so I can't
comment further.
-Travis
|
258 messages has been excluded from this view by a project administrator.
