numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] Performance of the array protocol

[Francesc A. <fa...@ca...>]

Hi,

I'm trying to start using the array protocol for conversion between
Numeric <--> numarray (and newcore in the future), but I'm a bit
disappointed because of its performance. For numarray --> Numeric we
have:

>>> t1=3Dtimeit.Timer("num=3DNumeric.array(na)", "import numarray; import
Numeric; na=3Dnumarray.arange(10)")
>>> t1.repeat(3,10000)
[0.59375977516174316, 0.57908082008361816, 0.56574010848999023]
>>>t2=3Dtimeit.Timer("num=3DNumeric.fromstring(na._data,typecode=3Dna.typec=
ode())", "import numarray; import Numeric; na=3Dnumarray.arange(10)")
>>> t2.repeat(3,10000)
[0.11653494834899902, 0.1140749454498291, 0.1141819953918457]

i.e. the array protocol seems 5x slower than the fromstring() method.

Conversely, for Numeric --> numarray:

>>> t3=3Dtimeit.Timer("na=3Dnumarray.array(num)", "import numarray; import
Numeric;num=3DNumeric.arange(10)")
>>> t3.repeat(3,10000)
[1.3475611209869385, 1.3277668952941895, 1.3417830467224121]
>>>t4=3Dtimeit.Timer("na=3Dnumarray.array(buffer(num),type=3Dnum.typecode()=
,shape=3Dnum.shape)", "import numarray; import Numeric; num=3DNumeric.arang=
e(10)")
>>> t4.repeat(3,10000)
[0.42027187347412109, 0.41690587997436523, 0.41626906394958496]

in this case, the array protocol is 3x slower than using the buffer
interface.

I'm wondering whether this relatively poor performance in present
implementation of the array protocol is surmountable or is an intrinsic
limitation of it.

Thanks,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"

Re: [Numpy-discussion] byteswap questions

[Francesc A. <fa...@ca...>]

El dl 31 de 10 del 2005 a les 16:53 -0800, en/na Russell E. Owen va
escriure:
> I've got a module that can send array data to ds9. It screws up on=20
> byteswapped data and I'm trying to fix it.

I don't exactly know what a ds9 is, but I'm supposing here that it is
kind of an exotic computer.

> I need to know if the order is bigendian or littleendian, and I can't=20
> find a documented way to get that, just an undocumented attribute=20
> _byteorder.

Yes, it's undocumented, but as far as I can tell, this works flawlessly
in numarray.

> 2) If the array is not in native byte order, make a native byte ordered=20
> copy before sending it.
>=20
> The following seems to work:
> if arr.isbyteswapped():
>   arr =3D arr.copy()
>=20
> but is this the recommended way to get native-order copy?

If you can afford doing a memory copy, yes, I think this is a good way
to do it.

> - Is "copy" guaranteed to return a copy in native byte order? The=20
> documentation doesn't say.

It does:

In [23]:a._byteorder
Out[23]:'big'

In [24]:a.copy()._byteorder
Out[24]:'little'

In general, copy() will return you a well-behaved (native-byte ordered,
non-strided and non-offsetted) array.

> - I was tempted to use num.array(arr) to make the copy, but the=20
> documentation doesn't say what "array" does if the input is a already a=20
> numarray array.

This also works:

In [25]:array(a)._byteorder
Out[25]:'little'

and it is very similar to copy() in performance:

In [34]:t1=3DTimer("b=3Da.copy()","import numarray;
a=3Dnumarray.arange(10);a.byteswap();a._byteorder=3D'big'")

In [35]:t1.repeat(3,10000)
Out[35]:[1.0613389015197754, 1.0313379764556885, 1.0303771495819092]

In [36]:t2=3DTimer("b=3Dnumarray.array(a)","import numarray;
a=3Dnumarray.arange(10);a.byteswap();a._byteorder=3D'big'")

In [37]:t2.repeat(3,10000)
Out[37]:[1.2250277996063232, 1.2067301273345947, 1.2336349487304688]

(perhaps copy is just a little bit faster)

> As an aside, I tried to use the byteswapped method, but it has a totally=20
> different effect:
>=20
> >>> d
> array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.],
> ...type=3DFloat32)
> >>> d.isbyteswapped()
> 1
> >>> dcopy =3D d.copy()
> >>> d
> array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.],
>  ...type=3DFloat32)
> >>> dcopy.isbyteswapped()
> 0
> >>> dswap =3D d.byteswapped()
> >>> dswap
> array([[  1.91063654e-38,   1.91063654e-38,   1.91063654e-38, ...,
>   ...type=3DFloat32)
>=20
> which I guess means I'd have to byteswap the byteswapped data. Aargh.

This is the intended behaviour. From the docstrings:

In [59]:a.byteswap?
[snip...]
Docstring:
    Byteswap data in place, leaving the _byteorder attribute untouched.


So, you already have done a byteswapping, but you still need to tell the
object that its byteorder has actually changed (if don't, numarray will
convert the data to you local native order before printing it on the
flight). Just issue a:

dswap._byteorder =3D {"little":"big","big":"little"}[dswap._byteorder]

and you are done. In fact, using byteswap() does the conversion
in-place, and doesn't need a memory copy. This is actually faster than
doing a copy() much of the times:

In [60]:t3=3DTimer("b=3Da.byteswap();a._byteorder=3D'little'","import
numarray; a=3Dnumarray.arange(10);a.byteswap();a._byteorder=3D'big'")

In [61]:t3.repeat(3,10000)
Out[61]:[0.23770284652709961, 0.23195695877075195, 0.23380589485168457]


Cheers,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"

[Numpy-discussion] byteswap questions

[Russell E. O. <ro...@ce...>]

I've got a module that can send array data to ds9. It screws up on 
byteswapped data and I'm trying to fix it.

The code sends the data to ds9 via a pipe using arr.tofile(). To send 
byteswapped data I think I have two choices (given that I don't want to 
modify the input array):

1) Figure out the byte order and specify a suitable flag to ds9 (it 
accepts "bigendian" or "littleendian" flags).

This avoids copying the data, so it sounds attractive, but can I do it 
without relying on "internals"? Numarrays have an "isbyteswapped" 
method, but all this says is "not native machine order"; it doesn't 
actually tell me the order.

I need to know if the order is bigendian or littleendian, and I can't 
find a documented way to get that, just an undocumented attribute 
_byteorder.


2) If the array is not in native byte order, make a native byte ordered 
copy before sending it.

The following seems to work:
if arr.isbyteswapped():
  arr = arr.copy()

but is this the recommended way to get native-order copy?

- Is "copy" guaranteed to return a copy in native byte order? The 
documentation doesn't say.
- I was tempted to use num.array(arr) to make the copy, but the 
documentation doesn't say what "array" does if the input is a already a 
numarray array.

As an aside, I tried to use the byteswapped method, but it has a totally 
different effect:

>>> d
array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.],
...type=Float32)
>>> d.isbyteswapped()
1
>>> dcopy = d.copy()
>>> d
array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.],
 ...type=Float32)
>>> dcopy.isbyteswapped()
0
>>> dswap = d.byteswapped()
>>> dswap
array([[  1.91063654e-38,   1.91063654e-38,   1.91063654e-38, ...,
  ...type=Float32)

which I guess means I'd have to byteswap the byteswapped data. Aargh.

-- Russell

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Francesc A. <fa...@ca...>]

El dv 28 de 10 del 2005 a les 14:52 -0600, en/na Travis Oliphant va
escriure:
> > Todd Miller wrote:
> > I don't know if it does right now.   Just to be clear,  what Francesc=20
> > was talking about is *supplying* the array buffer from numarray as=20
> > apparently defined in the __get_array_data__ method.  If 24b was the=20
> > first time the array interface ever worked it sounds ok to leave your=20
> > changes in and force an upgrade to 24 if a user uses that feature. =20
> > Francesc?
> Right, he was talking about just supplying the array data information=20
> from numarray.
>=20
> And, yes, Numeric 24b was the first time the array interface ever worked.

Yes, I think that if Numeric 24b was the first implementing the array
interface, then there is no need to touch nothing in numarray. As you
said, I should make Numeric 24.0 a requisite for running our software.

Thanks for clarificating the issue,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"

Re: [Numpy-discussion] numarray 1.4.1: Infinite loop during test

[Todd M. <jm...@st...>]

Edward C. Jones wrote:

> I have a PC with Debian unstable. I use Python 1.4.2. After installing 
> numarray 1.4.1, I ran the tests:
>
> >>> import numarray.testall as testall
> >>> testall.test()
> Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on 
> platform linux2
> numarray.numtest:                        2.89      ((0, 1205), (0, 1205))
> numarray.ieeespecial:                    0.07      (0, 86)
> numarray.records:                        0.05      (0, 48)
> numarray.strings:                        0.17      (0, 189)
> numarray.memmap:                         0.09      (0, 82)
> numarray.objects:                        0.14      (0, 105)
> numarray.memorytest:                     0.01      (0, 16)
> numarray.examples.convolve:              0.09      ((0, 20), (0, 20), 
> (0, 20), (0, 20))
> numarray.convolve:                       0.06      (0, 45)
> numarray.fft:                            0.11      (0, 75)
>
> After the above, nothing happens. "ps ax" show a lot of time being 
> used. Looks like an infinite loop. Process had to be killed.

I don't have access to a Debian system but some things come to mind:

a. The loop looks like it's in numarray.linear_algebra.

b. I changed numarray's setup to make the "config" command automatic for 
1.4.0.  That changes how the builtin lapack_lite module is 
configured/built which affects linear_algebra.

c. My impression is that debian builds numarray against external 
libraries so (b) shouldn't be the problem. 

Does the numarray tarball from source forge:

        
http://prdownloads.sourceforge.net/numpy/numarray-1.4.1.tar.gz?download

install and self-test OK using "python setup.py install"?

Regards,
Todd

[Numpy-discussion] numarray 1.4.1: Infinite loop during test

[Edward C. J. <edc...@co...>]

I have a PC with Debian unstable. I use Python 1.4.2. After installing 
numarray 1.4.1, I ran the tests:

 >>> import numarray.testall as testall
 >>> testall.test()
Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on 
platform linux2
numarray.numtest:                        2.89      ((0, 1205), (0, 1205))
numarray.ieeespecial:                    0.07      (0, 86)
numarray.records:                        0.05      (0, 48)
numarray.strings:                        0.17      (0, 189)
numarray.memmap:                         0.09      (0, 82)
numarray.objects:                        0.14      (0, 105)
numarray.memorytest:                     0.01      (0, 16)
numarray.examples.convolve:              0.09      ((0, 20), (0, 20), 
(0, 20), (0, 20))
numarray.convolve:                       0.06      (0, 45)
numarray.fft:                            0.11      (0, 75)

After the above, nothing happens. "ps ax" show a lot of time being used. 
Looks like an infinite loop. Process had to be killed.

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Travis O. <oli...@ee...>]

> Todd Miller wrote:

> I don't know if it does right now.   Just to be clear,  what Francesc 
> was talking about is *supplying* the array buffer from numarray as 
> apparently defined in the __get_array_data__ method.  If 24b was the 
> first time the array interface ever worked it sounds ok to leave your 
> changes in and force an upgrade to 24 if a user uses that feature.  
> Francesc?


Right, he was talking about just supplying the array data information 
from numarray.

And, yes, Numeric 24b was the first time the array interface ever worked.

-Travis

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Todd M. <jm...@st...>]

Travis Oliphant wrote:

> Todd Miller wrote:
>
>> Francesc Altet wrote:
>>
>>> Hi Todd,
>>>
>>> I've just installed numarray 1.4.1 and pass my tests over it.
>>> Everything goes well, except some small detail:
>>>
>>> Python 2.4.2 (#2, Sep 29 2005, 00:23:59)
>>> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>  
>>>
>>>>>> import Numeric
>>>>>> import numarray
>>>>>> Numeric.__version__
>>>>>>       
>>>>>
>>>>>
>>> '24.0b2'
>>>  
>>>
>>>>>> numarray.__version__
>>>>>>       
>>>>>
>>>>>
>>> '1.4.1'
>>>  
>>>
>>>>>> na=numarray.array([ 51.], type=numarray.Float64)
>>>>>> Numeric.array(na, typecode='d')
>>>>>>       
>>>>>
>>>>>
>>> Traceback (most recent call last):
>>>  File "<stdin>", line 1, in ?
>>> TypeError: expected a writeable buffer object
>>>  
>>>
>
>> This looks like a coordinated change in both numarray and Numeric.   
>> (Travis?)  I updated to Numeric-24.0 and all was well.
>
>
>
>> It appears the __array_data__ protocol improved to know about 
>> readonly-ness.  In order to support the readonly behavior in 
>> numarray,  I think the change must be made in lockstep.  We could 
>> however back out support for readonly while Numeric-24.0 becomes more 
>> pervasive.
>
>
> Yes, basically the __array_data__ protocol was essentially pointless 
> before because it just returned a buffer object which
>
> 1) did nothing more than the object itself supporting the buffer 
> protocol and
> 2) could not handle strided (discontiguous) arrays.
>
> The new array protocol handles the situation much better and allows 
> discontiguous arrays to be shared.
>
> However, it does create some backward compatibility issues.  But, 
> because Numeric 24.0b2 was a *beta* release I don't see this as a real 
> problem.

Yeah, that sounds ok...  I was thinking the array interface was a 
release or two older.

>   Get the final release of Numeric, which will handle the array 
> protocol correctly (note it will also handle the old __array_data__ 
> format as well).
>
> Todd,  I was not sure how to change numarray so it would consume the 
> new protocol correctly. 

I think numarray's multiple factory functions in numarray, strings, and 
records all need to be beefed up to look for it.

> So, I'm not sure how numarray interprets the __array_data__ attribute.

I don't know if it does right now.   Just to be clear,  what Francesc 
was talking about is *supplying* the array buffer from numarray as 
apparently defined in the __get_array_data__ method.  If 24b was the 
first time the array interface ever worked it sounds ok to leave your 
changes in and force an upgrade to 24 if a user uses that feature.  
Francesc?

Todd

[Numpy-discussion] numarray 1.4.0: unitary operator "+" fails

[Edward C. J. <edc...@co...>]

The unitary "+" does not work for numarray 1.4.0:

 > python
Python 2.4.2 (#1, Oct  1 2005, 18:47:35)
[GCC 4.0.2 20050821 (prerelease) (Debian 4.0.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import numarray
 >>> arr = numarray.zeros((2,2))
 >>> +arr
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
AttributeError: copy

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Travis O. <oli...@ee...>]

Todd Miller wrote:

> Francesc Altet wrote:
>
>> Hi Todd,
>>
>> I've just installed numarray 1.4.1 and pass my tests over it.
>> Everything goes well, except some small detail:
>>
>> Python 2.4.2 (#2, Sep 29 2005, 00:23:59)
>> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>  
>>
>>>>> import Numeric
>>>>> import numarray
>>>>> Numeric.__version__
>>>>>       
>>>>
>> '24.0b2'
>>  
>>
>>>>> numarray.__version__
>>>>>       
>>>>
>> '1.4.1'
>>  
>>
>>>>> na=numarray.array([ 51.], type=numarray.Float64)
>>>>> Numeric.array(na, typecode='d')
>>>>>       
>>>>
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in ?
>> TypeError: expected a writeable buffer object
>>  
>>

> This looks like a coordinated change in both numarray and Numeric.   
> (Travis?)  I updated to Numeric-24.0 and all was well.


> It appears the __array_data__ protocol improved to know about 
> readonly-ness.  In order to support the readonly behavior in 
> numarray,  I think the change must be made in lockstep.  We could 
> however back out support for readonly while Numeric-24.0 becomes more 
> pervasive.

Yes, basically the __array_data__ protocol was essentially pointless 
before because it just returned a buffer object which

1) did nothing more than the object itself supporting the buffer 
protocol and
2) could not handle strided (discontiguous) arrays.

The new array protocol handles the situation much better and allows 
discontiguous arrays to be shared.

However, it does create some backward compatibility issues.  But, 
because Numeric 24.0b2 was a *beta* release I don't see this as a real 
problem.   Get the final release of Numeric, which will handle the array 
protocol correctly (note it will also handle the old __array_data__ 
format as well).

Todd,  I was not sure how to change numarray so it would consume the new 
protocol correctly.   So, I'm not sure how numarray interprets the 
__array_data__ attribute.

-Travis

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Travis O. <oli...@ee...>]

Francesc Altet wrote:

>A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure:
>  
>
>>This has been a problem for a remarkably long time! I thought it had
>>been fixed. It's not really a big deal, it's only a matter of what the
>>defaults are in customize.py. The comments in there should be enough to
>>figure it out.
>>    
>>
>
>Yup, it worked. Thanks.
>
>Anyway, I think that the sensible default would be to not suppose that
>neither BLAS, LAPACK, ATALS are installed and update the README to
>give instructuctions on how to adapt Numeric to your system.
>
>  
>
Yes, that was always the plan.  I just forgot to reset customize.py to 
the original before making a tar ball.   I've fixed the problem now.  
The new tar ball does uses default lapack_lite. 

(I'm sure I'll upset somebody because the tar ball changed slightly with 
no version number increase --- but I don't want to make a new release 
--- the other files should be fine).

-Travis

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Francesc A. <fa...@ca...>]

A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure:
> This has been a problem for a remarkably long time! I thought it had
> been fixed. It's not really a big deal, it's only a matter of what the
> defaults are in customize.py. The comments in there should be enough to
> figure it out.

Yup, it worked. Thanks.

Anyway, I think that the sensible default would be to not suppose that
neither BLAS, LAPACK, ATALS are installed and update the README to
give instructuctions on how to adapt Numeric to your system.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Chris B. <Chr...@no...>]

Francesc Altet wrote:
> /usr/bin/ld: cannot find -llapack
> collect2: ld returned 1 exit status
> error: command 'gcc' failed with exit status 1
> 
> The installation for Numeric usually worked on my system. Perhaps it
> is now assumed that lapack should be a pre-requisite?

This has been a problem for a remarkably long time! I thought it had 
been fixed. It's not really a big deal, it's only a matter of what the 
defaults are in customize.py. The comments in there should be enough to 
figure it out.

The default really should be to use the built-in lapack-lite.

-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Francesc A. <fa...@ca...>]

A Divendres 28 Octubre 2005 19:58, Todd Miller va escriure:
> This looks like a coordinated change in both numarray and Numeric.
> (Travis?)  I updated to Numeric-24.0 and all was well.

Ops, I was not aware that Numeric-24.0 was out. Anyway, I've tried to
compile it, but I did not succeed with my Debian system:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes=
=20
=2DfPIC -IInclude -IPackages/FFT/Include -IPackages/RNG/Include=20
=2DI/usr/include/python2.4 -c Src/lapack_litemodule.c -o=20
build/temp.linux-i686-2.4/Src/lapack_litemodule.o
gcc -pthread -shared build/temp.linux-i686-2.4/Src/lapack_litemodule.o=20
=2DL/usr/lib/atlas -llapack -lcblas -lf77blas -latlas -lg2c -o=20
build/lib.linux-i686-2.4/lapack_lite.so
/usr/bin/ld: cannot find -llapack
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

The installation for Numeric usually worked on my system. Perhaps it
is now assumed that lapack should be a pre-requisite?

> It appears the __array_data__ protocol improved to know about
> readonly-ness.  In order to support the readonly behavior in numarray,
> I think the change must be made in lockstep.  We could however back out
> support for readonly while Numeric-24.0 becomes more pervasive.

Yes, please, backward compatibility with pre-Numeric 24.0 would be a
nice thing to have.

Regards,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Todd M. <jm...@st...>]

Francesc Altet wrote:

>Hi Todd,
>
>I've just installed numarray 1.4.1 and pass my tests over it.
>Everything goes well, except some small detail:
>
>Python 2.4.2 (#2, Sep 29 2005, 00:23:59)
>[GCC 4.0.2 (Debian 4.0.1-9)] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
>  
>
>>>>import Numeric
>>>>import numarray
>>>>Numeric.__version__
>>>>        
>>>>
>'24.0b2'
>  
>
>>>>numarray.__version__
>>>>        
>>>>
>'1.4.1'
>  
>
>>>>na=numarray.array([ 51.], type=numarray.Float64)
>>>>Numeric.array(na, typecode='d')
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
>TypeError: expected a writeable buffer object
>  
>
>
>This worked in 1.3.x and before.
>
>I hope that this is not the intended behaviour.
>  
>
This looks like a coordinated change in both numarray and Numeric.   
(Travis?)  I updated to Numeric-24.0 and all was well. 

It appears the __array_data__ protocol improved to know about 
readonly-ness.  In order to support the readonly behavior in numarray,  
I think the change must be made in lockstep.  We could however back out 
support for readonly while Numeric-24.0 becomes more pervasive.

Regards,
Todd

Re: [Numpy-discussion] ANN: numarray-1.4.1

[Francesc A. <fa...@ca...>]

Hi Todd,

I've just installed numarray 1.4.1 and pass my tests over it.
Everything goes well, except some small detail:

Python 2.4.2 (#2, Sep 29 2005, 00:23:59)
[GCC 4.0.2 (Debian 4.0.1-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Numeric
>>> import numarray
>>> Numeric.__version__
'24.0b2'
>>> numarray.__version__
'1.4.1'
>>> na=3Dnumarray.array([ 51.], type=3Dnumarray.Float64)
>>> Numeric.array(na, typecode=3D'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: expected a writeable buffer object
>>>

This worked in 1.3.x and before.

I hope that this is not the intended behaviour.

Thanks,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

[Numpy-discussion] ANN: numarray-1.4.1

[Todd M. <jm...@st...>]

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modelled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, arrays of
heterogeneous records, string arrays, and in-place operation on
memory mapped files.

=========================================================================

Release Notes for numarray-1.4.1

II. BUGS FIXED / CLOSED

1339713        segfault when comparing string-array to self

=========================================================================

Release Notes for numarray-1.4.0

I. ENHANCEMENTS

1. Speed improvement for numarray operators.  The Python level hook
mapping numarray operators onto universal functions has been moved
down to C.

2. Speed improvement for string-array comparisons, any(),
all(). String correlation is ~10x faster.

3. Better operation with py2exe to help it automatically detect the
core numarray extensions to include in an installer.

4. scipy newcore compatible lower case type names (e.g. int32 not Int32)

5. scipy newcore 'dtype' keyword and .dtypechar attribute.

II. BUGS FIXED / CLOSED

1323355      Apps fail with import_libnumarray
1315212     Infinite loop converting some scalar strings into a list
1298916     rank-0 tostring() broken
1297948     records.array fails to create empty fields
1286291     import sys missing from array_persist.py
1286168     Generic sequences in ``strings.array()``
1236392     Outdated web link in announcements
1235219     LinearAlgebraError not imported in linear_algebra

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

III. CAUTIONS

This release should be backward binary compatible with numarray-1.3.x


WHERE
-----------

Numarray-1.4.0 windows executable installers, source code, and manual is
here:

http://sourceforge.net/project/showfiles.php?group_id=1369

Numarray is hosted by Source Forge in the same project which hosts
Numeric:

http://sourceforge.net/projects/numpy/

The web page for Numarray information is at:

http://www.stsci.edu/resources/software_hardware/numarray

Trackers for Numarray Bugs, Feature Requests, Support, and Patches are
at the Source Forge project for NumPy at:

http://sourceforge.net/tracker/?group_id=1369

REQUIREMENTS
------------------------------

numarray-1.4.0 requires Python 2.2.2 or greater.  


AUTHORS, LICENSE
------------------------------

Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC
Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science
Institute.  We'd like to acknowledge the assitance of Francesc Alted,
Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav
Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant,
Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and
everyone else who has contributed with comments and feedback.

Numarray is made available under a BSD-style License.  See
LICENSE.txt in the source distribution for details.

-- 
Todd Miller             jm...@st...

Re: [Numpy-discussion] Re: ANN: numarray-1.4.0

[Todd M. <jm...@st...>]

Andrew Jaffe wrote:

> Hi-
>
>> 4. scipy newcore compatible lower case type names (e.g. int32 not Int32)
>
>
> Is this actually true? For example, numarray.Float64 exists as before, 
> but numarray.float64 does not.

I added the "string aliases" but didn't create new Python object 
bindings.  So you can currently say something like

    arange(10, dtype='float32')

but cannot yet say:

    arange(10, dtype=float32)

Good catch.   I'll add those soon.

>> 5. scipy newcore 'dtype' keyword and .dtypechar attribute.
>
>
> And more generally, is there any more information on the relationship 
> between numarray and scipy_core? 

I think scipy_core is intended to add the best numarray-like features to 
Numeric.

> Will numarray be phased out eventually (i.e., should we start moving 
> over to scipy?)?

When scipy_core matures enough to replace numarray,  numarray will be 
phased out.    If you rely on numarray,  it is definitely worth your 
time to keep an eye on scipy_core and switch when it meets your needs 
better.

Regards,
Todd

[Numpy-discussion] Re: ANN: numarray-1.4.0

[Andrew J. <a.h...@gm...>]

Hi-

> 4. scipy newcore compatible lower case type names (e.g. int32 not Int32)

Is this actually true? For example, numarray.Float64 exists as before, 
but numarray.float64 does not.

> 5. scipy newcore 'dtype' keyword and .dtypechar attribute.

And more generally, is there any more information on the relationship 
between numarray and scipy_core? Will numarray be phased out eventually 
(i.e., should we start moving over to scipy?)?

But in any event thanks for the hard work!

Andrew

Re: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem

[Todd M. <jm...@st...>]

Todd Miller wrote:

> John A Chaves wrote:
>
>> After installing numarray-1.4.0, I get a Segmentation fault when 
>> comparing
>> two instances of the same string-array.  For example:
>>
>>     from numarray.strings import asarray
>>
>>     for x in range(1000,1000000,1000):
>>       print x
>>       raw = ['abcde']*x
>>       arr = asarray(raw)
>>       arr == arr
>>
>> gives:
>>
>>     1000
>>     2000
>>     3000
>>     4000
>>     5000
>>     6000
>>     Segmentation fault
>>  
>>
>> Can anyone else reproduce this, or is a problem with my environment?
>>  
>>
> Yes,  I was able to reproduce this too,  thanks for the report and for 
> logging on SF.    I understand the problem and am working out a fix now.
>
> Regards,
> Todd

I committed the fix for this and put out numarray-1.4.1.

Thanks again,
Todd

Re: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem

[Todd M. <jm...@st...>]

John A Chaves wrote:

>After installing numarray-1.4.0, I get a Segmentation fault when comparing
>two instances of the same string-array.  For example:
>
>	from numarray.strings import asarray
>
>	for x in range(1000,1000000,1000):
>	  print x
>	  raw = ['abcde']*x
>	  arr = asarray(raw)
>	  arr == arr
>
>gives:
>
>	1000
>	2000
>	3000
>	4000
>	5000
>	6000
>	Segmentation fault
>  
>
>Can anyone else reproduce this, or is a problem with my environment?
>  
>
Yes,  I was able to reproduce this too,  thanks for the report and for 
logging on SF.    I understand the problem and am working out a fix now.

Regards,
Todd

[Numpy-discussion] numarray-1.4.0 string-array comparisons problem

[John A C. <ch...@co...>]

After installing numarray-1.4.0, I get a Segmentation fault when comparing
two instances of the same string-array.  For example:

	from numarray.strings import asarray

	for x in range(1000,1000000,1000):
	  print x
	  raw = ['abcde']*x
	  arr = asarray(raw)
	  arr == arr

gives:

	1000
	2000
	3000
	4000
	5000
	6000
	Segmentation fault

I verified that this works correctly in numarray-1.3.3.  Also, the problem
can be avoided in 1.4.0 by replacing 'arr == arr' with 'arr = arr.copy()'.

Can anyone else reproduce this, or is a problem with my environment?

Thanks,
John

[Numpy-discussion] ANN: numarray-1.4.0

[Todd M. <jm...@st...>]

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modeled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, arrays of
heterogeneous records, string arrays, and in-place operation on
memory mapped files.

=========================================================================

Release Notes for numarray-1.4.0

I. ENHANCEMENTS

1. Speed improvement for numarray operators.  The Python level hook
mapping numarray operators onto universal functions has been moved
down to C.

2. Speed improvement for string-array comparisons, any(),
all(). String correlation is ~10x faster.

3. Better operation with py2exe to help it automatically detect the
core numarray extensions to include in an installer.

4. scipy newcore compatible lower case type names (e.g. int32 not Int32)

5. scipy newcore 'dtype' keyword and .dtypechar attribute.

II. BUGS FIXED / CLOSED

1323355         Apps fail with import_libnumarray
1315212         Infinite loop converting some scalar strings into a list
1298916         rank-0 tostring() broken
1297948         records.array fails to create empty fields
1286291         import sys missing from array_persist.py
1286168         Generic sequences in ``strings.array()``
1236392         Outdated web link in announcements
1235219         LinearAlgebraError not imported in linear_algebra

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

III. CAUTIONS

This release should be backward binary compatible with numarray-1.3.x


WHERE
-----------

Numarray-1.4.0 windows executable installers, source code, and manual is
here:

http://sourceforge.net/project/showfiles.php?group_id=1369

Numarray is hosted by Source Forge in the same project which hosts
Numeric:

http://sourceforge.net/projects/numpy/

The web page for Numarray information is at:

http://www.stsci.edu/resources/software_hardware/numarray

Trackers for Numarray Bugs, Feature Requests, Support, and Patches are
at the Source Forge project for NumPy at:

http://sourceforge.net/tracker/?group_id=1369

REQUIREMENTS
------------------------------

numarray-1.4.0 requires Python 2.2.2 or greater.


AUTHORS, LICENSE
------------------------------

Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC
Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science
Institute.  We'd like to acknowledge the assitance of Francesc Alted,
Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav
Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant,
Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and
everyone else who has contributed with comments and feedback.

Numarray is made available under a BSD-style License.  See
LICENSE.txt in the source distribution for details.

--
Todd Miller             jm...@st...

[Numpy-discussion] Re: Numeric: Change of v24.0

[Travis O. <oli...@ee...>]

Weissmann Markus wrote:

> Hi folks,
>
> Before beginning my rant: Thank you very much for Numeric - you're  
> doing a great job!
> <rant>
> I'm a bit pissed as you changed the - already released tarball of  
> version 24.0;
> This is a real mess, as all build systems out there have problems  
> distinguishing versions,
> the correct tarballs (hashes will fail) etc.

I'm not sure what you are talking about.

The old tarball was always a beta release.    It took a long time to get 
the actual release out, but that is due to the lack of reports. 

> I don't think this is my personal, or my projects impression, as e.  
> g. Novell [1] & O'Reilly [2]
> clearly were under the impression that the old tarball was v24.0.
> </rant>

Then they were mistaken.  It was always clearly marked as version 24b2 
(beta-release 2).

> Whatever happened that lead to this confusion, it would be great if  
> you could at least
> make a version bump to 24.1 right now so everyone can be sure that  
> 24.1 is 24.1 in contrast
> to 24.0 not being 24.0 all the time.
>
Someone could make a 24.1, but I don't see the need, and I'm not really 
interested in doing that.  I'm done with Numeric development as I've 
moved to scipy core.

I'm sorry for your consternation.

-Travis

[Numpy-discussion] numarray.objects

[Joris v. Z. <van...@sc...>]

Hi,

Are arrays of objects (i.e. numarray.objects) still supported in
Numeric3/Scipy? (I'm working on an application that should be able to work
with arrays of numbers and arrays of objects. Without
numarray.objects-like functionality, this would require using numarray
arrays for the numbers, and nested python lists for the objects. ;)
Thanks,

Joris van Zwieten