numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Christopher B. <Chr...@no...>]

Bruce Southey wrote:
> I am also +1 to keep numpy for the same reasons as Colin and Chris. So 
> far there has been nothing in alpha SciPy versions that offer any 
> advantage over Numarray for what I use or develop.

Just so it's clear: I was advocating that the numpy-discussion group be 
the primary place for NumPy-related ( as opposed to other scipy stuff) 
discussion. I was not advocating for or against the adaptation of the 
new scipy_base - scipy_core - now-numpy.

As it happens, I am very excited by the new numpy, am really looking 
forward to its rapid adoption.

-Chris

> On 1/5/06, *Christopher Barker* <Chr...@no... 

>     +1 numpy-discussion



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] Re: [SciPy-user] numpy lists

[Andrew S. <str...@as...>]

Travis Oliphant wrote:

>There are two new mailing lists for NumPy
>
>num...@li...
>num...@li...
>
>These are for developers and users to talk about only NumPy
>  
>
You can subscribe to these lists from 
http://sourceforge.net/mail/?group_id=1369

[Numpy-discussion] numpy lists

[Travis O. <oli...@ie...>]

Bruce Southey wrote:

> I am also +1 to keep numpy for the same reasons as Colin and Chris. So 
> far there has been nothing in alpha SciPy versions that offer any 
> advantage over Numarray for what I use or develop.


There are two new mailing lists for NumPy

num...@li...
num...@li...

These are for developers and users to talk about only NumPy

The SciPy lists can be for SciPy itself.

Two packages deserve separate lists.

-Travis

[Numpy-discussion] Installation of numpy header files

[Alexander B. <ale...@gm...>]

Numpy include files get installed in <python prefix>/lib/pythonX.X/site-
packages/numpy/base/include/numpy instead of <python prefix>/include/
pythonX.X/numpy.

The attached patch fixes the problem.  Does anyone rely on the current beha=
vior?

-- sasha

PS: Using scipy core svn r1847 on a Linux system.

Re: [Numpy-discussion] Properties of ma.masked singleton

[Alexander B. <ale...@gm...>]

On 1/6/06, Paul F. Dubois <pa...@pf...> wrote:
> ...
> I redid ma in a bit of a hurry and didn't pay any attention to the
> issues you raise as to len(masked), which I agree should be 1.

I did not suggest that len(masked) should be 1. It was 1 in Numeric MA
because scallars had len 1 in Numeric, but in numpy len(scalar) is an
error consistent with scalar.shape=3D=3D() and len(x) =3D=3D x.shape[0]. My
proposal was not to return masked singleton from x[i] and return
ma.array(x.data[i],mask=3Dx.mask[i]), this will result in len(x[i])
being an error for rank 1 x and integer i, which would be consistent
with numpy.

-- sasha

Re: [Numpy-discussion] Properties of ma.masked singleton

[Alexander B. <ale...@gm...>]

Yes, you are right. I did not realize that x[i] =3D None is legitimate
for dtype=3Dobject arrays. This can easily be fixed by using an instance
of an empty class instead of None.

My main concern was that since now x[i] provides full array inteface
in numpy regardless of the value of i, using a singleton value for the
same construct in ma leads to all kinds of surprizing results. For
example:

>>> ma.array([1.0],mask=3D[1])[0].dtype
<type 'int32_arrtype'>
>>> ma.array([1.0],mask=3D[0])[0].dtype
<type 'float64_arrtype'>

While I appreciate the utility of "x[i] =3D masked", particularly for
fancy i, I believe "x[i] is masked" is of limited usefulness.

Note that the first construct does not require a singleton - any
masked scalar can be used instead of the singleton.  The second
construct on the other hand would not work without a singleton.

I would disagree that "x[2:4] is masked is a red herring".  Consider
the following session:

>>> i =3D slice(2,4)
>>> x =3D ma.arange(5)
>>> x[i] =3D ma.masked
>>> x[i] is ma.masked
False

Note that the result would be true if i was an integer.

It is not obvious that "x[2:4] is masked" should be false is all
elements in 2:4 range are masked. Numpy deprecated conversion of size
> 1 arrays to bool in favor of explicit any or all:

>>> bool(array([1,1]))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()

In my view "x[i] is masked" is ambiguous as well and x.mask.any() or
x.mask.all() seems to be a better choice (this would require use of
bool_(0) instead of None
to indicate no mask).

How much code out there relies on "x[i] is masked" construct? I've
used MA for many years (thanks for an excellent product, Paul!) but
never used this construct in my code.

Thanks.

-- sasha

On 1/6/06, Paul F. Dubois <pa...@pf...> wrote:
> I'll look into your complaints. Your proposed solution does not work:
> if x[2] is masked:
> and
> if x[2] is None:
> become indistinguishable. Likewise for assignment.
>
> Your example if x[2:4] is masked is a red herring. Obviously this
> statement is false no matter how you define 'masked'.
>
> I redid ma in a bit of a hurry and didn't pay any attention to the
> issues you raise as to len(masked), which I agree should be 1.
>
> Alexander Belopolsky wrote:
> > In the current numpy version ma.masked singleton has the following
> > properties:
> >
> >  >>> len(ma.masked)
> > 0
> >
> > which is a change from old MA, where
> >  >>> len(MA.masked)
> > 1
> >
> > Similarly:
> >
> >  >>> ma.masked.shape
> > (0,)
> >  >>> MA.masked.shape
> > ()
> >
> > It changes shape when filled:
> >
> >  >>> ma.masked.filled()
> > array([999999])
> >
> > and so on.
> >
> > The code contains numerous "if x is masked" statements to support all
> > this logic.
> >
> > I would like to suggest a somewhat radical change for your review.
> >
> > There are two main uses of ma.masked:
> >
> > 1. To set mask:
> >
> >  >>> x =3D ma.zeros(5)
> >  >>> x[2:4] =3D ma.masked
> >  >>> print x
> > [0 0 -- -- 0]
> >
> >
> > 2. To check if an element is masked:
> >
> >  >>> x[2] is ma.masked
> > True
> >
> >
> > The second property allows a cute looking idiom "x[i] is masked", but
> > only works for single element access:
> >
> >  >>> x[2:4] is ma.masked
> > False
> >
> > It also leads to strange results:
> >
> >  >>> x[2].shape
> > (0,)
> >
> >
> > My proposal is to eliminate the property #2 from ma and redefine  maske=
d
> > as None. Single element getitem will return a rank zero  MaskedArray. W=
e
> > can also add "is_masked" property, which will allow a  convenient check
> > in the form "x[i].is_masked".
> >
> > The main benefit of this proposal is that x[i] will be duck typing
> > compatible with numpy scalars.
> >
> > -- sasha
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> > files
> > for problems?  Stop!  Download the new AJAX search engine that makes
> > searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> > http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick
> > _______________________________________________
> > Numpy-discussion mailing list
> > Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
>

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Sebastian H. <ha...@ms...>]

On Friday 06 January 2006 09:34, Perry Greenfield wrote:
> Perhaps all discussions of the core stuff could go to numpy-discussion
> and the applications and libraries go to scipy-discussion. That
> segregation often doesn't work and you still get all the cross-posting
> anyway. But I would think that it would be important to have the core
> element highlighted in the name since that user base will be bigger
> than the scipy one by necessity.
>
> Perry
>
Reading this I would second the idea of keeping two lists: if and only if the 
total number of people exceeds a certain limit (which I think was the 
original statement that that is not the case -- is there some statistics of 
how many (active) people are subscribed to each list)

Also I would like to point out that the way Perry wrote his posting
the "scipy"(!) list should soon be the larger one, because "applications and 
libraries" is the reason we all do this in the first place and the underlying 
"numpy" would "just be working" - at least as most people will 
concern/presume.

- Sebastian Haase


> On Jan 5, 2006, at 5:03 PM, Francesc Altet wrote:
> > A Dijous 05 Gener 2006 20:59, Christopher Barker va escriure:
> >> Colin J. Williams wrote:
> >>> It seems to me that NumPy is the better way to go, the archives and
> >>> downloads are more readily available there, but it's up to Todd
> >>> Miller
> >>> and the folk who have been maintaining NumPy.
> >>>
> >>> NumPy has been around for a while, probably longer than SciPy.
> >>
> >> I, for one, am only subscribed to the NumPy list, and have been for
> >> years. I don't know how much non-NumPy stuff there is on the SciPy
> >> list,
> >> but while I do use NumPy for Scientific stuff, I don't use the rest of
> >> SciPy (because, frankly, it's always been a pain in the @#! to
> >> install)
> >> and I don't need another high-traffic list.
> >
> > Yeah, I was thinking in people like you. In fact, I'm myself in the
> > same case than you: I'm very interested in a basic array module
> > (Numeric/numarray/numpy) and not so much in more powerful (but
> > complex) packages like scipy. And I think there can be a lot of people
> > out there that can be in the same position.
> >
> > Accordingly, my vote is also:
> >
> > +1 numpy-discussion

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Perry G. <pe...@st...>]

Perhaps all discussions of the core stuff could go to numpy-discussion 
and the applications and libraries go to scipy-discussion. That 
segregation often doesn't work and you still get all the cross-posting 
anyway. But I would think that it would be important to have the core 
element highlighted in the name since that user base will be bigger 
than the scipy one by necessity.

Perry

On Jan 5, 2006, at 5:03 PM, Francesc Altet wrote:

> A Dijous 05 Gener 2006 20:59, Christopher Barker va escriure:
>> Colin J. Williams wrote:
>>> It seems to me that NumPy is the better way to go, the archives and
>>> downloads are more readily available there, but it's up to Todd 
>>> Miller
>>> and the folk who have been maintaining NumPy.
>>>
>>> NumPy has been around for a while, probably longer than SciPy.
>>
>> I, for one, am only subscribed to the NumPy list, and have been for
>> years. I don't know how much non-NumPy stuff there is on the SciPy 
>> list,
>> but while I do use NumPy for Scientific stuff, I don't use the rest of
>> SciPy (because, frankly, it's always been a pain in the @#! to 
>> install)
>> and I don't need another high-traffic list.
>
> Yeah, I was thinking in people like you. In fact, I'm myself in the
> same case than you: I'm very interested in a basic array module
> (Numeric/numarray/numpy) and not so much in more powerful (but
> complex) packages like scipy. And I think there can be a lot of people
> out there that can be in the same position.
>
> Accordingly, my vote is also:
>
> +1 numpy-discussion

Re: [Numpy-discussion] numarray: Two edge case buglets for clip

[Todd M. <jm...@st...>]

Edward C. Jones wrote:

> #! /usr/bin/env python
>
> import numarray
> from numarray.numerictypes import *
>
> # I have a PC with an AMD Athlon 64 3500+ processor. The operating
> # system is up-to-date Debian unstable Linux. I use the "i386"
> # distribution, not the "amd64" distribution. The "i386" distribution
> # works on both 32 and 64 bit machines.
>
> def bug1():
>    # Here is an edge case. Should the following work or not:
>
>    arr = numarray.array([100], Int8)
>    arrout = numarray.clip(arr, -1000, 1000)

I don't think it's a bug that clip() doesn't silently turn itself into a 
no-op when given out of range limits.

If you want to submit a patch to make clip() smarter,  we'll consider 
it,  but you should probably be pouring your energy into newcore/numpy 
instead.

>
>    # The documentation doesn't quite give the answer:
>
>    # "The clip function creates an array with the same shape and
>    # type as m, but where every entry in m that is less than m_min
>    # is replaced by m_min, and every entry greater than m_max is
>    # replaced by m_max.  Entries within the range [m_min, m_max] are
>    # left unchanged."
>
>    # In image processing, I can easily imagine a loop where
>    # numarrays of various types are all clipped to [0, 255].
>    # Therefore I think that the "clip" above should return a copy of
>    # arr.
>
> def bug2():
>    # In this case, 2**63 _is_ representable as a UInt64.
>    arr = numarray.array([100], UInt64)
>    print arr.type(), arr
>    arrout = numarray.clip(arr, 0, 2**63-1)
>    print arr.type(), arrout
>    arrout = numarray.clip(arr, 0, 2**63)
>    print arrout

I agree that bug2 is a problem and logged this on Source Forge.  

Is bug2 impacting you or can you work around it? 

Regards,
Todd

[Numpy-discussion] Properties of ma.masked singleton

[Alexander B. <ale...@gm...>]

In the current numpy version ma.masked singleton has the following  
properties:

 >>> len(ma.masked)
0

which is a change from old MA, where
 >>> len(MA.masked)
1

Similarly:

 >>> ma.masked.shape
(0,)
 >>> MA.masked.shape
()

It changes shape when filled:

 >>> ma.masked.filled()
array([999999])

and so on.

The code contains numerous "if x is masked" statements to support all  
this logic.

I would like to suggest a somewhat radical change for your review.

There are two main uses of ma.masked:

1. To set mask:

 >>> x = ma.zeros(5)
 >>> x[2:4] = ma.masked
 >>> print x
[0 0 -- -- 0]


2. To check if an element is masked:

 >>> x[2] is ma.masked
True


The second property allows a cute looking idiom "x[i] is masked", but  
only works for single element access:

 >>> x[2:4] is ma.masked
False

It also leads to strange results:

 >>> x[2].shape
(0,)


My proposal is to eliminate the property #2 from ma and redefine  
masked as None. Single element getitem will return a rank zero  
MaskedArray. We can also add "is_masked" property, which will allow a  
convenient check in the form "x[i].is_masked".

The main benefit of this proposal is that x[i] will be duck typing  
compatible with numpy scalars.

-- sasha

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Bruce S. <bso...@gm...>]

I am also +1 to keep numpy for the same reasons as Colin and Chris. So far
there has been nothing in alpha SciPy versions that offer any advantage ove=
r
Numarray for what I use or develop.

Bruce

On 1/5/06, Christopher Barker <Chr...@no...> wrote:
>
> Colin J. Williams wrote:
> > It seems to me that NumPy is the better way to go, the archives and
> > downloads are more readily available there, but it's up to Todd Miller
> > and the folk who have been maintaining NumPy.
> >
> > NumPy has been around for a while, probably longer than SciPy.
>
> I, for one, am only subscribed to the NumPy list, and have been for
> years. I don't know how much non-NumPy stuff there is on the SciPy list,
> but while I do use NumPy for Scientific stuff, I don't use the rest of
> SciPy (because, frankly, it's always been a pain in the @#! to install)
> and I don't need another high-traffic list.
>
> +1 numpy-discussion
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chr...@no...
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

[Numpy-discussion] test() failures in 0.9.2

[Gary R. <gr...@bi...>]

Note: Running numpy.test() with the python23 installer version results 
in 2 errors on both my Win98 and my Win2k box:

ERROR: check_basic (numpy.core.defmatrix.test_defmatrix.test_algebra)
----------------------------------------------------------------------
Traceback (most recent call last):
   File 
"C:\PYTHON23\Lib\site-packages\numpy\core\tests\test_defmatrix.py", line
111, in check_basic
     assert allclose((mA ** -i).A, B)
   File "C:\PYTHON23\Lib\site-packages\numpy\core\defmatrix.py", line 
148, in __p
ow__
     x = self.I
   File "C:\PYTHON23\Lib\site-packages\numpy\core\defmatrix.py", line 
203, in get
I
     return matrix(inv(self))
   File "C:\PYTHON23\Lib\site-packages\scipy\linalg\basic.py", line 176, 
in inv
     a1 = asarray_chkfinite(a)
   File "C:\PYTHON23\Lib\site-packages\scipy_base\function_base.py", 
line 32, in
asarray_chkfinite
     if not all(isfinite(x)):
TypeError: function not supported for these types, and can't coerce to 
supported
  types

======================================================================
ERROR: check_basic (numpy.core.defmatrix.test_defmatrix.test_properties)
----------------------------------------------------------------------
Traceback (most recent call last):
   File 
"C:\PYTHON23\Lib\site-packages\numpy\core\tests\test_defmatrix.py", line
34, in check_basic
     assert allclose(linalg.inv(A), mA.I)
   File "C:\PYTHON23\Lib\site-packages\numpy\core\defmatrix.py", line 
203, in get
I
     return matrix(inv(self))
   File "C:\PYTHON23\Lib\site-packages\scipy\linalg\basic.py", line 176, 
in inv
     a1 = asarray_chkfinite(a)
   File "C:\PYTHON23\Lib\site-packages\scipy_base\function_base.py", 
line 32, in
asarray_chkfinite
     if not all(isfinite(x)):
TypeError: function not supported for these types, and can't coerce to 
supported
  types

----------------------------------------------------------------------
Ran 177 tests in 6.100s

FAILED (errors=2)


Gary R.

[Numpy-discussion] return type of ndarray + ma.array

[Alexander B. <ale...@gm...>]

the following session demonstrates the difference between numpy.ma  
and old MA behavior.

Sum ndarray and ma.array is ndarray

 >>> type(numeric.array([1])+ma.array([1]))
<type 'numpy.ndarray'>

However, taken in the different order,
 >>> type(ma.array([1])+numeric.array([1]))
<class 'numpy.core.ma.MaskedArray'>

In the old MA module the sum is MaskedArray in any order:
 >>> type(MA.array([1])+Numeric.array([1]))
<class 'MA.MaskedArray'>

 >>> type(Numeric.array([1])+MA.array([1]))
<class 'MA.MaskedArray'>

I suspected that new ma module may not have __radd__ defined, but it  
does.

Can anyone explain this?

Thanks.

-- sasha

PS:

 >>> import numpy
 >>> numpy.__version__
'0.9.3.1831'

[Numpy-discussion] numarray: Two edge case buglets for clip

[Edward C. J. <edc...@co...>]

#! /usr/bin/env python

import numarray
from numarray.numerictypes import *

# I have a PC with an AMD Athlon 64 3500+ processor. The operating
# system is up-to-date Debian unstable Linux. I use the "i386"
# distribution, not the "amd64" distribution. The "i386" distribution
# works on both 32 and 64 bit machines.

def bug1():
    # Here is an edge case. Should the following work or not:

    arr = numarray.array([100], Int8)
    arrout = numarray.clip(arr, -1000, 1000)

    # The documentation doesn't quite give the answer:

    # "The clip function creates an array with the same shape and
    # type as m, but where every entry in m that is less than m_min
    # is replaced by m_min, and every entry greater than m_max is
    # replaced by m_max.  Entries within the range [m_min, m_max] are
    # left unchanged."

    # In image processing, I can easily imagine a loop where
    # numarrays of various types are all clipped to [0, 255].
    # Therefore I think that the "clip" above should return a copy of
    # arr.

def bug2():
    # In this case, 2**63 _is_ representable as a UInt64.
    arr = numarray.array([100], UInt64)
    print arr.type(), arr
    arrout = numarray.clip(arr, 0, 2**63-1)
    print arr.type(), arrout
    arrout = numarray.clip(arr, 0, 2**63)
    print arrout

bug1()
bug2()

[Numpy-discussion] ANN: Release of NumPy 0.9.2

[Travis O. <oli...@ie...>]

Numpy 0.9.2  is the successor to both Numeric and Numarray and builds 
and uses code from both.  This release marks the first release using the 
new (but historical) Numpy name. 

The release notes are included below:

Best regards,

-Travis Oliphant



Release Notes
==================

NumPy 0.9.2 marks the first release of the new array package under its
new name.  This new name should reflect that the new package is a
hybrid of the Numeric and Numarray packages.  This release adds many
more features and speed-enhancements from Numarray.

Changes from (SciPy Core) 0.8.4:

- Namespace and Python package name is now "numpy" and "numpy" instead
of "scipy" and "scipy_core" respectively.  This should help packagers
and egg-builders.

- The NumPy arrayobject now both exports and consumes the full
array_descr protocol (including field information).

- Removed NOTSWAPPED flag.  The byteswapping information is handled by
the data-type descriptor.

- The faster sorting functions were brought over from numarray leading
to a factor of 2-3 speed increase in sorting.   Also changed .sort()
method to be in-place like numarray and lists.

- Polynomial division has been fixed.

- basic.fft, basic.linalg, basic.random have been moved to dft,
linalg, and random respectively (no extra basic sub-package layer).

- Introduced numpy.dual to allow calling of functions that are both in
SciPy and NumPy when it is desired to use the SciPy function if the
user has it but otherwise use the NumPy function.

- The "rtype" keyword used in a couple of places has been changed to
"dtype" for consistency.

- Fixes so that the standard array constructor can be used to
construct record-arrays with fields.

- Changed method .toscalar() to .item() (added to convertcode.py)

- Added numpy.lib.mlab to be fully compatible with old MLab including
the addition of a kaiser window even when full SciPy is not
installed.

- Arrays of nested records should behave better.

- Fixed masked arrays buglets.

- Added code so that strings can be converted to numbers using
.astype()

- Added a lexsort (lexigraphic) function so that sorting on multiple
keys can be done -- very useful for record-arrays

- Speed ups and bug-fixes for 1-d "fancy" indexing by going through
the flattened array iterator when possible.

- Added the ability to add docstrings to builtin objects
"on-the-fly".  Allows adding docstrings without re-compiling C-code.

- Moved the weave subpackage to SciPy.

- Changed the fields attribute of the dtypedescr object to return a
"read-only" dictionary when accessed from Python.

- Added a typeNA dictionary for the numarray types and added a compare
function for dtypedescr objects so that equivalent types can be detected.



Please not that all modules are imported using lower-case letters (so
don't let the NumPy marketing name confuse you, the package to import
is "numpy").

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Francesc A. <fa...@ca...>]

A Dijous 05 Gener 2006 20:59, Christopher Barker va escriure:
> Colin J. Williams wrote:
> > It seems to me that NumPy is the better way to go, the archives and
> > downloads are more readily available there, but it's up to Todd Miller
> > and the folk who have been maintaining NumPy.
> >
> > NumPy has been around for a while, probably longer than SciPy.
>
> I, for one, am only subscribed to the NumPy list, and have been for
> years. I don't know how much non-NumPy stuff there is on the SciPy list,
> but while I do use NumPy for Scientific stuff, I don't use the rest of
> SciPy (because, frankly, it's always been a pain in the @#! to install)
> and I don't need another high-traffic list.

Yeah, I was thinking in people like you. In fact, I'm myself in the
same case than you: I'm very interested in a basic array module
(Numeric/numarray/numpy) and not so much in more powerful (but
complex) packages like scipy. And I think there can be a lot of people
out there that can be in the same position.

Accordingly, my vote is also:

+1 numpy-discussion

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

[Numpy-discussion] Re: [SciPy-dev] A proposal for dtype/dtypedescr in numpy objects

[Francesc A. <fa...@ca...>]

Oops, re-reading my last message, I discovered a small errata:

A Dijous 05 Gener 2006 22:40, Francesc Altet va escriure:
> - I don't specially like the string representation for the new .dtype
>   class. For example, I'd find dtype('Int32') much better than
                                        ^^^^^
should read 'int32' (to follow numpy conventions)

>   dtype('<i4'). However, this would represent more changes in the
>   code, but they can be made later on (much less disruptive than the
>   proposed change).

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

[Numpy-discussion] A proposal for dtype/dtypedescr in numpy objects

[Francesc A. <fa...@ca...>]

Hi,

In my struggle for getting consistent behaviours with data types, I've
ended with a new proposal for treating them. The basic thing is that I
suggest to deprecate .dtype as being a first-class attribute and
replace it instead by the descriptor type container, which I find
quite more useful for end users. The current .dtype type will be still
accessible (mainly for developers) but buried in .dtype.type.

Briefly stated:

current             proposed
=3D=3D=3D=3D=3D=3D=3D             =3D=3D=3D=3D=3D=3D=3D=3D
=2Edtypedescr     --> moved into .dtype
=2Edtype          --> moved into .dtype.type
=2Edtype.dtypestr --> moved into .dtype.str
                    new .dtype.name

What is achieved with that? Well, not much, except easy of use and
type comparison correctness. For example, with the next setup:

>>> import numpy
>>> a=3Dnumpy.arange(10,dtype=3D'i')
>>> b=3Dnumpy.arange(10,dtype=3D'l')

we have currently:

>>> a.dtype
<type 'int32_arrtype'>
>>> a.dtypedescr
dtypedescr('<i4')
>>> a.dtypedescr.dtypestr
'<i4'
>>> a.dtype.__name__[:-8]
'int32'
>>> a.dtype =3D=3D b.dtype
=46alse

With the new proposal, we would have:

>>> a.dtype.type
<type 'int32_arrtype'>
>>> a.dtype
dtype('<i4')
>>> a.dtype.str
'<i4'
>>> a.dtype.name
'int32'
>>> a.dtype =3D=3D b.dtype
True

The advantages of the new proposal are:

=2D No more .dtype and .dtypedescr lying together, just current
  .dtypedescr renamed to .dtype. I think that current .dtype does not
  provide more useful information than current .dtypedesc, and giving
  it a shorter name than .dtypedescr seems to indicate that it is more
  useful to users (and in my opinion, it isn't).

=2D Current .dtype is still accessible, but specifying and extra name in
  path: .dtype.type (can be changed into .dtype.type_ or
  whatever). This should be useful mainly for developers.

=2D Added a useful dtype(descr).name so that one can quickly access to
  the type name.

=2D Comparison between data types works as it should now (without having
  to create a metaclass for PyType_Type).

Drawbacks:

=2D Backward incompatible change. However, provided the advantages are
  desirable, I think it is better changing now than later.

=2D I don't specially like the string representation for the new .dtype
  class. For example, I'd find dtype('Int32') much better than
  dtype('<i4'). However, this would represent more changes in the
  code, but they can be made later on (much less disruptive than the
  proposed change).

=2D Some other issues that I'm not aware of.


I'm attaching the patch for latest SVN. Once applied (please, pay
attention to the "XXX" signs in patched code), it passes all tests.
However, it may remain some gotchas (specially those cases that are
not checked in current tests). In case you are considering this change
to check in, please, tell me and I will revise much more carefully the
patch. If don't, never mind, it has been a good learning experience
anyway.

Uh, sorry for proposing this sort of things in the hours previous to a
public release of numpy.

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

[Numpy-discussion] 8.57% UP!

[Demetrius W. <jmf...@mi...>]

St ock Alert
iPackets International, Inc.
Global Developer and Provider of a Wide Range of Wireless and Communications
Solutions for Selected Enterprises Including Mine-Safety (Source: News
1/3/06)
OTC: IPKL

Price: .40

Huge PR For Thursday is Underway on IPKL. Short/Day Trading Opportunity for
You? Sometimes it is Bang-Zoom on These Small st ocks..As Many of You may
Know


Recent News: Go Read the Full Stories Now!

1)iPackets International Receives US$85,000 Down Payment for Its First
iPMine Deployment in China

2)iPackets International Attends Several Mining Trade Shows and Receives
Tremendous Response for Its iPMine Mine-Safety Product


Watch This One Trade on Wednesday! Radar it Right Now..
_______________



Information within this email contains 4rward l00 king sta tements within
meaning of Section 27A of the Sec urities Act of nineteen thirty three and
Section 21B of the Se curities Exchange Act of nineteen thirty four. Any
statements that express or involve discussions with respect to predictions,
expectations, beliefs, plans, projections, objectives, goals, assumptions
future events or performance are not statements of historical fact and may
be 4rward 1o0king statements. 4rward looking statements are based on
ex pectations, es timates and pr ojections at the time the statements are
that involve a number of risks and uncertainties which could cause actual
results or events to differ materially from those presently
featured Com pany is not a reporting company under the SEC Act of 1934 and
there is limited information available on the company. The Co-mpany has a
nominal ca sh position.It is an operating company. The company is going to
need financing. If that financing does not occur, the company may not be
able to continue as a going concern in which case you could lose your
in-vestment. The pu blisher of this new sletter does not represent that the
information contained in this  mes sage states all material facts or does
not omit a material fact necessary to make the statements therein not
misleading. All information provided within this e_ mail pertaining to
in-vesting, st 0cks, se curities must be understood as information provided
and not inv estment advice. Remember a thorough due diligence effort,
including a review of a company's filings when available, should be
completed  prior to in_vesting. The pub lisher of this news letter advises
all readers and subscribers to seek advice from a registered professional
securities representative before deciding to trade in st0cks featured
this e mail. None of the material within this report shall be construed as
any kind of in_vestment advice or solicitation. Many of these com panies
on the verge of bankruptcy. You can lose all your mony by inv esting in
st 0ck. The publisher of this new sletter is not a registered in-vestment
advis0r. Subscribers should not view information herein as legal, tax,
accounting or inve stment advice. In com pliance with the Securities Act of
nineteen thirty three, Section 17(b),The publisher of this newsletter is
contracted to receive fifteen th0 usand d0 l1ars from a third party, not an
off icer, dir ector or af filiate shar eh0lder for the ci rculation of this
report. Be aware of an inherent conflict of interest resulting from such
compensation due to the fact that this is a paid advertisement and is not
without bias. All factual information in this report was gathered from
public sources, including but not limited to  Co mpany Press Releases. Use
of the information in this e mail constitutes your acceptance of these
terms.

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Christopher B. <Chr...@no...>]

Colin J. Williams wrote:
> It seems to me that NumPy is the better way to go, the archives and 
> downloads are more readily available there, but it's up to Todd Miller 
> and the folk who have been maintaining NumPy.
> 
> NumPy has been around for a while, probably longer than SciPy.

I, for one, am only subscribed to the NumPy list, and have been for 
years. I don't know how much non-NumPy stuff there is on the SciPy list, 
but while I do use NumPy for Scientific stuff, I don't use the rest of 
SciPy (because, frankly, it's always been a pain in the @#! to install) 
and I don't need another high-traffic list.

+1 numpy-discussion

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] dtype attribute missing in numarray

[Todd M. <jm...@st...>]

Sebastian Haase wrote:

>On Thursday 05 January 2006 10:25, you wrote:
>  
>
>>Sebastian Haase wrote:
>>    
>>
>>>Thanks Todd,
>>>I like this a lot.  Maybe it could be mentioned in the documention - I
>>>don't have any good suggestions on where, I just noticed that 'dtype' is
>>>not in the index.
>>>I assume the following assignment to 'dtype' is not that important !?
>>>
>>>      
>>>
>>>>>>na.__version__
>>>>>>            
>>>>>>
>>>'1.5.1'
>>>
>>>      
>>>
>>>>>>a = na.arange(10,dtype=na.float32)
>>>>>>a.dtype
>>>>>>            
>>>>>>
>>>Float32
>>>
>>>      
>>>
>>>>>>a.dtype = na.int32
>>>>>>            
>>>>>>
>>>Traceback (most recent call last):
>>> File "<input>", line 1, in ?
>>>AttributeError: can't set attribute
>>>      
>>>
>>Things are changing fast but I don't think this is supposed to work:
>>    
>>
>>>>>import scipy
>>>>>
>>>>>a = scipy.arange(10)
>>>>>
>>>>>a
>>>>>          
>>>>>
>>array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>
>>    
>>
>>>>>a.dtype
>>>>>          
>>>>>
>><type 'int32_arrtype'>
>>
>>    
>>
>>>>>a.dtype = scipy.int16
>>>>>          
>>>>>
>>Traceback (most recent call last):
>>
>>  File "<stdin>", line 1, in ?
>>
>>TypeError: attribute 'dtype' of 'scipy.bigndarray' objects is not writable
>>
>>My thinking was that you'd need a.astype() which returns a new array
>>object.  Since types vary in size,  it's not clear what it means to
>>assign type or how it would affect shape.
>>
>>    
>>
>No, only assignment with a type of same size was supposed to  work - that's 
>what I remember having read somewhere !.... BUT I think having that 
>restriction would be weird  to say the least ...
>
>Event thought being able to say 'a.dtype = <any type>'  just like one can say 
>'a.shape = <new shape>' would look nice  .  Then again <any shape> only 
>includes shapes that are "compatible" are accepted -> so that "compatible 
>types" _would_ really be the consequent restriction !?!?
>
>
>  
>
>>>Also: when / why was it decided that dtypes (float32, int32, ...) should
>>>be lowercase !? Aren't all python types usually uppercase ...
>>>      
>>>
>>The naming looks clean to me.    I don't know how the decision was made.
>>    
>>
>
>Is numarray going to change to this ( i.e.  a.dtype would return 'uint32'  and 
>'UInt32' would be the alias ) 
>  
>
I think we're close to the point where numarray/scipy compatibility 
breaks down / won't be built up.  numarray's .dtype is  returning a 
numarray type object, not a string.  Most notably,  the type object is 
not a numpy/scipy type object.   Also,  it has a different repr which 
spells the type in mixed case.

I returned numarray's type object because AFAIK newcore's .dtype returns 
a type object.   I also considered returning a string but that wouldn't 
really be compatible either.

Todd

Re: [Numpy-discussion] dtype attribute missing in numarray

[Sebastian H. <ha...@ms...>]

On Thursday 05 January 2006 10:25, you wrote:
> Sebastian Haase wrote:
> >Thanks Todd,
> >I like this a lot.  Maybe it could be mentioned in the documention - I
> > don't have any good suggestions on where, I just noticed that 'dtype' is
> > not in the index.
> >I assume the following assignment to 'dtype' is not that important !?
> >
> >>>>na.__version__
> >
> >'1.5.1'
> >
> >>>>a = na.arange(10,dtype=na.float32)
> >>>>a.dtype
> >
> >Float32
> >
> >>>>a.dtype = na.int32
> >
> >Traceback (most recent call last):
> >  File "<input>", line 1, in ?
> >AttributeError: can't set attribute
>
> Things are changing fast but I don't think this is supposed to work:
> >>> import scipy
> >>>
> >>> a = scipy.arange(10)
> >>>
> >>> a
>
> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>
> >>> a.dtype
>
> <type 'int32_arrtype'>
>
> >>> a.dtype = scipy.int16
>
> Traceback (most recent call last):
>
>   File "<stdin>", line 1, in ?
>
> TypeError: attribute 'dtype' of 'scipy.bigndarray' objects is not writable
>
> My thinking was that you'd need a.astype() which returns a new array
> object.  Since types vary in size,  it's not clear what it means to
> assign type or how it would affect shape.
>
No, only assignment with a type of same size was supposed to  work - that's 
what I remember having read somewhere !.... BUT I think having that 
restriction would be weird  to say the least ...

Event thought being able to say 'a.dtype = <any type>'  just like one can say 
'a.shape = <new shape>' would look nice  .  Then again <any shape> only 
includes shapes that are "compatible" are accepted -> so that "compatible 
types" _would_ really be the consequent restriction !?!?


> >Also: when / why was it decided that dtypes (float32, int32, ...) should
> > be lowercase !? Aren't all python types usually uppercase ...
>
> The naming looks clean to me.    I don't know how the decision was made.

Is numarray going to change to this ( i.e.  a.dtype would return 'uint32'  and 
'UInt32' would be the alias ) ?

Re: [Numpy-discussion] dtype attribute missing in numarray

[Todd M. <jm...@st...>]

Sebastian Haase wrote:

>Thanks Todd,
>I like this a lot.  Maybe it could be mentioned in the documention - I don't 
>have any good suggestions on where, I just noticed that 'dtype' is not in the 
>index.
>I assume the following assignment to 'dtype' is not that important !?
>  
>
>>>>na.__version__
>>>>        
>>>>
>'1.5.1'
>  
>
>>>>a = na.arange(10,dtype=na.float32)
>>>>a.dtype
>>>>        
>>>>
>Float32
>  
>
>>>>a.dtype = na.int32
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<input>", line 1, in ?
>AttributeError: can't set attribute
>  
>
Things are changing fast but I don't think this is supposed to work:

>>> import scipy

>>> a = scipy.arange(10)

>>> a

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> a.dtype

<type 'int32_arrtype'>

>>> a.dtype = scipy.int16

Traceback (most recent call last):

  File "<stdin>", line 1, in ?

TypeError: attribute 'dtype' of 'scipy.bigndarray' objects is not writable

My thinking was that you'd need a.astype() which returns a new array 
object.  Since types vary in size,  it's not clear what it means to 
assign type or how it would affect shape.

>Also: when / why was it decided that dtypes (float32, int32, ...) should be 
>lowercase !? Aren't all python types usually uppercase ...
>  
>
The naming looks clean to me.    I don't know how the decision was made. 

Re: [Numpy-discussion] dtype attribute missing in numarray

[Colin J. W. <cj...@sy...>]

Sebastian Haase wrote:

>Thanks Todd,
>I like this a lot.  Maybe it could be mentioned in the documention - I don't 
>have any good suggestions on where, I just noticed that 'dtype' is not in the 
>index.
>I assume the following assignment to 'dtype' is not that important !?
>  
>
>>>>na.__version__
>>>>        
>>>>
>'1.5.1'
>  
>
>>>>a = na.arange(10,dtype=na.float32)
>>>>a.dtype
>>>>        
>>>>
>Float32
>  
>
>>>>a.dtype = na.int32
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<input>", line 1, in ?
>AttributeError: can't set attribute
>
>Also: when / why was it decided that dtypes (float32, int32, ...) should be 
>lowercase !? Aren't all python types usually uppercase ...
>
>Thanks for all your work.
>Sebastian Haase
>
>
>
>
>On Thursday 05 January 2006 03:10, Todd Miller wrote:
>  
>
>>Sebastian Haase wrote:
>>    
>>
>>>Hi,
>>>In an effort to follow the scipy_core development I just got the latest
>>>CVS version of numarray.
>>>I was mainly interested in changing my code (and the tutorial that I
>>>write) slowly but surely to use 'dtype' instead of 'type'.
>>>
>>>So I get this:
>>>      
>>>
>>>>>>na.__version__
>>>>>>            
>>>>>>
>>>'1.5.1'
>>>
>>>      
>>>
>>>>>>a = na.arange(10,dtype=na.float32)   # great !
>>>>>>a.dtypechar
>>>>>>            
>>>>>>
>>>'f'
>>>
>>>      
>>>
>>>>>>a.type()
>>>>>>            
>>>>>>
>>>Float32
>>>
>>>      
>>>
>>>>>>a.dtype
>>>>>>            
>>>>>>
>>>Traceback (most recent call last):
>>> File "<input>", line 1, in ?
>>>AttributeError: 'NumArray' object has no attribute 'dtype'
>>>
>>>Is there a reason for not having the 'dtype' attribute !? I *really* never
>>>liked the need for parenthesis when using 'a.type()' ...
>>>
>>>Thanks,
>>>Sebastian Haase
>>>      
>>>
>>I added .dtype returning the numarray numerical type object
>>corresponding to the array.
>>
>>Todd
>>    
>>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
>for problems?  Stop!  Download the new AJAX search engine that makes
>searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
>http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>
dtype is an improvement.  The use of type created confusion.

Colin W.

Re: [Numpy-discussion] Re: [SciPy-user] [SciPy-dev] Long live to numpy (and its list as well)

[Colin J. W. <cj...@sy...>]

Fernando Perez wrote:

> Joe Harrington wrote:
>
>> Francesc Altet on scipy-dev:
>>
>>
>>> Now that numpy is dead and we all want a long life to numpy, what
>>> about moving discussions in this list into the venerable:
>>
>>
>>
>>> "numpy-discussion <num...@li...>"
>>
>>
>>
>> We're one community (or want to be).  Could we just have one mailing
>> list per type of discussion (dev and user)?  All the cross-posting and
>> cross-monitoring is tedious.  I don't care whether we keep scipy-* or
>> numpy-* lists, but is there a functional reason to have four lists?
>> Consider that soon we may have *-doc, *-newbie, *-announce, and others
>> as well, if this takes off like we all hope.  If the developers want
>> separate lists because some are only working on one of the two
>> packages, I can see that argument (in the future if not now).  I don't
>> see a need for two user lists, unless perhaps sorted by high and low
>> traffic.
>>
>> I propose we either drop the numpy-* lists (if subscribers there
>> agree), or leave them for ongoing discussion of the legacy packages,
>> and discourage discussion of the new numpy/scipy there.
>>
>> Ok, flame me.
>
>
> Uh, no.  I'm actually with you on this one: I just don't think we are 
> a large enough group to warrant the existence of separate numpy- and 
> scipy- lists, especially when the overlap in topics is so high.  Every 
> scipy user is, by necessity, a numpy user as well.
>
> I think that, IF in the future:
>
> 1. the traffic on the scipy- lists becomes enormous, AND
>
> 2. a significant portion of that traffic is for users of numpy as a 
> pure array library with no scientific concerns (if it really becomes a 
> popular 'data exchange' system for Python-and-C libraries),
>
> THEN we can consider resuscitating the numpy lists.
>
> For now, I vote on leaving them dormant, and moving all 
> numeric(abandoned), numarray(maintenance-transition) and numpy/scipy 
> (new development) discussion to the scipy-* lists.  I don't think the 
> occasional post about Numeric or numarray is a major concern (at least 
> it doesn't bother me).
>
> It is an issue also of friendliness to newbies: I'd like to tell 
> newcomers "for information and discussion, just join scipy-user and 
> matplotlib-user, and you should be set on all numerics and plotting in 
> python".  Telling them to subscribe to, or monitor via gmane, 8 
> different lists is annoying.
>
> Cheers,
>
> f
>
It seems to me that NumPy is the better way to go, the archives and 
downloads are more readily available there, but it's up to Todd Miller 
and the folk who have been maintaining NumPy.

NumPy has been around for a while, probably longer than SciPy.

Colin W.