numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Numeric to Numarray migration

[Todd M. <jm...@st...>]

On Tue, 2004-03-16 at 05:59, Vineet Jain wrote:
> I'm looking to port my application to Numarray and would like some help:
> 
> 1. How do I do the following in numarray (since from numarray import
> PyObject fails):
> 
> 	self.startDateTimeList          = zeros(n, PyObject)

>>> import numarray.objects as obj
>>> self.startDateTimeList = obj.fromlist([0]*n)

Note that numarray.objects is an independent implementation of object
arrays which is just now coming into wider use.  I expect that there
will be gaps in functionality relative to Numeric so please ask if you
have questions.  The docstrings are currently the only documentation for
numarray's object arrays.

> 
> 2. I have a time series array which has invalid elements. I use the
> following two functions:
> 

Maybe someone else with more intuition about what you're trying to do
can answer this. 

If not,  get a copy of the numarray manual here:
http://prdownloads.sourceforge.net/numpy/numarray-0.9.pdf?download

I'd look at the functions compress(), putmask(), put(), take(), and
nonzero().  Array indexing may also come in handy.

> To get my compressed array (without the invalid elements) I use:
> 
>             self.high = extract(prices1.high, prices1.mask)
> 
> I then do some computations on the high in an external c library which
> populates a new array
> in the compressed format. Then I use:
> 
>         insert(self.outIndicator,self.prices1.mask,self._outIndicator)
> 
> This function insert from the scipy_base package was a lot faster than my
> python equivalent. But since it
> is called a lot of times the insert operation is takes a siginificant amount
> of time.
> 
>    608400  195.333    0.000  195.333    0.000
> C:\Python23\lib\site-packages\scipy_base\function_base.py:281(insert)

If you want to use numarray and don't get the speed you want with Python
code, you might try porting the function from SciPy to numarray using
the Numeric compatibility layer.

> 
> It represents about 25% of the application overall time. How do I do the
> same in numarray and will the performance
> be better than the above implementation. My arrays are usually greater than
> 20k in length.

Since your array sizes are large,  numarray should be fairly competitive
and depending on how the actual code works out, might even be faster. 
My guess though, is that numarray will be a little slower than Numeric
for the array sizes you're talking about and the functions you're likely
to use.

Regards,
Todd

> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

[Numpy-discussion] corrcoeff on big matrix

[CL WU <ane...@ho...>]

Hi, group,
              I have a big "Float64" matrix (42x22300) and I want to get 
its correlation coefficient matrix, but I got the error as the following:

 >>> data.shape
(42, 22300)
 >>> mlab.corrcoef(data)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python23\Lib\site-packages\numarray\linear_algebra\mlab.py", 
line 300, in corrcoef
    c = cov(x, y)
  File "C:\Python23\Lib\site-packages\numarray\linear_algebra\mlab.py", 
line 294, in cov
    val = squeeze(dot(transpose(m),conjugate(y)) / fact)
  File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
1150, in dot
    return ufunc.innerproduct(array1, _gen.swapaxes(array2, -1, -2))
  File "C:\Python23\Lib\site-packages\numarray\ufunc.py", line 2047, in 
innerproduct
    r = a.__class__(shape=adots+bdots, type=rtype)
ValueError: new_memory: invalid region size: -633294592.

I suspect corrcoef function can not handle such a big matrix. If so, 
what is the upper limit for array size? How can I get around this 
problem in numarray?

BTW, I am using numarray 0.9/python 2.3.3 on win2kSP4

Thanks.

Chunlei

[Numpy-discussion] Numeric to Numarray migration

[Vineet J. <vi...@es...>]

I'm looking to port my application to Numarray and would like some help:

1. How do I do the following in numarray (since from numarray import
PyObject fails):

	self.startDateTimeList          = zeros(n, PyObject)

2. I have a time series array which has invalid elements. I use the
following two functions:

To get my compressed array (without the invalid elements) I use:

            self.high = extract(prices1.high, prices1.mask)

I then do some computations on the high in an external c library which
populates a new array
in the compressed format. Then I use:

        insert(self.outIndicator,self.prices1.mask,self._outIndicator)

This function insert from the scipy_base package was a lot faster than my
python equivalent. But since it
is called a lot of times the insert operation is takes a siginificant amount
of time.

   608400  195.333    0.000  195.333    0.000
C:\Python23\lib\site-packages\scipy_base\function_base.py:281(insert)

It represents about 25% of the application overall time. How do I do the
same in numarray and will the performance
be better than the above implementation. My arrays are usually greater than
20k in length.

Thanks for Numeric, I love the package and it is helped me immensely.

Vineet

Re: [Numpy-discussion] Re: numarray.objects missing in documentation

[Todd M. <jm...@st...>]

Hi Faheem,

In order to create an object array, do this:

>>> import numarary.objects as obj
>>> a = obj.fromlist(["t","u","w",1])
>>> a
ObjectArray(["t","u","w",1)

Unfortunately, the ability to apply arbitrary functions to all the elements
of an object array has never been implemented.  Nevertheless, the following
might do:

def apply(array, function, result=None):
        rank1 = (array.rank == 1)
        if result is None:
                result0 = array.copy()
        else:
                result0 = result
        for x in range(array.shape[0]):
            if rank1:
                result0[x] = function(array[x])
            else:
                apply(array[x], function, result0[x])
        return result0

Re-ordering the loop and the if statement results in faster but slightly
more verbose code.

Regards,
Todd Miller


----- Original Message ----- 
From: "Faheem Mitha" <fa...@em...>
To: <num...@li...>
Sent: Sunday, March 14, 2004 12:20 PM
Subject: [Numpy-discussion] Re: numarray.objects missing in documentation


> On Sun, 14 Mar 2004 09:55:52 -0500, Todd Miller <jm...@st...> wrote:
> > On Sun, 2004-03-14 at 00:26, Faheem Mitha wrote:
> >> Dear People,
> >>
> >> I'm posting this to gmane.comp.python.numeric.general. I am not sure
> >> what mailing list this corresponds to. I earlier tried to post this to
> >> the mailing list mentioned in the numarray documentation
> >> (nu...@li...), but it bounced.
> >>
> >> I don't see numarray.objects listed in
> >> http://stsdas.stsci.edu/numarray/numarray-0.9.html/genindex.html
> >
> > The section on object arrays has yet to be written.  What is it that you
> > want to know?
>
> Thanks for the quick reply. I'm trying to make an array of python
> objects as follows. Specifically, given a user defined class, I want
> to creata an array of instances of that class.
>
> Consider the following class.
>
> class cell:
>     def setrow(self,row):
>         self.row = row
>     def setcol(self,col):
>         self.col = col
>     ...
>
> I create an "empty" cell.
>
> empcell = cell()
>
> Now I want to create an array of objects identical to empcell, and
> later initialize them.
>
> I tried stuff along the lines of
>
> ******************************************************************
> import string
> from numarray import *
>
> empcell = cell()
>
> base = [empcell]
>
> mat = fromlist(base,shape=(2,2))
> ******************************************************************
>
> But the
>
> mat = fromlist(base,shape=(2,2))
>
> line fails with
>
> AttributeError: cell instance has no attribute '__len__'
>
> Similar things fail with the same error message. No doubt I'm doing
> something stupid, but would someone please enlighten me?
>
> I got the fromlist command from
>
> >>> help("numarray.objects")
>
> I was a bit disconcerted to find on trying to run the first example
> from the help page that I got
>
> >>> a = fromlist(["t","u","w"])
> ...
> TypeError: Expecting a python numeric type, got something else.
>
> What am I doing wrong?
>
> I'm running
>
> ii  python                    2.3.3-5                   An interactive
> high-level object-oriented language (default versio
> ii  python-numarray           0.8.1-1                   An array
> processing package modelled after Python-Numeric
> ii  python-numeric            23.1-1                    Numerical
> (matrix-oriented) Mathematics for Python
>
> on Debian sarge.
>
> Thanks in advance and sorry for the long-winded message.
>
>                                                           Faheem.
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

[Numpy-discussion] PyTables & numarray 0.9 warning

[Francesc A. <fa...@py...>]

Hi,

Many of you will know that a new version of numarray (0.9) has been
released past week. This new version has a number of cool features
(specially, being faster in certain situations that affect directly to
PyTables efficency :-).

Unfortunately, the new version of numarray has deprecated the "buffer"
keyword on the array() constructor, and that precise keyword was used in
PyTables (just in one line). I've uploaded a new version of PyTables in
the SourceForge repository with a cure on that. Please, if you have
downloaded PyTables *before* March, 9th, download again from the
PyTables web site (http://www.pytables.org) and rebuild the software (or
install the new autoinstallable binaries for Windows).

If you don't feel like having to do that, you can just apply this patch
to get rid of the problem:

-8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-
--- tables/Array.py.orig        2004-03-15 13:56:01.000000000 +0100
+++ tables/Array.py             2004-03-15 13:56:13.000000000 +0100
@@ -483,7 +483,7 @@
         if repr(self.type) == "CharType":
             arr = strings.array(None, itemsize=self.itemsize, shape=shape)
         else:
-            arr = numarray.array(buffer=None, type=self.type, shape=shape)
+            arr = numarray.array(None, type=self.type, shape=shape)
             # Set the same byteorder than on-disk
             arr._byteorder = self.byteorder
         # Protection against reading empty arrays
-8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<--8<-

My apologies for the inconveniences,

-- 
Francesc Alted

[Numpy-discussion] map for numarray and automatic calls of elements?

[Shin <sd...@em...>]

In my experience, when using Numeric (numpy), the following two features
turned out to be very useful in many numerical operations.
Please let me know if numarray already supports the features.
Any comments are welcome.

1. A function like map but it returns an array directly, not a list.
For instance, if we call the function as xmap,
>> f = lambda x: x+1
>> map(f, [1,2,3])
[2,3,4]
>> xmap(f, [1,2,3]) # == array(map(f,[1,2,3]))
array([2,3,4])

When dealing with a large volume of arrays, this function
can save memory and improve speed significantly by skipping the creation
of a list.

2. A way to call methods or attributes of object elements from an object
array.

For instance, 
>> from datetime import date
>> x = array([date(1990,1,1), date(2000,1,1)])
>> x.year
array([1990, 2000])

If an object array is called by a methods or an attribute, which the
array doesn't have, the array calls the methods or the attributes of its
elements automatically and returns a collection of the results.

Thanks.

-- 
Daehyok Shin
Geography Department
University of North Carolina, Chapel Hill
email: sd...@em...

[Numpy-discussion] Re: numarray.objects missing in documentation

[Faheem M. <fa...@em...>]

On Sun, 14 Mar 2004 09:55:52 -0500, Todd Miller <jm...@st...> wrote:
> On Sun, 2004-03-14 at 00:26, Faheem Mitha wrote:
>> Dear People,
>> 
>> I'm posting this to gmane.comp.python.numeric.general. I am not sure
>> what mailing list this corresponds to. I earlier tried to post this to
>> the mailing list mentioned in the numarray documentation
>> (nu...@li...), but it bounced.
>> 
>> I don't see numarray.objects listed in
>> http://stsdas.stsci.edu/numarray/numarray-0.9.html/genindex.html
>
> The section on object arrays has yet to be written.  What is it that you
> want to know?

Thanks for the quick reply. I'm trying to make an array of python
objects as follows. Specifically, given a user defined class, I want
to creata an array of instances of that class.

Consider the following class.

class cell:
    def setrow(self,row):
        self.row = row
    def setcol(self,col):
        self.col = col
    ...

I create an "empty" cell.

empcell = cell()

Now I want to create an array of objects identical to empcell, and
later initialize them.

I tried stuff along the lines of

******************************************************************
import string
from numarray import *

empcell = cell()

base = [empcell]

mat = fromlist(base,shape=(2,2))
******************************************************************

But the

mat = fromlist(base,shape=(2,2))

line fails with

AttributeError: cell instance has no attribute '__len__'

Similar things fail with the same error message. No doubt I'm doing
something stupid, but would someone please enlighten me?

I got the fromlist command from

>>> help("numarray.objects")

I was a bit disconcerted to find on trying to run the first example
from the help page that I got

>>> a = fromlist(["t","u","w"])
...
TypeError: Expecting a python numeric type, got something else.

What am I doing wrong?

I'm running

ii  python                    2.3.3-5                   An interactive
high-level object-oriented language (default versio
ii  python-numarray           0.8.1-1                   An array
processing package modelled after Python-Numeric
ii  python-numeric            23.1-1                    Numerical
(matrix-oriented) Mathematics for Python

on Debian sarge.

Thanks in advance and sorry for the long-winded message.

                                                          Faheem.

Re: [Numpy-discussion] numarray.objects missing in documentation

[Todd M. <jm...@st...>]

On Sun, 2004-03-14 at 00:26, Faheem Mitha wrote:
> Dear People,
> 
> I'm posting this to gmane.comp.python.numeric.general. I am not sure
> what mailing list this corresponds to. I earlier tried to post this to
> the mailing list mentioned in the numarray documentation
> (nu...@li...), but it bounced.
> 
> I don't see numarray.objects listed in
> http://stsdas.stsci.edu/numarray/numarray-0.9.html/genindex.html

The section on object arrays has yet to be written.  What is it that you
want to know?

> 
> I can't find it anywhere else in the documentation either. I could not
> find a single page html version of the docuemtation, otherwise I would
> have done a search.
> 
> I do find nice documentation when doing
> >>> help("numarray.objects").
> 
> However there is a minor typo at the beginning.
> 
> "...It defines functions which correspond to most of the most of the
> operators defined in Python's operator module, and provides names
> compatible with most of numarray's universal functions."
> 
> "most of" is repeated.

Thanks.  This is fixed now in CVS.

> 
> Please cc me on any reply; I'm not subscribed. Thanks.
> 
>                                                            Faheem.

Regards,
Todd

-- 
Todd Miller <jm...@st...>

RE: [Numpy-discussion] Installing Numpy

[Nadav H. <na...@vi...>]

Try to get distutils either from rpmfind or your distribution CD =
(downloading and installing the latest python is a good option). It is =
highly recommended to have distutils if you want to install any add-on =
package to python.

  Nadav.

-----Original Message-----
From:	Hoa Nguyen [mailto:hoa...@ya...]
Sent:	Sat 13-Mar-04 20:54
To:	num...@li...
Cc:=09
Subject:	[Numpy-discussion] Installing Numpy
Hi,

Please help me with installing NumPy.
When I run:
        python setup.py install
there was error.
It said that:
        ImportError: No module named distutils

I'm using Mandrake 9.

Thanks,

Hoa Nguyen

__________________________________
Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
http://mail.yahoo.com


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=3D1470&alloc_id=3D3638&op=3Dcli=
ck
_______________________________________________
Numpy-discussion mailing list
Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] numarray.objects missing in documentation

[Faheem M. <fa...@em...>]

Dear People,

I'm posting this to gmane.comp.python.numeric.general. I am not sure
what mailing list this corresponds to. I earlier tried to post this to
the mailing list mentioned in the numarray documentation
(nu...@li...), but it bounced.

I don't see numarray.objects listed in
http://stsdas.stsci.edu/numarray/numarray-0.9.html/genindex.html

I can't find it anywhere else in the documentation either. I could not
find a single page html version of the docuemtation, otherwise I would
have done a search.

I do find nice documentation when doing
>>> help("numarray.objects").

However there is a minor typo at the beginning.

"...It defines functions which correspond to most of the most of the
operators defined in Python's operator module, and provides names
compatible with most of numarray's universal functions."

"most of" is repeated.

Please cc me on any reply; I'm not subscribed. Thanks.

                                                           Faheem.

[Numpy-discussion] Installing Numpy

[Hoa N. <hoa...@ya...>]

Hi,

Please help me with installing NumPy.
When I run:
        python setup.py install
there was error.
It said that:
        ImportError: No module named distutils

I'm using Mandrake 9.

Thanks,

Hoa Nguyen

__________________________________
Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
http://mail.yahoo.com

[Numpy-discussion] ANN: numarray-0.9

[Todd M. <jm...@st...>]

Release Notes for numarray-0.9

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modelled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, and improved type
promotions.

I. ENHANCEMENTS

1. Support for "from __future__ import division"

True division has been implemented for numarray.  This means that
modules that wish to use true division can also use numarray and
numarray division will work as follows:

a. dividing any two integer arrays using "/" results in a Float32
array.

b. dividing two floating point arrays using "//" results in truncation
of the result as in: a // b == floor(a/b).

2. C-coded array slicing

Array slicing has been re-implemented in C-code as part of the
_ndarray module.  This means faster slicing.  Thanks go to Warren
Hack, Chris Hanley, and Ivo Busko for helping debug a huge refcount
error.

3. Decreased Ufunc overhead

Ufunc execution speed has clawed and scratched its way back to where
it was around numarray-0.5.  Improvements here included optimization
of the ufunc caching, smarter thread handling, and smarter support for
subclasses.  The ufunc caching is based on a simple 20 element table
for each ufunc.

4. Faster array creation from C

Code which creates NumArrays from C (including numarray itself) can
now do so faster because the API functions have been modified to do
the array __init__ inline rather than through an expensive Python
callback.

II. BUGS FIXED / CLOSED

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

913781 Another memory leak in example in Chapter 12
908399 Numarray 0.7: "del a[1]" dumps core
899259 astype Int16 copy4bytes: access beyond buffer
895801 Buffer overflow in sum w/ 0-sized array
894810 MemoryError When Creating Large Arrays
890703 getnan() and getinf() failure
883124 and and operator.and respond differently
865410 Usage of __dict__
854480 Slice assignment of float to integer
839367 Overlapping slice assign fails
828941 Numarray: determinant returns scalar or array
820122 Linearalgebra2.determinant problem
817343 Sub-classing of NumArray inhibited by complex values
793336 crash in _sort.pyd
772548 Reference counting errors
683957 Adding certain arrays fails in Numarray

III. CAUTIONS

1. numarray extension writers should note that the documented use of
PyArray_INCREF and PyArray_XDECREF (in numarray) has been found to be
incompatible with Numeric and has therefore been deprecated.  numarray
wrapper functions using PyArray_INCREF and PyArray_XDECREF should
switch to ordinary Py_INCREF and Py_XDECREF.

WHERE
-----------

Numarray-0.9 windows executable installers, source code, and manual is
here:

http://sourceforge.net/project/showfiles.php?group_id=1369

Numarray is hosted by Source Forge in the same project which hosts
Numeric:

http://sourceforge.net/projects/numpy/

The web page for Numarray information is at:

http://stsdas.stsci.edu/numarray/index.html

Trackers for Numarray Bugs, Feature Requests, Support, and Patches are
at the Source Forge project for NumPy at:

http://sourceforge.net/tracker/?group_id=1369

REQUIREMENTS
------------------------------

numarray-0.9 requires Python 2.2.2 or greater.  


AUTHORS, LICENSE
------------------------------

Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC
Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science
Institute.  We'd like to acknowledge the assitance of Francesc Alted,
Paul Dubois, Sebastian Haase, Tim Hochberg, Nadav Horesh, Edward
C. Jones, Eric Jones, Jochen Küpper, Travis Oliphant, Pearu Peterson,
Peter Verveer, Colin Williams, and everyone else who has contributed
with comments and feedback.

Numarray is made available under a BSD-style License.  See
LICENSE.txt in the source distribution for details.

-- 
Todd Miller             jm...@st...

Re: [Numpy-discussion] A simple question about type().

[<fe...@la...>]

As others have pointed out, array is a function.  You want ArrayType.

> type(my_array) is ArrayType
> True

-r

Andrea Riciputi writes:
 > You are right! But I can't understand why. When I try type() with an=20
 > array I get:
 > 
 > type(my_array)
 > <type 'array'>
 > 
 > but:
 > 
 > type(my_array) is array
 > False
 > 
 > Can someone explain this?
 > 
 > Cheers,
 > 	Andrea.
 > 
 > On 11 Mar 2004, at 13:42, Paulo J. S. Silva wrote:
 > 
 > > Hi,
 > >
 > > Funny, I have just entered the list and I can answer this one!
 > >
 > > Try
 > >
 > > type(my_array) is arraytype
 > >
 > > That worked to me.
 > >
 > > Best,
 > >
 > > Paulo
 > > --=20
 > > Paulo Jos=E9 da Silva e Silva
 > > Professor Assistente do Dep. de Ci=EAncia da Computa=E7=E3o
 > > (Assistant Professor of the Computer Science Dept.)
 > > Universidade de S=E3o Paulo - Brazil
 > >
 > > e-mail: rs...@im...          Web: http://www.ime.usp.br/~rsilva
 > >
 > > Teoria =E9 o que n=E3o entendemos o    (Theory is something we don't)
 > > suficiente para chamar de pr=E1tica. (understand well enough to call)
 > >                                    (practice)
 > >
 > >
 > > -------------------------------------------------------
 > > This SF.Net email is sponsored by: IBM Linux Tutorials
 > > Free Linux tutorial presented by Daniel Robbins, President and CEO of
 > > GenToo technologies. Learn everything from fundamentals to system
 > > administration.http://ads.osdn.com/?ad_id=1470&alloc_id638&op=3Dclick
 > > _______________________________________________
 > > Numpy-discussion mailing list
 > > Num...@li...
 > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
 > >
 > >
 > ---
 > Andrea Riciputi
 > 
 > "Science is like sex: sometimes something useful comes out,
 >    but that is not the reason we are doing it" -- (Richard Feynman)
 > 
 > 
 > 
 > -------------------------------------------------------
 > This SF.Net email is sponsored by: IBM Linux Tutorials
 > Free Linux tutorial presented by Daniel Robbins, President and CEO of
 > GenToo technologies. Learn everything from fundamentals to system
 > administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
 > _______________________________________________
 > Numpy-discussion mailing list
 > Num...@li...
 > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
 > 
 > 

Re: [Numpy-discussion] A simple question about type().

[Paulo J. S. S. <rs...@im...>]

This is a more tricky question.

If I understand well enough. The name "array" in a python section
relates to a factory function that builds arraytype objects. It is not
the name for the array type. For example, if you type:

type(array)

you get

<type 'builtin_function_or_method'>

Which shows that it is a function, not a class object.

There are more subtlety associated to the fact that Numeric does not use
the new class styles as numarray does. Actually if you use numarray
instead of Numeric you'll get a more sensible behavior

>>> import numarray
>>> a =3D numarray.array([1.,2.,3.])
>>> type(a)
<class 'numarray.numarraycore.NumArray'>
>>> type(a) is numarray.NumArray
True
>>> isinstance(a, numarray.NumArray)
True
>>>

Best regards,

Paulo

Obs: You may consider using a "try: except:" statement instead of always
checking type. In Python is usually better to ask "forgiveness" in a
"except" clause than to ask "permission" by checking types.

--=20
Paulo Jos=E9 da Silva e Silva=20
Professor Assistente do Dep. de Ci=EAncia da Computa=E7=E3o
(Assistant Professor of the Computer Science Dept.)
Universidade de S=E3o Paulo - Brazil

e-mail: rs...@im...          Web: http://www.ime.usp.br/~rsilva

Teoria =E9 o que n=E3o entendemos o    (Theory is something we don't)
suficiente para chamar de pr=E1tica. (understand well enough to call)=20
                                   (practice)

Re: [Numpy-discussion] A simple question about type().

[Alexandre <Ale...@lo...>]

On Thu, Mar 11, 2004 at 02:25:53PM +0100, Andrea Riciputi wrote:
> You are right! But I can't understand why. When I try type() with an=20
> array I get:
>=20
> type(my_array)
> <type 'array'>
>=20
> but:
>=20
> type(my_array) is array
> False
>=20
> Can someone explain this?

array is a function. try "type(array)", and print(array) to check.=20

--=20
Alexandre Fayolle                              LOGILAB, Paris (France).
http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org

Re: [Numpy-discussion] A simple question about type().

[Andrea R. <ari...@pi...>]

You are right! But I can't understand why. When I try type() with an=20
array I get:

type(my_array)
<type 'array'>

but:

type(my_array) is array
False

Can someone explain this?

Cheers,
	Andrea.

On 11 Mar 2004, at 13:42, Paulo J. S. Silva wrote:

> Hi,
>
> Funny, I have just entered the list and I can answer this one!
>
> Try
>
> type(my_array) is arraytype
>
> That worked to me.
>
> Best,
>
> Paulo
> --=20
> Paulo Jos=E9 da Silva e Silva
> Professor Assistente do Dep. de Ci=EAncia da Computa=E7=E3o
> (Assistant Professor of the Computer Science Dept.)
> Universidade de S=E3o Paulo - Brazil
>
> e-mail: rs...@im...          Web: http://www.ime.usp.br/~rsilva
>
> Teoria =E9 o que n=E3o entendemos o    (Theory is something we don't)
> suficiente para chamar de pr=E1tica. (understand well enough to call)
>                                    (practice)
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id638&op=3Dclick
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
---
Andrea Riciputi

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)

Re: [Numpy-discussion] A simple question about type().

[Paulo J. S. S. <rs...@im...>]

Hi,

Funny, I have just entered the list and I can answer this one!

Try

type(my_array) is arraytype

That worked to me.

Best,

Paulo
--=20
Paulo Jos=E9 da Silva e Silva=20
Professor Assistente do Dep. de Ci=EAncia da Computa=E7=E3o
(Assistant Professor of the Computer Science Dept.)
Universidade de S=E3o Paulo - Brazil

e-mail: rs...@im...          Web: http://www.ime.usp.br/~rsilva

Teoria =E9 o que n=E3o entendemos o    (Theory is something we don't)
suficiente para chamar de pr=E1tica. (understand well enough to call)=20
                                   (practice)

[Numpy-discussion] A simple question about type().

[Andrea R. <ari...@pi...>]

Hi,
it is surely a silly question, but I'm quite new to Python and Numpy. 
So I hope you can help me, I've searched Numpy's manual without 
successs.

Suppose I want to test if a paremeter inside a function is an array or 
not. How can I test this?

type(my_array) is array

returns 'False'!!

Thanks in advance,
	Andrea.
---
Andrea Riciputi

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)

Re: [Numpy-discussion] numarray: shape of scalar

[Alexander S. <a.s...@gm...>]

Sebastian Haase <ha...@ms...> writes:

> Hi,
> Is this intended:
> >>> na.array([1,2,4]).shape
> (3,)
> >>> na.array([1,2]).shape
> (2,)
> >>> na.array([1]).shape
> (1,)
> >>> na.array(1).shape
> ()
> 
> Why is na.array([1]).shape not equal to na.array(1).shape ?

It is maybe a bit like n^0 == 1 (rather than 0, which might seem "more
intuitive").

A scalar is not the same as vector of 1 element and neither are the same as a
matrix of 1 element (nonwithstanding what matlab and maybe some dumb math
texts might suggest).

If you pretend they are you run into all sorts of trouble, as matlab (which
doesn't distinguish properly between scalars, vectors and matrices) indeed
does, for one thing because various useful identities suddenly no longer hold
for corner cases (e.g. since functions like diag that do different things
depending on whether they "think" something is a vector or not just do the
wrong thing on what you intend to be a 1xN or Nx1 matrix).

Here is a not particularly interesting identity that shows why you would like
shape(scalar) to be ():

  rank == len(shape)
  rank([x]) == rank(x) + 1

Here is a more interesting example that shows why you want to distinguish
between arrays of different rank (even if their shape only differes by a
leading 1 and the flat is the same):

  rank(a[integer]) = rank(a) - 1

The pleasant indexing facilities of numarray wouldn't be possible without
distinguishing between arrays which differ only by the amount of leading 1s in
their shape.

'as

Re: [Numpy-discussion] numarray: shape of scalar

[Todd M. <jm...@st...>]

On Tue, 2004-03-09 at 13:00, Sebastian Haase wrote:
> Hi,
> Is this intended:

Yes.

> >>> na.array([1,2,4]).shape
> (3,)
> >>> na.array([1,2]).shape
> (2,)
> >>> na.array([1]).shape
> (1,)
> >>> na.array(1).shape
> ()
> 
> Why is na.array([1]).shape not equal to na.array(1).shape ?

na.array(1) is how you say "make a rank-0 array with contents 1".  There
are places where it is natural and useful in the *implementation* of
numarray.

> Did I miss the respective section in the manual ?

rank-0 arrays are a dark corner of numarray which is mostly hidden.  I
don't think they're documented anywhere.


Regards,
Todd

-- 
Todd Miller <jm...@st...>

[Numpy-discussion] numarray: shape of scalar

[Sebastian H. <ha...@ms...>]

Hi,
Is this intended:
>>> na.array([1,2,4]).shape
(3,)
>>> na.array([1,2]).shape
(2,)
>>> na.array([1]).shape
(1,)
>>> na.array(1).shape
()

Why is na.array([1]).shape not equal to na.array(1).shape ?
Did I miss the respective section in the manual ?

Thanks,
Sebastian Haase

[Numpy-discussion] [ANN] PyTables 0.8

[Francesc A. <fa...@py...>]

I'm happy to announce the availability of PyTables 0.8.

PyTables is a hierarchical database package designed to efficiently
manage very large amounts of data. PyTables is built on top of the
HDF5 library and the numarray package. It features an object-oriented
interface that, combined with natural naming and C-code generated from
Pyrex sources, makes it a fast, yet extremely easy-to-use tool for
interactively saving and retrieving very large amounts of data.  It also
provides flexible indexed access on disk to anywhere in the data.

PyTables is not designed to work as a relational database competitor,
but rather as a teammate. If you want to work with large datasets of
multidimensional data (for example, for multidimensional analysis), or
just provide a categorized structure for some portions of your cluttered
RDBS, then give PyTables a try. It works well for storing data from data
acquisition systems (DAS), simulation software, network data monitoring
systems (for example, traffic measurements of IP packets on routers),
working with very large XML files or as a centralized repository for
system logs, to name only a few possible uses.
=20
In this release you will find:
	- Variable Length Arrays (VLA's) for saving a collection
          of variable length of elements in each row of an array.
	- Extensible Arrays (EA's) for extending homogeneous
       	  datasets on disk.
	- Powerful replication capabilities, ranging from single leaves
	  up to complete hierarchies.
	- With the introduction of the UnImplemented class, greatly=20
	  improved HDF5 native file import capabilities.
	- Two new useful utilities: ptdump & ptrepack.
	- Improved documentation (with the help of Scott Prater).
        - New record on data size achieved: 5.6 TB (!) in one single
          file.
	- Enhanced platform support. New platforms: MacOSX, FreeBSD,
	  Linux64, IRIX64 (yes, a clean 64-bit port is there) and
	  probably more.
	- More tests units (now exceeding 800).
	- Many other minor improvements.

More in detail:

What's new
=2D----------

	- The new VLArray class enables you to store large lists of rows=20
	  containing variable numbers of elements. The elements can=20
	  be scalars or fully multimensional objects, in the PyTables=20
	  tradition. This class supports two special objects as rows:=20
	  Unicode strings (UTF-8 codification is used internally) and=20
	  generic Python objects (through the use of cPickle).

	- The new EArray class allows you to enlarge already existing
	  multidimensional homogeneous data objects. Consider it
	  an extension of the already existing Array class, but=20
	  with more functionality. Online compression or other filters=20
	  can be applied to EArray instances, for example.

	  Another nice feature of EA's is their support for fully
	  multidimensional data selection with extended slices.  You
	  can write "earray[1,2:3,...,4:200]", for example, to get the
	  desired dataset slice from the disk. This is implemented
	  using the powerful selection capabilities of the HDF5
	  library, which results in very highly efficient I/O
	  operations. The same functionality has been added to Array
	  objects as well.

	- New UnImplemented class. If a dataset contains unsupported
	  datatypes, it will be associated with an UnImplemented
	  instance, then inserted into to the object tree as usual.
	  This allows you to continue to work with supported objects
	  while retaining access to attributes of unsupported
	  datasets.  This has changed from previous versions, where a
	  RuntimeError occurred when an unsupported object was
	  encountered.

	  The combination of the new UnImplemented class with the=20
	  support for new datatypes will enable PyTables to greatly=20
	  increase the number of types of native HDF5 files that can
	  be read and modified.

	- Boolean support has been added for all the Leaf objects.

	- The Table class has now an append() method that allows you
	  to save large buffers of data in one go (i.e. bypassing the
	  Row accessor). This can greatly improve data gathering
	  speed.

	- The standard HDF5 shuffle filter (to further enhance the
          compression level) is supported.

	- The standard HDF5 fletcher32 checksum filter is supported.

	- As the supported number of filters is growing (and may be
          further increased in the future), a Filters() class has been
          introduced to handle filters more easily.  In order to add
          support for this class, it was necessary to make a change in
          the createTable() method that is not backwards compatible:
          the "compress" and "complib" parameters are deprecated now
          and the "filters" parameter should be used in their
          place. You will be able to continue using the old parameters
          (only a Deprecation warning will be issued) for the next few
          releases, but you should migrate to the new version as soon
          as possible. In general, you can easily migrate old code by
          substituting code in its place:

 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0table =3D fileh.createTable(g=
roup, 'table', Test, '',
 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0complevel, complib)
	 =A0should be replaced by

 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0table =3D fileh.createTable(g=
roup, 'table', Test, '',
 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0Filters(complevel, compl=
ib))


	- A copy() method that supports slicing and modification of
 =A0=A0=A0=A0=A0=A0=A0=A0=A0filtering capabilities has been added for all t=
he Leaf
 =A0=A0=A0=A0=A0=A0=A0=A0=A0objects. See the User's Manual for more informa=
tion.

	- A couple of new methods, namely copyFile() and copyChilds(),
 =A0=A0=A0=A0=A0=A0=A0=A0=A0have been added to File class, to permit easy r=
eplication
 =A0=A0=A0=A0=A0=A0=A0=A0=A0of complete hierarchies or sub-hierarchies, eve=
n to
 =A0=A0=A0=A0=A0=A0=A0=A0=A0other files. You can change filters during the =
copy
 =A0=A0=A0=A0=A0=A0=A0=A0=A0process as well.

	- Two new utilities has been added: ptdump and
 =A0=A0=A0=A0=A0=A0=A0=A0=A0ptrepack. The utility ptdump allows the user to=
 examine=20
          the=A0contents of PyTables files (both metadata and actual
 =A0=A0=A0=A0=A0=A0=A0=A0=A0data). The powerful ptrepack utility lets you=20
 =A0=A0=A0=A0=A0=A0=A0=A0=A0selectively copy (portions of) hierarchies to s=
pecific
 =A0=A0=A0=A0=A0=A0=A0=A0=A0locations in other files. It can be also used a=
s an
 =A0=A0=A0=A0=A0=A0=A0=A0=A0importer for generic HDF5 files.

 =A0=A0=A0=A0=A0=A0=A0- The meaning of the stop parameter in read() methods=
 has
 =A0=A0=A0=A0=A0=A0=A0=A0=A0changed. Now a value of 'None' means the last r=
ow, and a
 =A0=A0=A0=A0=A0=A0=A0=A0=A0value of 0 (zero) means the first row. This is =
more
 =A0=A0=A0=A0=A0=A0=A0=A0=A0consistent with the range() function in python =
and the
 =A0=A0=A0=A0=A0=A0=A0=A0=A0__getitem__() special method in numarray.

	- The method Table.removeRows() is no longer limited by table=20
	  size.  You can now delete rows regardless of the size of the=20
	  table.

	- The "numarray" value has been added to the flavor parameter
 =A0=A0=A0=A0=A0=A0=A0=A0=A0in the Table.read() method for completeness.

	- The attributes (.attr instance variable) are Python
 =A0=A0=A0=A0=A0=A0=A0=A0=A0properties now. Access to their values is no lo=
nger
 =A0=A0=A0=A0=A0=A0=A0=A0=A0lazy, i.e. you will be able to see both system =
or user
 =A0=A0=A0=A0=A0=A0=A0=A0=A0attributes from the command line using the tab-=
completion
 =A0=A0=A0=A0=A0=A0=A0=A0=A0capability of your python console (if enabled).

	- Documentation has been greatly improved to explain all the
 =A0=A0=A0=A0=A0=A0=A0=A0=A0new functionality. In particular, the internal =
format of
 =A0=A0=A0=A0=A0=A0=A0=A0=A0PyTables is now fully described. You can now bu=
ild
 =A0=A0=A0=A0=A0=A0=A0=A0=A0"native" PyTables files using any generic HDF5=
=A0software=20
          by just duplicating their format.

	- Many new tests have been added, not only to check new
 =A0=A0=A0=A0=A0=A0=A0=A0=A0functionality but also to more stringently chec=
k=20
 =A0=A0=A0=A0=A0=A0=A0=A0=A0existing functionality. There are more than 800=
 different
 =A0=A0=A0=A0=A0=A0=A0=A0=A0tests now (and the number is increasing :).

        - PyTables has a new record in the data size that fits in one
          single file: more than 5 TB (yeah, more than 5000 GB), that
          accounts for 11 GB compressed, has been created on an AMD
          Opteron machine running Linux-64 (the 64 bits version of the
          Linux kernel). See the gory details in:
          http://pytables.sf.net/html/HowFast.html.

	- New platforms supported: PyTables has been compiled and tested
	  under Linux32 (Intel), Linux64 (AMD Opteron and Alpha), Win32
	  (Intel), MacOSX (PowerPC), FreeBSD (Intel), Solaris (6, 7, 8
	  and 9 with UltraSparc), IRIX64 (IRIX 6.5 with R12000) and it
	  probably works in many more architectures. In particular,
	  release 0.8 is the first one that provides a relatively clean
	  porting to 64-bit platforms.

	- As always, some bugs have been solved (especially bugs that
 =A0=A0=A0=A0=A0=A0=A0=A0=A0occur when deleting and/or overwriting attribut=
es).

	- And last, but definitely not least, a new donations section
	  has been=A0added to the PyTables web site
	  (http://sourceforge.net/projects/pytables, then follow the
	  "Donations" tag). If you like PyTables and want this effort
	  to continue, please, donate!


What is a table?
=2D---------------

A table is defined as a collection of records whose values are stored
in fixed-length fields. All records have the same structure and all
values in each field have the same data type. =A0The terms
"fixed-length" and "strict data types" seem to be quite a strange
requirement for an language like Python that supports dynamic data
types, but they serve a useful function if the goal is to save very
large quantities of data (such as is generated by many scientific
applications, for example) in an efficient manner that reduces demand
on CPU time and I/O resources.

What is HDF5?
=2D------------

=46or those people who know nothing about HDF5, it is is a general
purpose library and file format for storing scientific data made at
NCSA. HDF5 can store two primary objects: datasets and groups. A
dataset is essentially a multidimensional array of data elements, and
a group is a structure for organizing objects in an HDF5 file. Using
these two basic constructs, one can create and store almost any kind of
scientific data structure, such as images, arrays of vectors, and
structured and unstructured grids. You can also mix and match them in
HDF5 files according to your needs.

Platforms
=2D--------

I'm using Linux (Intel 32-bit) as the main development platform, but
PyTables should be easy to compile/install on many other UNIX
machines. This package has also passed all the tests on a UltraSparc
platform with Solaris 7 and Solaris 8. It also compiles and passes all
the tests on a SGI Origin2000 with MIPS R12000 processors, with the
MIPSPro compiler and running IRIX 6.5. It also runs fine on Linux 64-bit
platforms, like an AMD Opteron running SuSe Linux Enterprise Server. It
has also been tested in MacOSX platforms (10.2 but should also work on
newer versions).

Regarding Windows platforms, PyTables has been tested with Windows
2000 and Windows XP (using the Microsoft Visual C compiler), but it
should also work with other flavors as well.

An example?
=2D----------

=46or online code examples, have a look at

http://pytables.sourceforge.net/html/tut/tutorial1-1.html


and, for newly introduced Variable Length Arrays:

http://pytables.sourceforge.net/html/tut/vlarray2.html


Web site
=2D-------

Go to the PyTables web site for more details:

http://pytables.sourceforge.net/


Share your experience
=2D--------------------

Let me know of any bugs, suggestions, gripes, kudos, etc. you may
have.

Have fun!

=2D- Francesc Alted
fa...@py...

=2D-=20
=46rancesc Alted

[Numpy-discussion] [Ann] Discretizing with Numarray

[Magnus L. H. <ma...@he...>]

I've just written a small tutorial on discretizing time series with
numarray:

  http://hetland.org/python/disc-with-numarray

Just some basic stuff I've used when working with time series in my
research, such as mapping integer values to symbols, mapping real
values to integers using bins, and extracting features using
non-overlapping or overlapping windows (sliding window).

It was written in a flash, and I'm no numarray expert, so
feedback/corrections would be welcome.

-- 
Magnus Lie Hetland           "The mind is not a vessel to be filled,
http://hetland.org            but a fire to be lighted."  [Plutarch]

Re: [Numpy-discussion] Re: Garbage collection fails after numarray exception

[Todd M. <jm...@st...>]

> BTW Todd, if you are reading this: you said in your earlier
> response that you logged this bug on SourceForge somewhere, but I
> can't seem to find it in what I think is the numarray bug page,
> at: <http://sourceforge.net/tracker/?group_id=1369&atid=101369>.
> Am I looking in the wrong place?

Look here:

http://sourceforge.net/tracker/index.php?func=detail&aid=894892&group_id=1369&atid=450446

Alternately, look for:

[ 894892 ] Memory leak after ufunc exception

in the tracker:

Numarray Bugs 

Note that "Numarray Bugs" is distinct from the easier to find "Bugs"
which is where Numeric bugs go.

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] Re: Garbage collection fails after numarray exception

[Todd M. <jm...@st...>]

On Sun, 2004-02-29 at 04:28, Leo Breebaart wrote:
> Rob Hooft <ro...@ho...> writes:
> 
> > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote:
> > > 
> > >>   >>> import numarray
> > >>   >>> a = numarray.zeros(100000000)
> > >>   >>> del a
> > >>   >>> a = numarray.zeros(100000000)
> > >>   >>> a + ''
> > >>   Traceback (most recent call last):
> > >>     File "<stdin>", line 1, in ?
> > >>     File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__
> > >>       return ufunc.add(self, operand)
> > >>     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2
> > >>       (in1, in2), insig, scalar = _inputcheck(n1, n2)
> > >>     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck    raise TypeError(
> > >>   TypeError: UFunc arguments must be numarray, scalars or numeric sequences
> > >>   >>> del a
> > >>   >>>
> > >>
> > >> If I execute these statements alongside 'top' or another
> > >> memory monitor, I of course see a big memory increase after
> > >> each call to 'zeros', as well as a big decrease after the
> > >> first 'del a' -- but no change whatsoever any more after the
> > >> second 'del a', not even if I explicitly call the garbage
> > >> collector via the gc module. [...]
> > 
> > Hmm.... Isn't this the standard python thing that it keeps the
> > context of the last exception somewhere for queries? Just

This is a good point, something I rediscovered for myself a couple weeks
ago:  since Python holds onto the context at the point of an exception, 
it will keep arrays alive in the exception context.

> > generate another exception and you should see that the memory
> > taken by a is liberated because the indirect reference from
> > sys.exc_traceback to a disappears.
> 
> Well, I've tried to test this, but even after multiple subsequent
> exceptions the memory taken up by the numarrays is not freed, at
> least not on my machine and with Python 2.3.3. Does it work for
> you if you try running the above code and then generating an
> exception?

But... I looked into this some more and found that there were errors in
the exception handling for ufuncs which resulted in leaked memory.

Regards,
Todd

-- 
Todd Miller <jm...@st...>