numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] RecArray.tolist() suggestion

[Francesc A. <fa...@py...>]

Hi,

As Perry said not too long ago that numarray crew would ask for suggestions
for RecArray improvements, I'm going to suggest a couple.

I find quite inconvenient the .tolist() method when applied to RecArray
objects as it is now:

>>> r[2:4]
array(
[(3, 33.0, 'c'),
(4, 44.0, 'd')],
formats=['1UInt8', '1Float32', '1a1'],
shape=2,
names=['c1', 'c2', 'c3'])
>>> r[2:4].tolist()
[<numarray.records.Record instance at 0x406a946c>, <numarray.records.Record instance at 0x406a912c>]


The suggested behaviour would be:

>>> r[2:4].tolist()
[(3, 33.0, 'c'),(4, 44.0, 'd')]

Another thing is that an element of recarray would be returned as a tuple
instead as a records.Record object:

>>> r[2]
<numarray.records.Record instance at 0x4064074c>

The suggested behaviour would be:

>>> r[2]
(3, 33.0, 'c')

I think the latter would be consistent with the convention that a
__getitem__(int) of a NumArray object returns a python type instead of a
rank-0 array. In the same way, a __getitem__(int) of a RecArray should
return a a python type (a tuple in this case).

Below is the code that I use right now to simulate this behaviour, but it
would be nice if the code would included into numarray.records module.

    def tolist(arr):
        """Converts a RecArray or Record to a list of rows"""
        outlist = []
        if isinstance(arr, records.Record):
            for i in range(arr.array._nfields):
                outlist.append(arr.array.field(i)[arr.row])
            outlist = tuple(outlist)  # return a tuple for records
        elif isinstance(arr, records.RecArray):
            for j in range(arr.nelements()):
                tmplist = []
                for i in range(arr._nfields):
                    tmplist.append(arr.field(i)[j])
                outlist.append(tuple(tmplist))
        return outlist


Cheers,

-- 
Francesc Alted

Re: [Numpy-discussion] How to read data from text files fast?

[Chris B. <Chr...@no...>]

Chris Barker wrote:

>> can't
>> you just preallocate the array and read your data directly into it?
> 
> The short answer is that I'm not very smart! The longer answer is that 
> this is because at first I misunderstood what PyArray_FromDimsAndData 
> was for. For ScanFileN, I'll re-do it as you suggest.

I've re-done it. Now I don't double allocate storage for ScanFileN. 
There was no noticeable difference in performance, but why use memory 
you don't have to?

For ScanFile, it is unknown at the beginning how big the final array is, 
so I now have two versions. One is what I had before, it allocates 
memory in blocks of some Buffersize as it reads the file (now set to 
1024 elements). Once it's all read in, it creates an appropriate size 
PyArray, and copies the data to it. This results in a double copy of all 
the data until the temporary memory is freed.

I now also have a ScanFile2, which scans the whole file first, then 
creates a PyArray, and re-reads the file to fill it up. This version 
takes about twice as long, confirming my expectation that the time to 
allocate and copy data is tiny compared to reading and parsing the file.

Here's a simple benchmark:

Reading with Standard Python methods
(62936, 2)
it took 2.824013 seconds to read the file with standard Python methods
Reading with FileScan
(62936, 2)
it took 0.400936 seconds to read the file with FileScan
Reading with FileScan2
(62936, 2)
it took 0.752649 seconds to read the file with FileScan2
Reading with FileScanN
(62936, 2)
it took 0.441714 seconds to read the file with FileScanN

So it takes twice as long to count the numbers first, but it's still 
three times as fast as just doing all this with Python. However, I 
usually don't think it's worth all this effort for a 3 times speed up, 
and I tend to make copies my arrays all over the place with NumPy 
anyway, so I'm inclined to stick with the first method. Also, if you are 
really that tight on memory, you could always read it in chunks with 
ScanFileN.

Any feedback anyone wants to give is very welcome.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] How to read data from text files fast?

[Chris B. <Chr...@no...>]

Todd Miller wrote:

> I looked this over to see how hard it would be to port to numarray.  At
> first glance,  it looks easy.  I didn't really read it closely enough to
> pick up bugs, but what I saw looks good.  One thing I did notice was a
> calloc of temporary data space.  That seemed like a possible waste:  can't
> you just preallocate the array and read your data directly into it?

The short answer is that I'm not very smart! The longer answer is that 
this is because at first I misunderstood what PyArray_FromDimsAndData 
was for. For ScanFileN, I'll re-do it as you suggest.

For ScanFile, it is unknown at the beginning how big the final array is, 
and I did scheme that would allocate the memory as it went, in 
reasonable sized chunks. However, this does require a full copy, which 
is a problem. Since posting, I thought of a MUCH easier scheme:

scan the file, without storing the data, to see how many numbers there are.

rewind the file

allocate the Array

Read the data.

This requires scanning the file twice, which would cost, but would be 
easier, and prevent an unnecessary copy of the data. I hope I"ll get a 
change to try it out and see what the performance is like. IN the 
meantime, anyone else have any thoughts?

By the way, does it matter whether I use malloc or calloc? I can't 
really tell the difference from K&R.

> This is
> probably a very minor speed issue,  but might be a significant storage issue
> as people are starting to max out 32-bit systems.

yup. This is all pointless if it's not a lot of data, after all.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] How to read data from text files fast?

[Chris B. <Chr...@no...>]

Fer...@co... wrote:
\> Just a quick note Travis sent to me privately: he suggested using
> io.numpyio.fread instead of Numeric.fromstring() for speed reasons.  I don't
> know if it will help in your case, I just mention it in case it helps.

Thanks, but those are for binary files, which I have to do sometimes, so 
I'll keep it in mind. However, my problem at hand is text files, and my 
solution is working nicely, though I'd love a pair of more experienced 
eyes on the code....

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] How to read data from text files fast?

[<Fer...@co...>]

Quoting Chris Barker <Chr...@no...>:

> Thanks to Fernando Perez  and Travis Oliphant for pointing me to:
>
> > scipy.io.read_array
>
> In testing, I've found that it's very slow (for my needs), though quite
> nifty in other ways, so I'm sure I'll find a use for it in the future.

Just a quick note Travis sent to me privately: he suggested using
io.numpyio.fread instead of Numeric.fromstring() for speed reasons.  I don't
know if it will help in your case, I just mention it in case it helps.

Cheers,

F

Re: [Numpy-discussion] "buffer not aligned on 8 byte boundary" errors when running numarray.testall.test()

[Todd M. <jm...@st...>]

On Thu, 2004-07-08 at 13:28, Wendy Langer wrote: 
> Hi there all :)
> 
> I am having trouble with my installation of numarray.  :(
> 
> I am a python newbie and a numarray extreme-newbie, so it could be that I
> don't yet have the first clue what I am doing. ;)
> 
> 
> 
> Python 2.3.3 (#51, Feb 16 2004, 04:07:52) [MSC v.1200 32 bit (Intel)] on
> win32
> numarray 1.0
> 
> 
> The Python I am using is the one that comes with the "Enthought" version
> (www.enthought.com), a distro specifically designed to be useful for
> scientists, so it comes with numerical stuff and scipy and chaco and things
> like that preinstalled.
> 
> I used the windows binary installer. However it came with Numeric and not
> numarray, so I installed numarray "by hand". This seemed to go ok, and it
> seems that there is no problem having both Numeric and numarray in the same
> installation, since they have (obviously) different names (still getting
> used to this whole modules and namespaces &c &c)

I don't normally use SciPy,  but I normally have both numarray and
Numeric installed so there's no inherent conflict there.

> At the bottom of this email I have pasted an example of what it was I was
> trying to do, and the error messages that the interpreter gave me - but
> before anyone bothers reading them in any detail, the essential error seems
> to be as follows:
> 
> error: multiply_Float64_scalar_vector: buffer not aligned on 8 byte
> boundary.

This is a low level exception triggered by a misaligned data buffer. 
It's low level so it's impossible to tell what the real problem is
without more information.

> I have no idea what this means, but I do recall that when I ran the
> numarray.testall.test() procedure after first completing my installation a
> couple of days ago, it  reported a *lot* of problems, many of which sounded
> quite similar to this.

That sounds pretty bad.  Here's roughly how it should look these days:
% python
>>> import numarray.testall as testall
>>> testall.test()
numarray:                               ((0, 1165), (0, 1165))
numarray.records:                       (0, 48)
numarray.strings:                       (0, 176)
numarray.memmap:                        (0, 82)
numarray.objects:                       (0, 105)
numarray.memorytest:                    (0, 16)
numarray.examples.convolve:             ((0, 20), (0, 20), (0, 20), (0,
20))
numarray.convolve:                      (0, 52)
numarray.fft:                           (0, 75)
numarray.linear_algebra:                ((0, 46), (0, 51))
numarray.image:                         (0, 27)
numarray.nd_image:                      (0, 390)
numarray.random_array:                  (0, 53)
numarray.ma:                            (0, 671)

The tuple results for your test should all have leading zeros as above. 
The number of tests varies from release to release.


> I hoped for the best and  thought that perhaps I had "run the test wrong"(!)
> since numarray seemed to be working ok, and I had investigated many of the
> examples in chapters 3 and 4 of the user manual withour any obvious problems
> (chapter 3 = "high level overview" and chapter 4 = "array basics")
> 
> I decided at the time to leave well enough alone until I actually came
> across odd or mysterious behaviour ...however that time has come
> all-too-soon...
> 
> 
> 
> 
> The procedure I am using to run the test is as described on page 11 of the
> excellent user's manual (release 0.8 at
> http://www.pfdubois.com/numpy/numarray.pdf):

There's an updated manual here:
http://prdownloads.sourceforge.net/numpy/numarray-1.0.pdf?download

> --
> Testing your Installation
> Once you have installed numarray, test it with:
> C:\numarray> python
> Python 2.2.2 (#18, Dec 30 2002, 02:26:03) [MSC 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license" for more information.
> >>> import numarray.testall as testall
> >>> testall.test()
> numeric: (0, 1115)
> records: (0, 48)
> strings: (0, 166)
> objects: (0, 72)
> memmap: (0, 75)
> Each line in the above output indicates that 0 of X tests failed. X grows
> steadily with each release, so the numbers
> shown above may not be current.
> --
> 
> Anyway, when I ran this, instead of the nice, comforting output above,  I
> got about a million(!) errors and then a final count of 320 failures. This
> number is not always constant - I recall the first time I ran it it was 209.
> [I just ran it again and this time it was 324...it all has a rather
> disturbing air of semi-randomness...]
> 
> 
> So below is the (heavily snipped) output from the testall.test() run, and
> below that is the code where I first noticed a possibly similar error, and
> below *that*  is the output of that code, with the highly suspicous
> error....
> 
> 
> Any suggestions greatly appreciated!

If you've ever had numarray installed before,  go to your site-packages
directory and delete numarray as well as any numarray.pth.  Then
reinstall numarray-1.0.

Also,  just do:

>>> import numarray
>>> numarray

and see what kind of path is involved getting to the numarray module.

> I can give you more info about the setup on my computer and so on if you
> need :)

I think you already included everything important;  the exact variant of
Windows you're using might be helpful;  I'm not aware of any problems
there though.  

It looks like you're on a well supported platform.  I just tested pretty
much the same configuration on Windows 2000 Pro, with Python-2.3.4, and
it worked fine even with SciPy-0.3.

> wendy langer
> 
> 
> ======================================================================
> <output when I ran numarray.testall.test() >
> 

<SNIP>

There's something hugely wrong with your test output.  I've never seen
anything like it other than during development.

> </output when I ran numarray.testall.test() >
> 
> =========================================================================
> <my code>
> ========================
> 
> 
> import numarray
> 
> class anXmatrix:
>     def __init__(self, stepsize = 3):
>         self.stepsize = stepsize
>         self.populate_matrix()
> 
> 
>     def describe(self):
>         print "I am a ", self.__class__
>         print "my stepsize is", self.stepsize
>         print "my matrix is: \n"
>         print self.matrix
> 
>     def populate_matrix(self):
> 
>         def xvalues(i,j):
>             return self.stepsize*j
> 
>         mx = numarray.fromfunction(xvalues, (4,4))
>         self.matrix = mx
> 
> 
> if __name__ == '__main__':
> 
> 
>     print "      "
>     print "Making anXmatrix..."
>     r = anXmatrix(stepsize = 5)
>     r.describe()
>     r = anXmatrix(stepsize = 0.02)
>     r.describe()
> 
> </my code>
> 
> ============================================================================
Here's what I get when I run your code, windows or linux:

Making anXmatrix...
I am a  __main__.anXmatrix
my stepsize is 5
my matrix is:

[[ 0  5 10 15]
 [ 0  5 10 15]
 [ 0  5 10 15]
 [ 0  5 10 15]]
I am a  __main__.anXmatrix
my stepsize is 0.02
my matrix is:

[[ 0.    0.02  0.04  0.06]
 [ 0.    0.02  0.04  0.06]
 [ 0.    0.02  0.04  0.06]
 [ 0.    0.02  0.04  0.06]]

Regards,
Todd

Re: [Numpy-discussion] How to read data from text files fast?

[Chris B. <Chr...@no...>]

Thanks to Fernando Perez  and Travis Oliphant for pointing me to:

> scipy.io.read_array

In testing, I've found that it's very slow (for my needs), though quite 
nifty in other ways, so I'm sure I'll find a use for it in the future.

Travis Oliphant wrote:

 > Alternatively, we could move some of the Python code in read_array to 
 > C to improve the speed.

That was beyond me, so I wrote a very simple module in C that does what 
I want, and it is very much faster than read_array or straight python 
version. It has two functions:

FileScan(file)
"""
Reads all the values in rest of the ascii file, and produces a Numeric
vector full of Floats (C doubles).

All text in the file that is not part of a floating point number is
skipped over.
"""

FileScanN(file, N)

"""
Reads N values in the ascii file, and produces a Numeric vector of
length N full of Floats (C doubles).

Raises an exception if there are fewer than N  numbers in the file.

All text in the file that is not part of a floating point number is
skipped over.

After reading N numbers, the file is left before the next non-whitespace
character in the file. This will often leave the file at the start of
the next line, after scanning a line full of numbers.
"""

I implemented them separately, 'cause I wasn't sure how to deal with 
optional arguments in a C function. They could easily have wrapped in a 
Python function if you wanted one interface.

FileScan was much more complex, as I had to deal with all the dynamic 
memory allocation. I probably took a more complex approach to this than 
I had to, but it was an exercise for me, being a newbie at C.

I also decided not to specify a shape for the resulting array, always 
returning a rank-1 array, as that made the code easier, and you can 
always set A.shape afterward. This could be put in a Python wrapper as well.

It has the obvious limitation that it only does doubles. I'd like to add 
longs as well, but probably won't have a need for anything else. The way 
memory is these days, it seems just as easy to read the long ones, and 
convert afterward if you want.

Here is a quick benchmark (see below) run with a file that is 63,000 
lines, with two comma-delimited numbers on each line. Run on a 1GHz P4 
under Linux.

Reading with read_array
it took 16.351712 seconds to read the file with read_array
Reading with Standard Python methods
it took 2.832078 seconds to read the file with standard Python methods
Reading with FileScan
it took 0.444431 seconds to read the file with FileScan
Reading with FileScanN
it took 0.407875 seconds to read the file with FileScanN

As you can see, read_array is painfully slow for this kind of thing, 
straight Python is OK, and FileScan is pretty darn fast.

I've enclosed the C code and setup.py, if anyone wants to take a look, 
and use it, or give suggestions or bug fixes or whatever, that would be 
great.

In particular, I don't think I've structured the code very well, and 
there could be memory leak, which I have not tested carefully for.

Tested only on Linux with Python2.3.3, Numeric 23.1. If someone wants to 
  port it to numarray, that would be great too.

-Chris


The benchmark:

def test6():
     """
     Testing various IO options
     """
     from scipy.io import array_import

     filename = "JunkBig.txt"
     file = open(filename)
     print "Reading with read_array"
     start = time.time()
     A = array_import.read_array(file,",")
     print "it took %f seconds to read the file with 
read_array"%(time.time() - start)
     file.close()

     file = open(filename)
     print "Reading with Standard Python methods"
     start = time.time()
     A = []
     for line in file:
         A.append( map ( float, line.strip().split(",") ) )
     A = array(A)
     print "it took %f seconds to read the file with standard Python 
methods"%(time.time() - start)
     file.close()

     file = open(filename)
     print "Reading with FileScan"
     start = time.time()
     A = FileScanner.FileScan(file)
     A.shape = (-1,2)
     print "it took %f seconds to read the file with 
FileScan"%(time.time() - start)
     file.close()

     file = open(filename)
     print "Reading with FileScanN"
     start = time.time()
     A = FileScanner.FileScanN(file, product(A.shape) )
     A.shape = (-1,2)
     print "it took %f seconds to read the file with 
FileScanN"%(time.time() - start)

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] "buffer not aligned on 8 byte boundary" errors when running numarray.testall.test()

[Wendy L. <wl...@bi...>]

Hi there all :)

I am having trouble with my installation of numarray.  :(

I am a python newbie and a numarray extreme-newbie, so it could be that I
don't yet have the first clue what I am doing. ;)



Python 2.3.3 (#51, Feb 16 2004, 04:07:52) [MSC v.1200 32 bit (Intel)] on
win32
numarray 1.0


The Python I am using is the one that comes with the "Enthought" version
(www.enthought.com), a distro specifically designed to be useful for
scientists, so it comes with numerical stuff and scipy and chaco and things
like that preinstalled.

I used the windows binary installer. However it came with Numeric and not
numarray, so I installed numarray "by hand". This seemed to go ok, and it
seems that there is no problem having both Numeric and numarray in the same
installation, since they have (obviously) different names (still getting
used to this whole modules and namespaces &c &c)




At the bottom of this email I have pasted an example of what it was I was
trying to do, and the error messages that the interpreter gave me - but
before anyone bothers reading them in any detail, the essential error seems
to be as follows:

error: multiply_Float64_scalar_vector: buffer not aligned on 8 byte
boundary.


I have no idea what this means, but I do recall that when I ran the
numarray.testall.test() procedure after first completing my installation a
couple of days ago, it  reported a *lot* of problems, many of which sounded
quite similar to this.

I hoped for the best and  thought that perhaps I had "run the test wrong"(!)
since numarray seemed to be working ok, and I had investigated many of the
examples in chapters 3 and 4 of the user manual withour any obvious problems
(chapter 3 = "high level overview" and chapter 4 = "array basics")

I decided at the time to leave well enough alone until I actually came
across odd or mysterious behaviour ...however that time has come
all-too-soon...




The procedure I am using to run the test is as described on page 11 of the
excellent user's manual (release 0.8 at
http://www.pfdubois.com/numpy/numarray.pdf):


---------------------------------------------
Testing your Installation
Once you have installed numarray, test it with:
C:\numarray> python
Python 2.2.2 (#18, Dec 30 2002, 02:26:03) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> import numarray.testall as testall
>>> testall.test()
numeric: (0, 1115)
records: (0, 48)
strings: (0, 166)
objects: (0, 72)
memmap: (0, 75)
Each line in the above output indicates that 0 of X tests failed. X grows
steadily with each release, so the numbers
shown above may not be current.
------------------------------------------------------------------------

Anyway, when I ran this, instead of the nice, comforting output above,  I
got about a million(!) errors and then a final count of 320 failures. This
number is not always constant - I recall the first time I ran it it was 209.
[I just ran it again and this time it was 324...it all has a rather
disturbing air of semi-randomness...]


So below is the (heavily snipped) output from the testall.test() run, and
below that is the code where I first noticed a possibly similar error, and
below *that*  is the output of that code, with the highly suspicous
error....


Any suggestions greatly appreciated!

I can give you more info about the setup on my computer and so on if you
need :)

wendy langer


======================================================================
<output when I ran numarray.testall.test() >
====================================

IDLE 1.0.2      ==== No Subprocess ====
>>> import numarray.testall as testall
>>> testall.test()
*****************************************************************
Failure in example: x+y
from line #50 of first pass
Exception raised:
Traceback (most recent call last):
  File "C:\PYTHON23\lib\doctest.py", line 442, in _run_examples_inner
    compileflags, 1) in globs
  File "<string>", line 1, in ?
  File "C:\PYTHON23\Lib\site-packages\numarray\numarraycore.py", line 733,
in __add__
    return ufunc.add(self, operand)
error: Int32asFloat64: buffer not aligned on 8 byte boundary.
*****************************************************************
Failure in example: x[:] = 0.1
from line #72 of first pass
Exception raised:
Traceback (most recent call last):
  File "C:\PYTHON23\lib\doctest.py", line 442, in _run_examples_inner
    compileflags, 1) in globs
  File "<string>", line 1, in ?
error: Float64asBool: buffer not aligned on 8 byte boundary.
*****************************************************************
Failure in example: y
from line #74 of first pass
Expected: array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])
Got: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])
*****************************************************************
Failure in example: x + z
from line #141 of first pass
Exception raised:
Traceback (most recent call last):
  File "C:\PYTHON23\lib\doctest.py", line 442, in _run_examples_inner
    compileflags, 1) in globs
  File "<string>", line 1, in ?
  File "C:\PYTHON23\Lib\site-packages\numarray\numarraycore.py", line 733,
in __add__
    return ufunc.add(self, operand)
error: Int32asFloat64: buffer not aligned on 8 byte boundary.
*****************************************************************


<BIG SNIP!!!!!!!!!!!!>

<snip a lot more  errors>

*****************************************************************
Failure in example: a2dma = average(a2dm, axis=1)
from line #812 of numarray.ma.dtest
Exception raised:
Traceback (most recent call last):
  File "C:\PYTHON23\lib\doctest.py", line 442, in _run_examples_inner
    compileflags, 1) in globs
  File "<string>", line 1, in ?
  File "C:\PYTHON23\Lib\site-packages\numarray\ma\MA.py", line 1686, in
average
    w = Numeric.choose(mask, (1.0, 0.0))
  File "C:\PYTHON23\Lib\site-packages\numarray\ufunc.py", line 1666, in
choose
    return _choose(selector, population, outarr, clipmode)
  File "C:\PYTHON23\Lib\site-packages\numarray\ufunc.py", line 1573, in
__call__
    result = self._doit(computation_mode, woutarr, cfunc, ufargs, 0)
  File "C:\PYTHON23\Lib\site-packages\numarray\ufunc.py", line 1558, in
_doit
    blockingparameters)
error: choose8bytes: buffer not aligned on 8 byte boundary.
*****************************************************************
Failure in example: alltest(a2dma, [1.5, 4.0])
from line #813 of numarray.ma.dtest
Exception raised:
Traceback (most recent call last):
  File "C:\PYTHON23\lib\doctest.py", line 442, in _run_examples_inner
    compileflags, 1) in globs
  File "<string>", line 1, in ?
NameError: name 'a2dma' is not defined
*****************************************************************
1 items had failures:
 320 of 671 in numarray.ma.dtest
***Test Failed*** 320 failures.
numarray.ma:                            (320, 671)



</output when I ran numarray.testall.test() >

=========================================================================
<my code>
========================


import numarray

class anXmatrix:
    def __init__(self, stepsize = 3):
        self.stepsize = stepsize
        self.populate_matrix()


    def describe(self):
        print "I am a ", self.__class__
        print "my stepsize is", self.stepsize
        print "my matrix is: \n"
        print self.matrix

    def populate_matrix(self):

        def xvalues(i,j):
            return self.stepsize*j

        mx = numarray.fromfunction(xvalues, (4,4))
        self.matrix = mx


if __name__ == '__main__':


    print "      "
    print "Making anXmatrix..."
    r = anXmatrix(stepsize = 5)
    r.describe()
    r = anXmatrix(stepsize = 0.02)
    r.describe()

</my code>

============================================================================
========
<output from interpreter when I run my code>



Making anXmatrix...
I am a  __main__.anXmatrix
my stepsize is 5
my matrix is:

[[ 0  5 10 15]
 [ 0  5 10 15]
 [ 0  5 10 15]
 [ 0  5 10 15]]
Traceback (most recent call last):
  File
"C:\Python23\Lib\site-packages\WendyStuff\wendycode\propagatorstuff\core_obj
ects\domain_objects.py", line 97, in ?
    r = anXmatrix(stepsize = 0.02)
  File
"C:\Python23\Lib\site-packages\WendyStuff\wendycode\propagatorstuff\core_obj
ects\domain_objects.py", line 72, in __init__
    self.populate_matrix()
  File
"C:\Python23\Lib\site-packages\WendyStuff\wendycode\propagatorstuff\core_obj
ects\domain_objects.py", line 86, in populate_matrix
    mx = numarray.fromfunction(xvalues, (4,4))
  File "C:\PYTHON23\Lib\site-packages\numarray\generic.py", line 1094, in
fromfunction
    return apply(function, tuple(indices(dimensions)))
  File
"C:\Python23\Lib\site-packages\WendyStuff\wendycode\propagatorstuff\core_obj
ects\domain_objects.py", line 84, in xvalues
    return self.stepsize*j
  File "C:\PYTHON23\Lib\site-packages\numarray\numarraycore.py", line 772,
in __rmul__
    r = ufunc.multiply(operand, self)
error: multiply_Float64_scalar_vector: buffer not aligned on 8 byte
boundary.
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


</output from interpreter when I run my code>
============================================================================
========

"You see, wire telegraph is a kind of a very, very long cat. You pull his
tail in New York and his head is meowing in Los Angeles. Do you understand
this? And radio operates exactly the same way: you send signals here, they
receive them there. The only difference is that there is no cat."  Albert
Einstein

[Numpy-discussion] Re: Correction -- optional arguments to the array constructor

[Philip A. <pa...@eo...>]

Oops, note the change below at --->

Todd Miller writes:
 > 
 > OK,  I think I see what you're after and agree that it's a bug.  Here's
 > how I'll change the behavior:
 > 
 > >>> import numarray
 > >>> a = numarray.arange(10)
 > >>> b = numarray.array(a, copy=0)
 > >>> a is b
 > True
 > >>> b = numarray.array(a, copy=1)
 > >>> a is b
 > False

Just to be clear -- the above is the current numarray v1.0 behavior
(at least on my machine).  Numeric compatibility would additonally
require that

import numarray
a = numarray.arange(10)
theTypeCode=repr(a.type())
b = numarray.array(a, theTypeCode, copy=0)
print a is b
b = numarray.array(a, copy=1)
print a is b

produce

True
False

While currently it produces

--->

False
False

Having said this, I can work around this difference -- so either
a note in the documentation or just removing the copy flag from 
numarray.array would also be ok.

-- Thanks, Phil

Re: [Numpy-discussion] optional arguments to the array constructor

[Philip A. <pa...@eo...>]

Todd Miller writes:
 > 
 > OK,  I think I see what you're after and agree that it's a bug.  Here's
 > how I'll change the behavior:
 > 
 > >>> import numarray
 > >>> a = numarray.arange(10)
 > >>> b = numarray.array(a, copy=0)
 > >>> a is b
 > True
 > >>> b = numarray.array(a, copy=1)
 > >>> a is b
 > False

Just to be clear -- the above is the current numarray v1.0 behavior
(at least on my machine).  Numeric compatibility would additonally
require that

import numarray
a = numarray.arange(10)
theTypeCode=repr(a.type())
b = numarray.array(a, theTypeCode, copy=0)
print a is b
b = numarray.array(a, copy=1)
print a is b

produce

True
False

While currently it produces

True
True

Having said this, I can work around this difference -- so either
a note in the documentation or just removing the copy flag from 
numarray.array would also be ok.

-- Thanks, Phil

Re: [Numpy-discussion] optional arguments to the array constructor

[Todd M. <jm...@st...>]

On Wed, 2004-07-07 at 14:25, Philip Austin wrote:
> Todd Miller writes:
>  > On Tue, 2004-07-06 at 21:42, Philip Austin wrote:
>  > > (for numpy v1.0 on Mandrake 10 i686)
>  > 
>  > My guess is you're talking about numarray here.  Please be charitable if
>  > I'm talking out of turn... I tend to see everything as a numarray
>  > issue.
> 
> Right -- I'm still working through the boost test suite for numarray, which is
> failing a couple of tests that passed (around numarray v0.3).
> 
>  > All this looks like a documentation problem.  The numarray array()
>  > signature has been tortured by Numeric backward compatibility,  so there
>  > has been more flux in it than you would expect.  Anyway, the manual is
>  > out of date.  Here's the current signature from the code:
>  > 
>  > def array(sequence=None, typecode=None, copy=1, savespace=0,
>  >           type=None, shape=None):
>  > 
> 
> Actually, it seems to be a difference in the way that numeric and
> numarray treat the copy flag when typecode is specified.  In numeric,
> if no change in type is requested and copy=0, then the constructor
> goes ahead and produces a view:
> 
> import Numeric as nc
> test=nc.array([1,2,3],'i')
> a=nc.array(test,'i',0)
> a[0]=99
> print test
> >> [99  2  3]
> 
> but makes a copy if a cast is required:
> 
> test=nc.array([1,2,3],'i')
> a=nc.array(test,'F',0)
> a[0]=99
> print test
> >>> [1 2 3]
> 
> Looking at numarraycore.py line 305 I see that:
> 
>         if type is None and typecode is None:
>             if copy:
>                 a = sequence.copy()
>             else:
>                 a = sequence
> 
> i.e. numarray skips the check for a type match and ignores
> the copy flag, even if the type is preserved:
> 
> import numarray as ny
> test=ny.array([1,2,3],'i')
> a=ny.array(test,'i',0)
> a._data is test._data
> >>> False
> 

OK,  I think I see what you're after and agree that it's a bug.  Here's
how I'll change the behavior:

>>> import numarray
>>> a = numarray.arange(10)
>>> b = numarray.array(a, copy=0)
>>> a is b
True
>>> b = numarray.array(a, copy=1)
>>> a is b
False

One possible point of note is that array() doesn't return views for
copy=0;  neither does Numeric; both return the original sequence.

Regards,
Todd

Re: [Numpy-discussion] optional arguments to the array constructor

[Philip A. <pa...@eo...>]

Todd Miller writes:
 > On Tue, 2004-07-06 at 21:42, Philip Austin wrote:
 > > (for numpy v1.0 on Mandrake 10 i686)
 > 
 > My guess is you're talking about numarray here.  Please be charitable if
 > I'm talking out of turn... I tend to see everything as a numarray
 > issue.

Right -- I'm still working through the boost test suite for numarray, which is
failing a couple of tests that passed (around numarray v0.3).

 > All this looks like a documentation problem.  The numarray array()
 > signature has been tortured by Numeric backward compatibility,  so there
 > has been more flux in it than you would expect.  Anyway, the manual is
 > out of date.  Here's the current signature from the code:
 > 
 > def array(sequence=None, typecode=None, copy=1, savespace=0,
 >           type=None, shape=None):
 > 

Actually, it seems to be a difference in the way that numeric and
numarray treat the copy flag when typecode is specified.  In numeric,
if no change in type is requested and copy=0, then the constructor
goes ahead and produces a view:

import Numeric as nc
test=nc.array([1,2,3],'i')
a=nc.array(test,'i',0)
a[0]=99
print test
>> [99  2  3]

but makes a copy if a cast is required:

test=nc.array([1,2,3],'i')
a=nc.array(test,'F',0)
a[0]=99
print test
>>> [1 2 3]

Looking at numarraycore.py line 305 I see that:

        if type is None and typecode is None:
            if copy:
                a = sequence.copy()
            else:
                a = sequence

i.e. numarray skips the check for a type match and ignores
the copy flag, even if the type is preserved:

import numarray as ny
test=ny.array([1,2,3],'i')
a=ny.array(test,'i',0)
a._data is test._data
>>> False

It look like there might have been a comment about this
in the docstring, but it got clipped at some point?:

array() constructs a NumArray by calling NumArray, one of its
    factory functions (fromstring, fromfile, fromlist), or by making a
    copy of an existing array.  If copy=0, array() will create a new
    array only if

    sequence             specifies the contents or storage for the array

Thanks, Phil

Re: [Numpy-discussion] optional arguments to the array constructor

[Todd M. <jm...@st...>]

On Tue, 2004-07-06 at 21:42, Philip Austin wrote:
> (for numpy v1.0 on Mandrake 10 i686)

My guess is you're talking about numarray here.  Please be charitable if
I'm talking out of turn... I tend to see everything as a numarray issue.

> As noted on p. 25 the array constructor takes up to 5 optional arguments
> 
> array(sequence=None, type=None, shape=None, copy=1, savespace=0,typecode=None)
> (and raises an exception if both type and typecode are set).  
> 
> Is there any way to make an alias (copy=0) of an array without passing
> keyword values?  

In numarray,  all you have to do to get an alias is:

>>> b = a.view()

It's an alias because:

>>> b._data is a._data
True

> That is, specifying the copy keyword alone works:
> 
> test=N.array((1., 3), "Float64", shape=(2,), copy=1, savespace=0)
> a=N.array(test, copy=0)
> a[1]=999
> print test
> 
> >>> [   1.  999.]
> 
> But when intervening keywords are specified copy won't toggle:
> 
> test=N.array((1., 3))
> a=N.array(sequence=test, type="Float64", shape=(2,), copy=0)
> a[1]=999.
> print test
> >>> [ 1.  3.]
> 
> Which is also the behaviour I see when I drop the keywords:
> 
> test=N.array((1., 3))
> a=N.array(test, "Float64", (2,), 0)
> a[1]=999.
> print test
> >>> [ 1.  3.]
> 
> an additional puzzle is that adding the savespace parameter raises
> the following exception:
> 
> 
> >>> a=N.array(test, "Float64", (2,), 0,0)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 312, in array
>     type = getTypeObject(sequence, type, typecode) 
>   File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 256, in getTypeObject
>     rtype = _typeFromTypeAndTypecode(type, typecode)
>   File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 243, in _typeFromTypeAndTypecode
>     raise ValueError("Can't define both 'type' and 'typecode' for an array.")
> ValueError: Can't define both 'type' and 'typecode' for an array.

All this looks like a documentation problem.  The numarray array()
signature has been tortured by Numeric backward compatibility,  so there
has been more flux in it than you would expect.  Anyway, the manual is
out of date.  Here's the current signature from the code:

def array(sequence=None, typecode=None, copy=1, savespace=0,
          type=None, shape=None):

Sorry about the confusion,
Todd

Re: [Numpy-discussion] non-intuitive behaviour for isbyteswapped()?

[Todd M. <jm...@st...>]

On Tue, 2004-07-06 at 19:07, Philip Austin wrote:
> With numarray 1.0 and Mandrake 10 i686 I get the following:
> 
> >>> y=N.array([1,1,2,1],type="Float64")
> >>> y
> array([ 1.,  1.,  2.,  1.])
> >>> y.byteswap()
> >>> y
> array([  3.03865194e-319,   3.03865194e-319,   3.16202013e-322,
>          3.03865194e-319])
> >>> y.isbyteswapped()
> 0
>
> Should this be 1?

The behavior of byteswap() has been controversial in the past, at one
time implementing exactly the behavior I think you expected.

Without giving any guarantee for the future,  here's how things work
now:  byteswap() just swaps the bytes.  There's a related method,
togglebyteorder(), which inverts the sense of the byteorder:

	>>> y.byteswap()
	>>> y.togglebyteorder()
	>>> y.isbyteswapped()
	1

The ability to munge bytes and change the sense of byteorder
independently is definitely needed... but you're certainly not the first
one to ask this question.

There is also (Numeric compatible) byteswapped(), which both swaps and
changes sense, but it creates a copy rather than operating in place:
	
	>>> x = y.byteswapped()
	>>> (x is not y) and (x._data is not y._data)
	1

Regards,
Todd

[Numpy-discussion] optional arguments to the array constructor

[Philip A. <pa...@eo...>]

(for numpy v1.0 on Mandrake 10 i686)

As noted on p. 25 the array constructor takes up to 5 optional arguments

array(sequence=None, type=None, shape=None, copy=1, savespace=0,typecode=None)
(and raises an exception if both type and typecode are set).  

Is there any way to make an alias (copy=0) of an array without passing
keyword values?  That is, specifying the copy keyword alone works:

test=N.array((1., 3), "Float64", shape=(2,), copy=1, savespace=0)
a=N.array(test, copy=0)
a[1]=999
print test

>>> [   1.  999.]

But when intervening keywords are specified copy won't toggle:

test=N.array((1., 3))
a=N.array(sequence=test, type="Float64", shape=(2,), copy=0)
a[1]=999.
print test
>>> [ 1.  3.]

Which is also the behaviour I see when I drop the keywords:

test=N.array((1., 3))
a=N.array(test, "Float64", (2,), 0)
a[1]=999.
print test
>>> [ 1.  3.]

an additional puzzle is that adding the savespace parameter raises
the following exception:


>>> a=N.array(test, "Float64", (2,), 0,0)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 312, in array
    type = getTypeObject(sequence, type, typecode) 
  File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 256, in getTypeObject
    rtype = _typeFromTypeAndTypecode(type, typecode)
  File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 243, in _typeFromTypeAndTypecode
    raise ValueError("Can't define both 'type' and 'typecode' for an array.")
ValueError: Can't define both 'type' and 'typecode' for an array.

Thanks for any insights -- Phil

[Numpy-discussion] non-intuitive behaviour for isbyteswapped()?

[Philip A. <pa...@eo...>]

With numarray 1.0 and Mandrake 10 i686 I get the following:

>>> y=N.array([1,1,2,1],type="Float64")
>>> y
array([ 1.,  1.,  2.,  1.])
>>> y.byteswap()
>>> y
array([  3.03865194e-319,   3.03865194e-319,   3.16202013e-322,
         3.03865194e-319])
>>> y.isbyteswapped()
0

Should this be 1?

Thanks, Phil

[Numpy-discussion] Missing header_pep.txt

[Todd M. <jm...@st...>]

Somehow header_pep.txt didn't make it into the numarray-1.0 source
tar-ball.  It's now in CVS and also attached.

Regards,
Todd

Re: [Numpy-discussion] Bug in numarray.typecode()?

[Philip A. <pa...@eo...>]

Todd Miller writes:
 > > 
 > > Should this print 'U'?
 > 
 > I think it could,  but I wouldn't go so far as to say it should. 
 > typecode() is there for backward compatibility with Numeric.  Since 'U'
 > doesn't work for Numeric,  I see no point in adding it to numarray. I'm
 > not sure it would hurt anything other than create the illusion that
 > something which works on numarray will also work on Numeric.
 > 
 > If anyone has a good reason to add it,  please speak up.
 > 

I don't necessarily need typecode, but I couldn't find the
inverse of

a = array([10], type = 'UInt8') (p. 19) 

in the manual.

That is, I need a method that returns the string representation of a
numarray type in a single call (as opposed to the two-step
repr(array.type()).  This is for code that uses the Boost C++ bindings
to numarray.  These bindings work via callbacks to python (which
eliminates the need to link to the numarray or numeric api). Currently
I use typecode() to get an index into a map of types when I need to
check that the type of a passed argument is correct:

void check_type(boost::python::numeric::array arr, 
		string expected_type){
  string actual_type = arr.typecode();
  if (actual_type != expected_type) {
    std::ostringstream stream;
    stream << "expected Numeric type " << kindstrings[expected_type]
	   << ", found Numeric type " << kindstrings[actual_type] << std::ends;
    PyErr_SetString(PyExc_TypeError, stream.str().c_str());
    throw_error_already_set();
  }
  return;
}

Unless I'm missing something, without typecode I need a second
interpreter call to repr, or I need to import numarray and load all
the types into storage for a type object comparison.  It's not a
showstopper, but since I check every argument in every call, I'd like
to avoid this unless absolutely necessary.

Regards, Phil




 

Re: [Numpy-discussion] inconsistencies between docs and C headers?

[Todd M. <jm...@st...>]

On Tue, 2004-07-06 at 05:41, Curzio Basso wrote:
> Hi all,
> can someone explain me why in the docs functions like NA_NewArray() 
> return a PyObject*, while in the headers they return a PyArrayObject*? 
> Is it just the documentation which is slow to catch up with the 
> development? 

Yes,  it's a bona fide inconsistency.  It's not great,  but it's fairly 
harmless since a PyArrayObject is a PyObject.

Re: [Numpy-discussion] Bug in numarray.typecode()?

[Todd M. <jm...@st...>]

On Sat, 2004-07-03 at 13:10, Philip Austin wrote:
> I'm in the process of switching to numarray, but I still
> need typecode().  I notice that, although it's discouraged,
> the typecode ids have been extended to all new numarray
> types described in table 4.1 (p. 19) of the manual, except UInt64.
> That is, the following script:
> 
> import numarray as Na
> print "Numarray version: ",Na.__version__
> print Na.array([1],'Int8').typecode()
> print Na.array([1],'UInt8').typecode()
> print Na.array([1],'Int16').typecode()
> print Na.array([1],'UInt16').typecode()
> print Na.array([1],'Int32').typecode()
> print Na.array([1],'UInt32').typecode()
> print Na.array([1],'Float32').typecode()
> print Na.array([1],'Float64').typecode()
> print Na.array([1],'Complex32').typecode()
> print Na.array([1],'Complex64').typecode()
> print Na.array([1],'Bool').typecode()
> print Na.array([1],'UInt64').typecode()
> 
> prints:
> 
> Numarray version:  1.0
> 1
> b
> s
> w
> l
> u
> f
> d
> F
> D
> 1
> Traceback (most recent call last):
>   File "<stdin>", line 14, in ?
>   File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 1092, in typecode
>     return _nt.typecode[self._type]
> KeyError: UInt64
> 
> Should this print 'U'?

I think it could,  but I wouldn't go so far as to say it should. 
typecode() is there for backward compatibility with Numeric.  Since 'U'
doesn't work for Numeric,  I see no point in adding it to numarray. I'm
not sure it would hurt anything other than create the illusion that
something which works on numarray will also work on Numeric.

If anyone has a good reason to add it,  please speak up.

Regards,
Todd

[Numpy-discussion] inconsistencies between docs and C headers?

[Curzio B. <cur...@un...>]

Hi all,
can someone explain me why in the docs functions like NA_NewArray() 
return a PyObject*, while in the headers they return a PyArrayObject*? 
Is it just the documentation which is slow to catch up with the 
development? Or am i missing something?

thanks, curzio

[Numpy-discussion] Bug in numarray.typecode()?

[Philip A. <pa...@eo...>]

I'm in the process of switching to numarray, but I still
need typecode().  I notice that, although it's discouraged,
the typecode ids have been extended to all new numarray
types described in table 4.1 (p. 19) of the manual, except UInt64.
That is, the following script:

import numarray as Na
print "Numarray version: ",Na.__version__
print Na.array([1],'Int8').typecode()
print Na.array([1],'UInt8').typecode()
print Na.array([1],'Int16').typecode()
print Na.array([1],'UInt16').typecode()
print Na.array([1],'Int32').typecode()
print Na.array([1],'UInt32').typecode()
print Na.array([1],'Float32').typecode()
print Na.array([1],'Float64').typecode()
print Na.array([1],'Complex32').typecode()
print Na.array([1],'Complex64').typecode()
print Na.array([1],'Bool').typecode()
print Na.array([1],'UInt64').typecode()

prints:

Numarray version:  1.0
1
b
s
w
l
u
f
d
F
D
1
Traceback (most recent call last):
  File "<stdin>", line 14, in ?
  File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 1092, in typecode
    return _nt.typecode[self._type]
KeyError: UInt64

Should this print 'U'?

Regards, Phil Austin

[Numpy-discussion] ANN: numarray-1.0 released

[Todd M. <jm...@st...>]

Release Notes for numarray-1.0

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modeled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, and improved type
promotions.

I. ENHANCEMENTS

1. User added ufuncs

There's a setup.py file in numarray-1.0/Examples/ufunc which
demonstrates how a numarray user can define their own universal
functions of one or two parameters.  Ever wanted to write your own
bessel() function for use on arrays?  Now you can.  Your ufunc can use
exactly the same machinery as add().

2. Ports of Numeric functions

A bunch of Numeric functions were ported to numarray in the new
libnumeric module.  To get these import from numarray.numeric.  Most
notable among these are put, putmask, take, argmin, and argmax.  Also
added were sort, argsort, concatenate, repeat and resize.  These are
independent ports/implementations in C done for the purpose of best
Numeric compatibility and small array performance.  The numarray
versions, which handle additional cases, still exist and are the
default in numarray proper.

3. Faster matrix multiply

The setup for numarray's matrix multiply was moved into C-code.  This
makes it faster for small matrices.

4. The numarray "header PEP"

A PEP has been started for the inclusion of numarray (and possibly
Numeric) C headers into the Python core.  The PEP will demonstrate how
to provide optional support for arrays (the end-user may or may not
have numarray installed and the extension will still work).  It may
also (eventually) demonstrate how to build extensions which support
both numarray and Numeric.  Thus, the PEP is seeking to make it
possible to distribute extensions which will still compile when
numarray (or either) is not present in a user's Python installation,
which will work when numarry (or either) is not installed, and which
will improve performance when either is installed.  The PEP is now in
numarray-1.0/Doc/header_pep.txt in docutils format.  We want feedback
and consensus before we submit to python-dev so please consider
reading it and commenting.

For the PEP, the C-API has been partitioned into two parts: a
relatively simple Numeric compatible part and the numarray native
part.  This broke source and binary compatibility with numarray-0.9.
See CAUTIONS below for more information.

5. Changes to the manual

There are now brief sections on numarray.mlab and numarray.objects in
the manual.  The discussion of the C-API has been updated.

II. CAUTIONS

1. The numarray-1.0 C-API is neither completely source level nor
binary compatible with numarray-0.9. First, this means that some 3rd
party extensions will no longer compile without errors.  Second, this
means that binary packages built against numarray-0.9 will fail,
probably disastrously, using numarray-1.0.  Don't install numarray-1.0
until you are ready to recompile or replace your extensions with
numarray-1.0 binaries because 0.9 binaries will not work.

In order to support the header PEP, the numarray C-API was partitioned
into two parts: Numeric compatible and numarry extensions. You can use
the Numeric compatible API (the PyArray_* functions) by including
arrayobject.h and calling import_array() in your module init function.
You can use the extended API (the NA_* functions) by including
libnumarray.h and calling import_libnumarray() in your init function.
Because of the partitioning, all numarray extensions must be
recompiled to work with 1.0.  Extensions using *both* APIs must
include both files in order to compile, and must do both imports in
order to run.  Both APIs share a common PyArrayObject struct.

2. numarray extension writers should note that the documented use of
PyArray_INCREF and PyArray_XDECREF (in numarray) was found to be
incompatible with Numeric and these functions have therefore been
removed from the supported API and will now result in errors.

3. The numarray.objects.ObjectArray parameter order was changed.

4. The undocumented API function PyArray_DescrFromTypeObj was removed
from the Numeric compatible API because it is not provided by Numeric.

III. BUGS FIXED / CLOSED

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

979834  convolve2d parameter order issues
979775  ObjectArray parameter order
979712  No exception for invalid axis
979702  too many slices fails silently
979123  A[n:n] = x no longer works
979028  matrixmultiply precision
976951  Unpickled numarray types unsable?
977472  CharArray concatenate
970356  bug in accumulate contiguity status
969162  object array bug/ambiguity
963921  bitwise_not over Bool type fails
963706  _reduce_out: problem with otype
942804  numarray C-API include file
932438  suggest moving mlab up a level
932436  mlab docs missing
857628  numarray allclose returns int
839401  Argmax's behavior has changed for ties
817348  a+=1j # Not converted to complex
957089  PyArray_FromObject dim check broken
923046  numarray.objects incompatibility  
897854  Type conflict when embedding on OS X  
793421  PyArray_INCREF / PyArray_XDECREF deprecated
735479  Build failure on Cygwin 1.3.22 (very current install).
870660  Numarray: CFLAGS build problem  
874198  numarray.random_array.random() broken?
874207  not-so random numbers in numarray.random_array
829662  Downcast from Float64 to UInt8 anomaly
867073  numarray diagonal bug?  
806705  a tale of two rank-0's
863155  Zero size numarray breaks for loop
922157  argmax returns integer in some cases
934514  suggest nelements -> size
953294  choose bug
955314  strings.num2char bug?
955336  searchsorted has strange behaviour
955409  MaskedArray problems
953567  Add read-write requirement to NA_InputArray
952705  records striding for > 1D arrays
944690  many numarray array methods not documented
915015  numarray/Numeric incompatabilities
949358  UsesOpPriority unexpected behavior
944678  incorrect help for "size" func/method
888430  NA_NewArray() creates array with wrong endianess
922798  The document Announce.txt is out of date
947080  numarray.image.median bugs
922796  Manual has some dated MA info
931384  What does True mean in a mask?
931379  numeric.ma called MA in manual
933842  Bool arrays don't allow bool assignment
935588  problem parsing argument "nbyte" in callStrideConvCFunc()
936162  problem parsing "nbytes" argument in copyToString()
937680  Error in Lib/numerictypes.py ?
936539  array([cmplx_array, int_array]) fails
936541  a[...,1] += 0 crashes interpreter.
940826  Ufunct operator don't work
935882  take for character arrays?
933783  numarray, _ufuncmodule.c: problem setting buffersize
930014  fromstring typecode param still broken
929841  searchsorted type coercion
924841  numarray.objects rank-0 results
925253  numarray.objects __str__ and __repr__
913782  Minor error in chapter 12: NUM_ or not?
889591  wrong header file for C extensions
925073  API manual comments
924854  take() errors
925754  arange() with large argument crashes interpreter
926246  ufunc reduction crash
902153  can't compile under RH9/gcc 3.2.2
916876  searchsorted/histogram broken in versions 0.8 and 0.9
920470  numarray arange() problem
915736  numarray-0.9: Doc/CHANGES not up to date

WHERE
-----------

Numarray-1.0 windows executable installers, source code, and manual is
here:

http://sourceforge.net/project/showfiles.php?group_id=1369

Numarray is hosted by Source Forge in the same project which hosts
Numeric:

http://sourceforge.net/projects/numpy/

The web page for Numarray information is at:

http://stsdas.stsci.edu/numarray/index.html

Trackers for Numarray Bugs, Feature Requests, Support, and Patches are
at the Source Forge project for NumPy at:

http://sourceforge.net/tracker/?group_id=1369

REQUIREMENTS
------------------------------

numarray-1.0 requires Python 2.2.2 or greater.  


AUTHORS, LICENSE
------------------------------

Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC
Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science
Institute.  We'd like to acknowledge the assitance of Francesc Alted,
Paul Dubois, Sebastian Haase, Tim Hochberg, Nadav Horesh, Edward
C. Jones, Eric Jones, Jochen K"upper, Travis Oliphant, Pearu Peterson,
Peter Verveer, Colin Williams, and everyone else who has contributed
with comments, bug reports, or patches.

Numarray is made available under a BSD-style License.  See
LICENSE.txt in the source distribution for details.

-- 
Todd Miller             jm...@st...

Re: [Numpy-discussion] bug in numarray.maximum.reduce ?

[Todd M. <jm...@st...>]

On Fri, 2004-07-02 at 13:45, Sebastian Haase wrote:
> On Friday 02 July 2004 09:02 am, Todd Miller wrote:
> > On Fri, 2004-07-02 at 11:27, Sebastian Haase wrote:
> > > On Tuesday 29 June 2004 05:05 pm, Sebastian Haase wrote:
> > > > Hi,
> > > >
> > > > Is this a bug?:
> > > > >>> # (import numarray as na ; 'd' is a 3 dimensional array)
> > > > >>> d.type()
> > > >
> > > > Float32
> > > >
> > > > >>> d[80, 136, 122]
> > > >
> > > > 80.3997039795
> > > >
> > > > >>> na.maximum.reduce(d[:,136, 122])
> > > >
> > > > 85.8426361084
> > > >
> > > > >>> na.maximum.reduce(d) [136, 122]
> > > >
> > > > 37.3658103943
> > > >
> > > > >>> na.maximum.reduce(d,0)[136, 122]
> > > >
> > > > 37.3658103943
> > > >
> > > > >>> na.maximum.reduce(d,1)[136, 122]
> > > >
> > > > Traceback (most recent call last):
> > > >   File "<input>", line 1, in ?
> > > > IndexError: Index out of range
> > > >
> > > > I was using  na.maximum.reduce(d)  to get a "pixelwise" maximum along Z
> > > > (axis 0). But as seen above it does not get it right.  I then tried to
> > > > reproduce
> > > >
> > > > this with some simple arrays, but here it works just fine:
> > > > >>> a = na.arange(4*4*4)
> > > > >>> a.shape=(4,4,4)
> > > > >>> na.maximum.reduce(a)
> > > >
> > > > [[48 49 50 51]
> > > >  [52 53 54 55]
> > > >  [56 57 58 59]
> > > >  [60 61 62 63]]
> > > >
> > > > >>> a = na.arange(4*4*4).astype(na.Float32)
> > > > >>> a.shape=(4,4,4)
> > > > >>> na.maximum.reduce(a)
> > > >
> > > > [[ 48.  49.  50.  51.]
> > > >  [ 52.  53.  54.  55.]
> > > >  [ 56.  57.  58.  59.]
> > > >  [ 60.  61.  62.  63.]]
> > > >
> > > >
> > > > Any hint ?
> > > >
> > > > Regards,
> > > > Sebastian Haase
> > >
> > > Hi again,
> > > I think the reason that no one responded to this is that it just sounds
> > > to unbelievable ...
> >
> > This just slipped through the cracks for me.
> >
> > > Sorry for the missing piece of information, but 'd' is actually a
> > > memmapped array !
> > >
> > > >>> d.info()
> > >
> > > class: <class 'numarray.numarraycore.NumArray'>
> > > shape: (80, 150, 150)
> > > strides: (90000, 600, 4)
> > > byteoffset: 0
> > > bytestride: 4
> > > itemsize: 4
> > > aligned: 1
> > > contiguous: 1
> > > data: <MemmapSlice of length:7290000 readonly>
> > > byteorder: big
> > > byteswap: 1
> > > type: Float32
> > >
> > > >>> dd = d.copy()
> > > >>> na.maximum.reduce(dd[:,136, 122])
> > >
> > > 85.8426361084
> > >
> > > >>> na.maximum.reduce(dd)[136, 122]
> > >
> > > 85.8426361084
> > >
> > >
> > > Apparently we are using memmap so frequently now that I didn't even think
> > > about that - which is good news for everyone, because it means that it
> > > works (mostly).
> > >
> > > I just see that 'byteorder' is 'big' - I'm running this on an Intel Linux
> > > PC. Could this be the problem?
> >
> > I think byteorder is a good guess at this point.  What version of Python
> > and numarray are you using?
> 
> Python 2.2.1 (#1, Feb 28 2004, 00:52:10)
> [GCC 2.95.4 20011002 (Debian prerelease)] on linux2
> 
> numarray 0.9 - from CVS on 2004-05-13.
> 
> Regards,
> Sebastian Haase

Hi Sebastian,

I logged this on SF as a bug but won't get to it until next week after
numarray-1.0 comes out.

Regards,
Todd

Re: [Numpy-discussion] bug in numarray.maximum.reduce ?

[Sebastian H. <ha...@ms...>]

On Friday 02 July 2004 09:02 am, Todd Miller wrote:
> On Fri, 2004-07-02 at 11:27, Sebastian Haase wrote:
> > On Tuesday 29 June 2004 05:05 pm, Sebastian Haase wrote:
> > > Hi,
> > >
> > > Is this a bug?:
> > > >>> # (import numarray as na ; 'd' is a 3 dimensional array)
> > > >>> d.type()
> > >
> > > Float32
> > >
> > > >>> d[80, 136, 122]
> > >
> > > 80.3997039795
> > >
> > > >>> na.maximum.reduce(d[:,136, 122])
> > >
> > > 85.8426361084
> > >
> > > >>> na.maximum.reduce(d) [136, 122]
> > >
> > > 37.3658103943
> > >
> > > >>> na.maximum.reduce(d,0)[136, 122]
> > >
> > > 37.3658103943
> > >
> > > >>> na.maximum.reduce(d,1)[136, 122]
> > >
> > > Traceback (most recent call last):
> > >   File "<input>", line 1, in ?
> > > IndexError: Index out of range
> > >
> > > I was using  na.maximum.reduce(d)  to get a "pixelwise" maximum along Z
> > > (axis 0). But as seen above it does not get it right.  I then tried to
> > > reproduce
> > >
> > > this with some simple arrays, but here it works just fine:
> > > >>> a = na.arange(4*4*4)
> > > >>> a.shape=(4,4,4)
> > > >>> na.maximum.reduce(a)
> > >
> > > [[48 49 50 51]
> > >  [52 53 54 55]
> > >  [56 57 58 59]
> > >  [60 61 62 63]]
> > >
> > > >>> a = na.arange(4*4*4).astype(na.Float32)
> > > >>> a.shape=(4,4,4)
> > > >>> na.maximum.reduce(a)
> > >
> > > [[ 48.  49.  50.  51.]
> > >  [ 52.  53.  54.  55.]
> > >  [ 56.  57.  58.  59.]
> > >  [ 60.  61.  62.  63.]]
> > >
> > >
> > > Any hint ?
> > >
> > > Regards,
> > > Sebastian Haase
> >
> > Hi again,
> > I think the reason that no one responded to this is that it just sounds
> > to unbelievable ...
>
> This just slipped through the cracks for me.
>
> > Sorry for the missing piece of information, but 'd' is actually a
> > memmapped array !
> >
> > >>> d.info()
> >
> > class: <class 'numarray.numarraycore.NumArray'>
> > shape: (80, 150, 150)
> > strides: (90000, 600, 4)
> > byteoffset: 0
> > bytestride: 4
> > itemsize: 4
> > aligned: 1
> > contiguous: 1
> > data: <MemmapSlice of length:7290000 readonly>
> > byteorder: big
> > byteswap: 1
> > type: Float32
> >
> > >>> dd = d.copy()
> > >>> na.maximum.reduce(dd[:,136, 122])
> >
> > 85.8426361084
> >
> > >>> na.maximum.reduce(dd)[136, 122]
> >
> > 85.8426361084
> >
> >
> > Apparently we are using memmap so frequently now that I didn't even think
> > about that - which is good news for everyone, because it means that it
> > works (mostly).
> >
> > I just see that 'byteorder' is 'big' - I'm running this on an Intel Linux
> > PC. Could this be the problem?
>
> I think byteorder is a good guess at this point.  What version of Python
> and numarray are you using?

Python 2.2.1 (#1, Feb 28 2004, 00:52:10)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2

numarray 0.9 - from CVS on 2004-05-13.

Regards,
Sebastian Haase