numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Re: Why floor and ceil change the type of the array?

[Sasha <nd...@ma...>]

On 2/23/06, Robert Kern <rob...@gm...> wrote:
> > I cannot really think of any reason for the current numpy behaviour
> > other than the consistency with transcendental functions.
>
> It's simply the easiest thing to do with the ufunc machinery.
>
That's what I had in mind with the curent rule the same code can be
use for ceil as for sin.  However, easiest to implement is not
necessarily right.

> > Speaking of
> > which, can someone explain this:
> >
> >>>>sin(array(1,'h')).dtype
> >
> > dtype('<f4')
> >
> >>>>sin(array(1,'i')).dtype
> >
> > dtype('<f8')
>
> AFAICT, the story goes like this: sin() has two implementations, one for
> single-precision floats and one for doubles. The ufunc machinery sees the=
 int16
> and picks single-precision as the smallest type of the two that can fit a=
n int16
> without losing precision. Naturally, you probably want the function to op=
erate
> in higher precision, but that's not really information that the ufunc mac=
hinery
> knows about.

According to your theory long (i8) integers should cast to long doubles, bu=
t
>>> sin(array(0,'i8')).dtype
dtype('<f8')


Given that python's floating point object is a double, I think it
would be natural to cast integer arguments to double for all sizes.  I
would also think that in choosing the precision for a function it is
also important that the output fits into data type.  I find the
following unfortunate:

>>> exp(400)
5.2214696897641443e+173
>>> exp(array(400,'h'))
inf

[Numpy-discussion] Thoughts on an ndarray super-class

[Travis O. <oli...@ie...>]

The bigndarray class is going to disappear (probably in the next release 
of NumPy).  It was a stop-gap measure as the future of 64-bit fixes in 
Python was unclear.  Python 2.5 will have removed the 64-bit limitations 
that led to the bigndarray and so it will be removed.  

I have been thinking, however, of replacing it with a super-class that 
does not define the dimensions or strides.   

In other words, the default array would be just a block of memory.  The 
standard array would inherit from the default and add dimension and 
strides pointers.

I was thinking that this might make it easier for sub-classes using 
fixed-sized dimensions and strides.  I'm not sure if that would actually 
be useful, but since I was thinking about the disappearance of the 
bigndarray, I thought I would ask for comments.

-Travis

Re: [Numpy-discussion] Timings for various round functions

[Travis O. <oli...@ie...>]

Sasha wrote:

>C99 defines three functions round, rint and nearbyint that are nearly
>identical.  The only difference is in setting the inexact flag and
>respecting the rounding mode.  Nevertheless, these functions differ
>significantly in their performance.  I've wraped these functions into
>ufuncs and go the following timings:
>  
>
>Obviously, I will use rint in my ndarray.round implementation,
>however, it may be useful to provide all three as ufuncs.
>
>The only question is what name to use for round?
>
>1) round (may lead to confusion with ndarray.round or built-in round)
>2) roundint (too similar to rint)
>3) round0 (ugly)
>
>Any suggestions?
>
>Another C99 function that may be worth including is "trunc".  Any
>objections to adding it as a ufunc?
>  
>

I think we have agreed that C99 functions are good candidates to become 
ufuncs.  The only problem is figuring out what to do on platforms that 
don't define them. 

For example, we could define a separate module of C99 functions that is 
only available on certain platforms. 

-Travis

[Numpy-discussion] Timings for various round functions

[Sasha <nd...@ma...>]

C99 defines three functions round, rint and nearbyint that are nearly
identical.  The only difference is in setting the inexact flag and
respecting the rounding mode.  Nevertheless, these functions differ
significantly in their performance.  I've wraped these functions into
ufuncs and go the following timings:

> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000)" "round(x)"
1000 loops, best of 3: 257 usec per loop
> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000)" "nearbyint(x)"
1000 loops, best of 3: 654 usec per loop
> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000)" "rint(x)"
10000 loops, best of 3: 103 usec per loop

Similarly for single precision:

> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000,dtype=3D'f')" "round(x)"
10000 loops, best of 3: 182 usec per loop
> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000,dtype=3D'f')" "nearbyint(x)"
1000 loops, best of 3: 606 usec per loop
> python -m timeit -s "from numpy import array, round, rint, nearbyint; x =
=3D array([1.5]*10000,dtype=3D'f')" "rint(x)"
10000 loops, best of 3: 85.5 usec per loop

Obviously, I will use rint in my ndarray.round implementation,
however, it may be useful to provide all three as ufuncs.

The only question is what name to use for round?

1) round (may lead to confusion with ndarray.round or built-in round)
2) roundint (too similar to rint)
3) round0 (ugly)

Any suggestions?

Another C99 function that may be worth including is "trunc".  Any
objections to adding it as a ufunc?

[Numpy-discussion] Re: Why floor and ceil change the type of the array?

[Robert K. <rob...@gm...>]

Sasha wrote:
> On 2/22/06, Robert Kern <rob...@gm...> wrote:
> 
>>Sasha wrote:
>>
>>>... wouldn't it
>>>be more natural if fllor and ceil return the argument unchanged (maybe
>>>a copy) if it is already integer?
>>
>>Only if floor() and ceil() returned integer arrays when given floats as input. I
>>presume there are good reasons for this, since it's the same behavior as the
>>standard C functions.
> 
> C does not have ceil(int).  It has
> 
>        double ceil(double x);
>        float ceilf(float x);
>        long double ceill(long double x);
> 
> and neither of these functions change the type of the argument.

That's exactly what I meant. These functions only apply to integers through
casting to the appropriate float type. That's precisely what numpy.floor() and
numpy.ceil() do.

Actually, I think the reasoning for having the float versions return floats
instead of integers is that an integer-valued double is possibly out of range
for an int or long on some platforms, so it's kept as a float. Since this
obviously isn't a problem if the input is already an integer type, I don't have
any particular objection to making floor() and ceil() return integers if their
inputs are integers.

> Numpy's "around" is a noop on integers (even for decimals<0, but
> that's a different story.

It's also a function and not a ufunc.

> I cannot really think of any reason for the current numpy behaviour
> other than the consistency with transcendental functions. 

It's simply the easiest thing to do with the ufunc machinery.

> Speaking of
> which, can someone explain this:
> 
>>>>sin(array(1,'h')).dtype
> 
> dtype('<f4')
> 
>>>>sin(array(1,'i')).dtype
> 
> dtype('<f8')

AFAICT, the story goes like this: sin() has two implementations, one for
single-precision floats and one for doubles. The ufunc machinery sees the int16
and picks single-precision as the smallest type of the two that can fit an int16
without losing precision. Naturally, you probably want the function to operate
in higher precision, but that's not really information that the ufunc machinery
knows about.

-- 
Robert Kern
rob...@gm...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

Re: [Numpy-discussion] Re: Why floor and ceil change the type of the array?

[Sasha <nd...@ma...>]

On 2/22/06, Robert Kern <rob...@gm...> wrote:
> Sasha wrote:
> > ... wouldn't it
> > be more natural if fllor and ceil return the argument unchanged (maybe
> > a copy) if it is already integer?
>
> Only if floor() and ceil() returned integer arrays when given floats as i=
nput. I
> presume there are good reasons for this, since it's the same behavior as =
the
> standard C functions.

C does not have ceil(int).  It has

       double ceil(double x);
       float ceilf(float x);
       long double ceill(long double x);

and neither of these functions change the type of the argument.

Numpy's "around" is a noop on integers (even for decimals<0, but
that's a different story.

I cannot really think of any reason for the current numpy behaviour
other than the consistency with transcendental functions.  Speaking of
which, can someone explain this:

>>> sin(array(1,'h')).dtype
dtype('<f4')
>>> sin(array(1,'i')).dtype
dtype('<f8')

[Numpy-discussion] Re: Why floor and ceil change the type of the array?

[Robert K. <rob...@gm...>]

Sasha wrote:
> I was looking for a model to implement "round" in C and discovered
> that floor and ceil functions change the type of their arguments:
> 
>>>>floor(array([1,2,3],dtype='i2')).dtype
> 
> dtype('<f4')
> 
>>>>floor(array([1,2,3],dtype='i4')).dtype
> 
> dtype('<f8')
> 
> I know that this is the same behavior as in Numeric, but wouldn't it
> be more natural if fllor and ceil return the argument unchanged (maybe
> a copy) if it is already integer?

Only if floor() and ceil() returned integer arrays when given floats as input. I
presume there are good reasons for this, since it's the same behavior as the
standard C functions.

-- 
Robert Kern
rob...@gm...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

Re: [Numpy-discussion] Multiple inheritance from ndarray

[Travis O. <oli...@ie...>]

Robert Lupton wrote:

> I have a swig extension that defines a class that inherits from
> both a personal C-coded image struct (actImage), and also from
> Numeric's UserArray.  This works very nicely, but I thought that
> it was about time to upgrade to numpy.
>
> The code looks like:
>
> from UserArray import *
>
> class Image(UserArray, actImage):
>     def __init__(self, *args):
>         actImage.__init__(self, *args)
>         UserArray.__init__(self, self.getArray(), 'd', copy=False,  
> savespace=False)
>
> I can't figure out how to convert this to use ndarray, as ndarray  
> doesn't
> seem to have an __init__ method, merely a __new__.


Yes, the ndarray method doesn't have an __init__ method (so you don't 
have to call it).

What you need to do is write a __new__ method for your class.  However, 
with multiple-inheritance the details matter.   You may actually want to 
have your C-coded actImage class inherit (in C) from the ndarray.   If 
you would like help on that approach let me know (I'll need to 
understand your actImage a bit better). 

But, this can all be done in Python, too, but it is a bit of effort to 
make sure things get created correctly.  Perhaps it might make sense to 
actually include a slightly modified form of the UserArray in NumPy as a 
standard "container-class" (instead of a sub-class) of the ndarray.    
In reality, a container class like UserArray and a sub-class are 
different things.

Here's an outline of what you need to do.  This is, of course, 
untested....  For example, I don't really know what actImage is.

from numpy import ndarray, array

class Image(ndarray, actImage):
    def __new__(subtype, *args)
        act1 = actImage.__new__(actImage, *args)
        actImage.__init__(act1, *args)
        arr = array(act1.getArray(), 'd', copy=False)
        self = arr.view(subtype)
        # you might need to copy attributes from act1 over to self here...
        return self


The problem here, is that apparently you are creating the array first in 
actImage.__init__ and then passing it to UserArray.   The ndarray 
constructor wants to either create the array itself or use a 
buffer-exposing object to use as the memory.  

Keep us posted as your example is a good one that can help us all learn.

-Travis

Re: [Numpy-discussion] optimizing power() for complex and real cases

[Tim H. <tim...@co...>]

David M. Cooke wrote:

>Tim Hochberg <tim...@co...> writes:
>
>  
>
>>David M. Cooke wrote:
>>    
>>
[SNIP]

>
>I've gone through your code you checked in, and fixed it up. Looks
>good. One side effect is that
>
>def zl(x):
>    a = ones_like(x)
>    a[:] = 0
>    return a
>
>is now faster than zeros_like(x) :-)
>  
>
I noticed that ones_like was faster than zeros_like, but I didn't think 
to try that. That's pretty impressive considering how ridicuously easy 
it was to write.

>One problem I had is that in PyArray_SetNumericOps, the "copy" method
>wasn't picked up on. It may be due to the order of initialization of
>the ndarray type, or something (since "copy" isn't a ufunc, it's
>initialized in a different place). I couldn't figure out how to fiddle
>that, so I replaced the x.copy() call with a call to PyArray_Copy().
>  
>
Interesting. It worked fine here.

>  
>
>>>Yes; because it's the implementation of __pow__, the second argument can
>>>be anything.
>>>
>>>
>>>      
>>>
>>No, you misunderstand.. What I was talking about was that the *first*
>>argument can also be something that's not a PyArrayObject, despite the
>>functions signature.
>>    
>>
>
>Ah, I suppose that's because the power slot in the number protocol
>also handles __rpow__.
>  
>
That makes sense. It was giving me fits whatever the cause.

[SNIP]

>
>>but the real fix would be to dig into
>>PyArray_EnsureArray and see why it's slow for Python_Ints. It is much
>>faster for numarray scalars.
>>    
>>
>
>Right; that needs to be looked at.
>  
>
It doesn't look to bad. But I haven't had a chance to try to do anything 
about it yet.

>>Another approach is to actually compute (x*x)*(x*x) for pow(x,4) at
>>the level of array_power. I think I could make this work. It would
>>probably work well for medium size arrays, but might well make things
>>worse for large arrays that are limited by memory bandwidth since it
>>would need to move the array from memory into the cache multiple times.
>>    
>>
>
>I don't like that; I think it would be better memory-wise to do it
>elementwise. Not just speed, but size of intermediate arrays.
>  
>
Yeah, for a while I was real hot on the idea since I could do everything 
without messing with ufuncs. But then I decided not to pursue it because 
I thought it would be slow because of memory usage -- it would be 
pullling data into the cache over and over again and I think that would 
slow things down a lot.

[SNIP]

>'int_power' we could do; that would the next step I think. The half
>integer powers we could maybe leave; if you want x**(-3/2), for
>instance, you could do y = x**(-1)*sqrt(x) (or do y = x**(-1);
>sqrt(y,y) if you're worried about temporaries).
>
>Or, 'fast_power' could be documented as doing the optimizations for
>integer and half-integer _scalar_ exponents, up to a certain size,
>like 100), and falling back on pow() if necessary. I think we could do
>a precomputation step to split the exponent into appropiate squarings
>and such that'll make the elementwise loop faster.
>
There's a clever implementation of this in complexobject.c.  Speaking of 
complexobject.c, I did implement fast integer powers for complex objects 
at the nc_pow level. For small powers at least, it's over 10 times as 
fast. And, since it's at the nc_pow level it works for matrix matrix 
powers as well. My implementation is arguably slightly faster than 
what's in complexobject, but I won't have a chance to check it in till 
next week -- I'm off for some snowboarding tomorrow.

I kind of like power and scalar_power. Then ** could be advertised as 
calling scalar_power for scalars and power for arrays. Scalar power 
would do optimizations on integer and half_integer powers. Of course 
there's no real way to enforce that scalar power is passed scalars, 
since presumably it would be a ufunc, short of making _scalar_power a 
ufunc instead and doing something like:

def scalar_power(x, y):
    "compute x**y, where y is a scalar optimizing integer and half 
integer powers possibly at some minor loss of accuracy"
    if not is_scalar(y): raise ValuerError("Naughty!!")
    return _scalar_power(x,y)

> Half-integer
>exponents are exactly representable as doubles (up to some number of
>course), so there's no chance of decimal-to-binary conversions making
>things look different. That might work out ok. Although, at that point
>I'd suggest we make it 'power', and have 'rawpower' (or ????) as the
>version that just uses pow().
>  
>
>Another point is to look at __div__, and use reciprocal if the
>dividend is 1.
>  
>
That would be easy, but wouldn't it be just as easy to optimize __div__ 
for scalar divisions. Should probably check that this isn't just as fast 
since it would be a lot more general.

>
>I've added a page to the developer's wiki at
>http://projects.scipy.org/scipy/numpy/wiki/PossibleOptimizationAreas
>to keep a list of areas like that to look into if someone has time :-)
>  
>
Ah, good plan.

-tim

[Numpy-discussion] Why floor and ceil change the type of the array?

[Sasha <nd...@ma...>]

I was looking for a model to implement "round" in C and discovered
that floor and ceil functions change the type of their arguments:

>>> floor(array([1,2,3],dtype=3D'i2')).dtype
dtype('<f4')
>>> floor(array([1,2,3],dtype=3D'i4')).dtype
dtype('<f8')


I know that this is the same behavior as in Numeric, but wouldn't it
be more natural if fllor and ceil return the argument unchanged (maybe
a copy) if it is already integer?

Re: [Numpy-discussion] simple subclassing of ndarray

[Travis O. <oli...@ie...>]

Zachary Pincus wrote:

> Hello folks,
>
> I'm interested in creating a simple subclass of ndarray that just has  
> a few additional methods. I've stared at defmatrix.py, but I'm not  
> sure what is necessary to do.
>
> Specifically, I'm not sure how to get new instances of my subclass  
> created properly.
>
> e.g.:
> numpy.matrix([1,2,3])
> Out: matrix([[1, 2, 3]])
>
> class m(numpy.ndarray):
>   pass


This is enough to define your own sub-class.  Now, you need to determine 
what you want to do.

You need to understand that m() is now analagous to numpy.ndarray()  so 
you should look at the numpy.ndarray()  docstring for the default 
arguments. 

The array() constructor is not the same thing as ndarray.__new__.

Look at the ndarray docstring.

help(ndarray)

You need to define the __new__ method *not* the __init__ method.   You 
could, of course, define an __init__ method if you want to, it's just 
not necessary.

> Out: m([[[    13691,         0,         0],
>         [   196608, 296292267, 296303312]]])

You just created an empty array of shape (1,2,3).  The first argument to 
the default constructor is the shape.  

> So clearly I need something else. Looking at the matrix class, it  
> looks like I need a custom __new__ operator. 

Yes, that is exactly right.

> Or perhaps there's a different and better way to construct instances  
> of my subclass? Something akin to the 'array' function would be  
> perfect. Now, how do I go about creating such a function (or getting  
> 'array' to do it)?

You could do

array(obj).view(m)

to get instances of your subclass.  This will not call __new__ or 
__init__, but it will call __array_finalize__(self, obj) where obj is 
the ndarray constructed from [1,2,3].

Actually __array_finalize__ is called every time a sub-class is 
constructed and so it could be used to pass along meta-data (or enforce 
rank-2 as it does in the matrix class).

-Travis

[Numpy-discussion] simple subclassing of ndarray

[Zachary P. <zp...@st...>]

Hello folks,

I'm interested in creating a simple subclass of ndarray that just has  
a few additional methods. I've stared at defmatrix.py, but I'm not  
sure what is necessary to do.

Specifically, I'm not sure how to get new instances of my subclass  
created properly.

e.g.:
numpy.matrix([1,2,3])
Out: matrix([[1, 2, 3]])

class m(numpy.ndarray):
   pass
m([1,2,3])
Out: m([[[    13691,         0,         0],
         [   196608, 296292267, 296303312]]])

So clearly I need something else. Looking at the matrix class, it  
looks like I need a custom __new__ operator. However, looking at  
matrix's __new__ operator, I see a lot of complexity that I just  
don't understand. What's the minimum set of things I need in __new__  
to get a proper constructor?

Or perhaps there's a different and better way to construct instances  
of my subclass? Something akin to the 'array' function would be  
perfect. Now, how do I go about creating such a function (or getting  
'array' to do it)?

Can anyone give me any pointers here?

Thanks,

Zach Pincus

Program in Biomedical Informatics and Department of Biochemistry
Stanford University School of Medicine

[Numpy-discussion] Re: Constructor parameters - was Re: dtype

[Robert K. <rob...@gm...>]

Colin J. Williams wrote:

> Robert,
>
> Many thank for this.  You have described the standard Python approach to
> constructing an instance.  As I understand it,
> numpy uses the __new__ method, but not __init__,  in most cases.
>
> My interest is in " any (positional as well as keyword) arguments".
> What should the user feed the constuctor?  This isn't clear from the
> online documentation.

Look in the code. The PyArrayDescr_Type method table gives arraydescr_new() as
the implementation of the tp_new slot (the C name for __new__). You can read the
implementation for information. Patches for documentation will be gratefully
accepted.

That said:

In [16]: a = arange(10)

In [17]: a.dtype
Out[17]: dtype('>i4')

In [18]: dtype('>i4')
Out[18]: dtype('>i4')

If you want complete documentation on data-type descriptors, it's in Chapter 7
of Travis's book.

> From a Python user's point of view, the module holding the dtype class
> appears to be multiarray.
>
> The standard Python approach is to put the information in a __module__
> attribute so that one doesn't have to go hunting around.  Please see below.

<shrug> dtype.__module__ (== 'numpy') tells you the canonical place to access it
from Python code. It will never be able to tell you what C source file to look
in. You'll have to break out grep no matter what.

> While on the subject of the Standand Python aproach, class names usually
> start with an upper case letter and the builtins have their own style,
> ListType etc.  numpy equates ArrayType to ndarray but ArrayType is
> deprecated.

ListType, TupleType et al. are also deprecated in favor of list and tuple, etc.

But yes, we do use all lower-case names for classes. This is a conscious
decision. It's just a style convention, just like PEP-8 is just a style
convention for the standard library.

-- 
Robert Kern
rob...@gm...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

Re: [Numpy-discussion] Constructor parameters - was Re: dtype

[Colin J. W. <cj...@sy...>]

Robert Kern wrote:

>Colin J. Williams wrote:
>  
>
>>I've been trying to gain some understanding of dtype from the builtin
>>documentation and would appreciate advice.
>>
>>I don't find anything in http://projects.scipy.org/scipy/numpy or
>>http://wiki.python.org/moin/NumPy
>>
>>Chapter 2.1 of the book has a good overview, but little reference material.
>>
>>In the following, dt= numpy.dtype
>>
>>Some specific problems are flagged ** below.
>>
>>Colin W.
>>[snip]
>>    
>>
>>| |  ----------------------------------------------------------------------
>>|  Data and other attributes defined here:
>>| |  __new__ = <built-in method __new__ of type object>   |     
>>T.__new__(S, ...) -> a new object with type S, a subtype of
>>T               ** What are the parameters?  In other words,
>>|                                                                                                                        
>>what does ... stand for?  **
>>    
>>
>
>http://www.python.org/2.2.3/descrintro.html#__new__
>
>"""Recall that you create class instances by calling the class. When the class
>is a new-style class, the following happens when it is called. First, the
>class's __new__ method is called, passing the class itself as first argument,
>followed by any (positional as well as keyword) arguments received by the
>original call. This returns a new instance. Then that instance's __init__ method
>is called to further initialize it. (This is all controlled by the __call__
>method of the metaclass, by the way.)
>"""
>
>  
>
>>**  There is no __module__ attribute.  How does one identify the modules
>>holding the code?  **
>>    
>>
>
>It's an extension type PyArray_Descr* in numpy/core/src/arrayobject.c .
>
>  
>
Robert,

Many thank for this.  You have described the standard Python approach to 
constructing an instance.  As I understand it,
numpy uses the __new__ method, but not __init__,  in most cases.

My interest is in " any (positional as well as keyword) arguments".  
What should the user feed the constuctor?  This isn't clear from the 
online documentation.

 From a Python user's point of view, the module holding the dtype class 
appears to be multiarray.

The standard Python approach is to put the information in a __module__ 
attribute so that one doesn't have to go hunting around.  Please see below.

While on the subject of the Standand Python aproach, class names usually 
start with an upper case letter and the builtins have their own style, 
ListType etc.  numpy equates ArrayType to ndarray but ArrayType is 
deprecated.

Colin W.
C:\>python
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy.core.multiarray as mu
 >>> dir(mu)
['_ARRAY_API', '__doc__', '__file__', '__name__', '__version__', 
'_fastCopyAndTranspose', '_flagdict
', '_get_ndarray_c_version', 'arange', 'array', 'bigndarray', 
'broadcast', 'can_cast', 'concatenate'
, 'correlate', 'dot', 'dtype', 'empty', 'error', 'flatiter', 
'frombuffer', 'fromfile', 'fromstring',
 'getbuffer', 'inner', 'lexsort', 'ndarray', 'newbuffer', 
'register_dtype', 'scalar', 'set_numeric_o
ps', 'set_string_function', 'set_typeDict', 'typeinfo', 'where', 'zeros']
 >>>

Re: [Numpy-discussion] mixing arrays and matrices: squeeze yes, flattened no?

[Christopher B. <Chr...@no...>]

Sven Schreiber wrote:
> - If I have >1 variable then everything is fine (provided I use your
> advice of slicing instead of indexing afterwards) and the variables are
> in the _columns_ of the 2d-array.
> - But if there's just one data _column_ in the file, then pylab/numpy
> gives me a 1d-array that sometimes works as a _row_ (and as you noted,
> sometimes not), but never works as a column.
> 
> Imho that's bad, because as a consequence I must use overhead code to
> distinguish between these cases.

I'd do that on load. You must have a way of knowing how many variables 
you're loading, so when it is one you can add this line:

a.shape = (1,-1)

and then proceed the same way after that.

-chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Method to shift elements in an array?

[Zachary P. <zp...@st...>]

Here is my eventual solution. I'm not sure it's speed-optimal for  
even a python implementation, but it is terse.

I agree that it might be nice to have this fast, and/or in C (I'm  
using it for finite difference and related things).

def cshift(l, offset):
   offset %= len(l)
   return numpy.concatenate((l[-offset:], l[:-offset]))

Zach



On Feb 22, 2006, at 11:40 AM, Mads Ipsen wrote:

> On Wed, 22 Feb 2006, Alan G Isaac wrote:
>
>> On Wed, 22 Feb 2006, Zachary Pincus apparently wrote:
>>> Does numpy have an built-in mechanism to shift elements along some
>>> axis in an array? (e.g. to "roll" [0,1,2,3] by some offset, here 2,
>>> to make [2,3,0,1])
>>
>> This sounds like the rotater command in GAUSS.
>> As far as I know there is no equivalent in numpy.
>> Please post your ultimate solution.
>>
>> Cheers,
>> Alan Isaac
>>
>
> Similar to cshift() (cyclic shift) in F90. Very nice for calculating
> finite differences, such as
>
>   x' = ( cshift(x,+1) - cshift(x-1) ) / dx
>
> This would be a very handy feature indeed.
>
> // Mads
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through  
> log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD  
> SPLUNK!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=103432&bid=230486&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Multiple inheritance from ndarray

[Charlie M. <cw...@gm...>]

Since no one has answered this, I am going to take a whack at it.=20
Experts feel free to shoot me down.

Here is a sample showing multiple inheritance with a mix of old style
and new style classes.  I don't claim there is any logic to the code,
but it is just for demo purposes.
--------------------------------------
from numpy import *

class actImage:
   def __init__(self, colorOrder=3D'RGBA'):
      self.colorOrder =3D colorOrder

class Image(actImage, ndarray):
   def __new__(cls, shape=3D(1024,768), dtype=3Dfloat32):
      return ndarray.__new__(cls, shape=3Dshape, dtype=3Ddtype)

x =3D Image()
assert isinstance(x[0,1], float32)
assert x.colorOrder =3D=3D 'RGBA'
--------------------------------------

Running "help(ndarray)" has some useful info as well.

- Charlie

On 2/19/06, Robert Lupton <rh...@as...> wrote:
> I have a swig extension that defines a class that inherits from
> both a personal C-coded image struct (actImage), and also from
> Numeric's UserArray.  This works very nicely, but I thought that
> it was about time to upgrade to numpy.
>
> The code looks like:
>
> from UserArray import *
>
> class Image(UserArray, actImage):
>      def __init__(self, *args):
>          actImage.__init__(self, *args)
>          UserArray.__init__(self, self.getArray(), 'd', copy=3DFalse,
> savespace=3DFalse)
>
> I can't figure out how to convert this to use ndarray, as ndarray
> doesn't
> seem to have an __init__ method, merely a __new__.
>
> So what's the approved numpy way to handle multiple inheritance?
> I've a nasty
> idea that this is a python question that I should know the answer to,
> but I'm
> afraid that I don't...
>
>                                 R
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi=
les
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D103432&bid=3D230486&dat=
=3D121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

Re: [Numpy-discussion] Method to shift elements in an array?

[Mads I. <mp...@os...>]

On Wed, 22 Feb 2006, Alan G Isaac wrote:

> On Wed, 22 Feb 2006, Zachary Pincus apparently wrote:
> > Does numpy have an built-in mechanism to shift elements along some
> > axis in an array? (e.g. to "roll" [0,1,2,3] by some offset, here 2,
> > to make [2,3,0,1])
>
> This sounds like the rotater command in GAUSS.
> As far as I know there is no equivalent in numpy.
> Please post your ultimate solution.
>
> Cheers,
> Alan Isaac
>

Similar to cshift() (cyclic shift) in F90. Very nice for calculating
finite differences, such as

  x' = ( cshift(x,+1) - cshift(x-1) ) / dx

This would be a very handy feature indeed.

// Mads

Re: [Numpy-discussion] Method to shift elements in an array?

[Tim H. <tim...@co...>]

Alan G Isaac wrote:

>On Wed, 22 Feb 2006, Zachary Pincus apparently wrote: 
>  
>
>>Does numpy have an built-in mechanism to shift elements along some 
>>axis in an array? (e.g. to "roll" [0,1,2,3] by some offset, here 2, 
>>to make [2,3,0,1]) 
>>    
>>
>
>This sounds like the rotater command in GAUSS.
>As far as I know there is no equivalent in numpy.
>Please post your ultimate solution.
>  
>

If you need to roll just a few elements the following should work fairly 
efficiently. If you don't want to roll in place, you could instead copy 
A on the way in and return the modified copy. However, in that case, 
concatenating slices might be better.

--------------------------------------------------------------------

import numpy

def roll(A, n):
    "Roll the array A in place. Positive n -> roll right, negative n -> 
roll left"
    if n > 0:
        n = abs(n)
        temp = A[-n:]
        A[n:] = A[:-n]
        A[:n] = temp
    elif n < 0:
        n = abs(n)
        temp = A[:n]
        A[:-n] = A[n:]
        A[-n:] = temp
    else:
        pass
       

A = numpy.arange(10)

print A
roll(A, 3)
print A
roll(A, -3)
print A

Re: [Numpy-discussion] Method to shift elements in an array?

[Alan G I. <ai...@am...>]

On Wed, 22 Feb 2006, Zachary Pincus apparently wrote: 
> Does numpy have an built-in mechanism to shift elements along some 
> axis in an array? (e.g. to "roll" [0,1,2,3] by some offset, here 2, 
> to make [2,3,0,1]) 

This sounds like the rotater command in GAUSS.
As far as I know there is no equivalent in numpy.
Please post your ultimate solution.

Cheers,
Alan Isaac

[Numpy-discussion] Re: dtype

[Robert K. <rob...@gm...>]

Colin J. Williams wrote:
> I've been trying to gain some understanding of dtype from the builtin
> documentation and would appreciate advice.
> 
> I don't find anything in http://projects.scipy.org/scipy/numpy or
> http://wiki.python.org/moin/NumPy
> 
> Chapter 2.1 of the book has a good overview, but little reference material.
> 
> In the following, dt= numpy.dtype
> 
> Some specific problems are flagged ** below.
> 
> Colin W.
> 
> [Dbg]>>> h(dt)
> Help on class dtype in module numpy:
> 
> class dtype(__builtin__.object)
> |  Methods defined here:
> | |  __cmp__(...)
> |      x.__cmp__(y) <==> cmp(x,y)
> | |  __getitem__(...)
> |      x.__getitem__(y) <==> x[y]
> | |  __len__(...)
> |      x.__len__() <==> len(x)
> | |  __reduce__(...)
> |      self.__reduce__()  for pickling.
> | |  __repr__(...)
> |      x.__repr__() <==> repr(x)
> | |  __setstate__(...)
> |      self.__setstate__()  for pickling.
> | |  __str__(...)
> |      x.__str__() <==> str(x)
> | |  newbyteorder(...)
> |      self.newbyteorder(<endian>) returns a copy of the dtype object
> |      with altered byteorders.  If <endian> is not given all byteorders
> |      are swapped.  Otherwise endian can be '>', '<', or '=' to force
> |      a byteorder.  Descriptors in all fields are also updated in the
> |      new dtype object.
> | |  ----------------------------------------------------------------------
> |  Data and other attributes defined here:
> | |  __new__ = <built-in method __new__ of type object>   |     
> T.__new__(S, ...) -> a new object with type S, a subtype of
> T               ** What are the parameters?  In other words,
> |                                                                                                                        
> what does ... stand for?  **

http://www.python.org/2.2.3/descrintro.html#__new__

"""Recall that you create class instances by calling the class. When the class
is a new-style class, the following happens when it is called. First, the
class's __new__ method is called, passing the class itself as first argument,
followed by any (positional as well as keyword) arguments received by the
original call. This returns a new instance. Then that instance's __init__ method
is called to further initialize it. (This is all controlled by the __call__
method of the metaclass, by the way.)
"""

> **  There is no __module__ attribute.  How does one identify the modules
> holding the code?  **

It's an extension type PyArray_Descr* in numpy/core/src/arrayobject.c .

-- 
Robert Kern
rob...@gm...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

Re: [Numpy-discussion] Trouble installing Numpy on AIX 5.2.

[<mf...@ae...>]

Thanks for this observation.  I will modify ufuncobject.h as you suggested,
instead.  The other problem still results in a complaint, but not an error;
it does not prevent compilation.

I have another little problem but I expect to be able to solve it.  I will
report when and if I have Numpy installed.

Mark F. Morss
Principal Analyst, Market Risk
American Electric Power


                                                                           
             Travis Oliphant                                               
             <oliphant.travis@                                             
             ieee.org>                                                  To 
                                       mf...@ae...                     
             02/22/2006 11:29                                           cc 
             AM                        numpy-discussion                    
                                       <num...@li... 
                                       .net>                               
                                                                   Subject 
                                       Re: [Numpy-discussion] Trouble      
                                       installing Numpy on AIX 5.2.        
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




mf...@ae... wrote:

>This problem was solved by adding "#include <fenv.h>" to ...numpy-0.9.5
>/numpy/core/src/umathmodule.c.src
>
>
I suspect this allowed compilation, but I'm not sure if it "solved the
problem."  It depends on whether or not the FE_OVERFLOW defined in
fenv.h is the same as FP_OVERFLOW on the _AIX (it might be...).  The
better solution is to change the constant to what it should be...

Did the long double *,  double * problem also resolve itself?  This
seems to an error with the modfl function you are picking up since the
AIX docs say that modfl should take and receive long double arguments.

Best,

-Travis

Re: [Numpy-discussion] Trouble installing Numpy on AIX 5.2.

[Travis O. <oli...@ie...>]

mf...@ae... wrote:

>This problem was solved by adding "#include <fenv.h>" to ...numpy-0.9.5
>/numpy/core/src/umathmodule.c.src
>  
>
I suspect this allowed compilation, but I'm not sure if it "solved the 
problem."  It depends on whether or not the FE_OVERFLOW defined in 
fenv.h is the same as FP_OVERFLOW on the _AIX (it might be...).  The 
better solution is to change the constant to what it should be...

Did the long double *,  double * problem also resolve itself?  This 
seems to an error with the modfl function you are picking up since the 
AIX docs say that modfl should take and receive long double arguments.

Best,

-Travis

Re: [Numpy-discussion] Trouble installing Numpy on AIX 5.2.

[Travis O. <oli...@ie...>]

mf...@ae... wrote:

>I built Python successfully on our AIX 5.2 server using "./configure
>--without-cxx --disable-ipv6".  (This uses the native IBM C compiler,
>invoking it as "cc_r".  We have no C++ compiler.)
>
>But I have been unable to install Numpy-0.9.5 using the same compiler.
>After "python setup.py install," the relevant section of the output was:
>
>compile options: '-Ibuild/src/numpy/core/src -Inumpy/core/include
>-Ibuild/src/numpy/core -Inumpy/core/src -Inumpy/core/include
>-I/pydirectory/include/python2.4 -c'
>cc_r: build/src/numpy/core/src/umathmodule.c
>"build/src/numpy/core/src/umathmodule.c", line 2734.25: 1506-045 (S)
>Undeclared identifier FE_OVERFLOW.
>  
>
Thanks for this check.   This is an error in the _AIX section of the 
header.  Change line 304 in ufuncobject.h from FE_OVERFLOW to FP_OVERFLOW. 

>"build/src/numpy/core/src/umathmodule.c", line 9307.32: 1506-280 (W)
>Function argument assignment between types "long double*" and "double*" is
>not allowed.
>  
>

I'm not sure where this error comes from.  It seems to appear when modfl 
is used.  What is the content of config.h (in your 
<python-site-packages>/numpy/core/include/numpy directory)?

Can you find out if modfl is defined on your platform already? 

>A closely related question is, how can I modify the Numpy setup.py and/or
>distutils files to enable me to control the options with which cc_r is
>invoked?  I inspected these files, but not being very expert in Python, I
>could not figure this out.
>  
>
The default CFLAGS are those you used to build Python with.  I think you 
can set the CFLAGS environment variable in order to change this. 

Thank you for your test.  I don't have access to _AIX platform and so I 
appreciate your feedback.

-Travis

Re: [Numpy-discussion] Trouble installing Numpy on AIX 5.2.

[<mf...@ae...>]

This problem was solved by adding "#include <fenv.h>" to ...numpy-0.9.5
/numpy/core/src/umathmodule.c.src

Mark F. Morss
Principal Analyst, Market Risk
American Electric Power


                                                                           
             mf...@ae...                                               
             Sent by:                                                      
             numpy-discussion-                                          To 
             ad...@li...         numpy-discussion                    
             eforge.net                <num...@li... 
                                       .net>                               
                                                                        cc 
             02/22/2006 09:06                                              
             AM                                                    Subject 
                                       [Numpy-discussion] Trouble          
                                       installing Numpy on AIX 5.2.        
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




I built Python successfully on our AIX 5.2 server using "./configure
--without-cxx --disable-ipv6".  (This uses the native IBM C compiler,
invoking it as "cc_r".  We have no C++ compiler.)

But I have been unable to install Numpy-0.9.5 using the same compiler.
After "python setup.py install," the relevant section of the output was:

compile options: '-Ibuild/src/numpy/core/src -Inumpy/core/include
-Ibuild/src/numpy/core -Inumpy/core/src -Inumpy/core/include
-I/pydirectory/include/python2.4 -c'
cc_r: build/src/numpy/core/src/umathmodule.c
"build/src/numpy/core/src/umathmodule.c", line 2566.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2584.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2602.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2620.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2638.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2654.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2674.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2694.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2714.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2734.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 9307.32: 1506-280 (W)
Function argument assignment between types "long double*" and "double*" is
not allowed.
"build/src/numpy/core/src/umathmodule.c", line 2566.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2584.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2602.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2620.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2638.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2654.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2674.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2694.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2714.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 2734.25: 1506-045 (S)
Undeclared identifier FE_OVERFLOW.
"build/src/numpy/core/src/umathmodule.c", line 9307.32: 1506-280 (W)
Function argument assignment between types "long double*" and "double*" is
not allowed.
error: Command "cc_r -DNDEBUG -O -Ibuild/src/numpy/core/src
-Inumpy/core/include -Ibuild/src/numpy/core -Inumpy/core/src
-Inumpy/core/include -I/app/sandbox/s625662/installed/include/python2.4 -c
build/src/numpy/core/src/umathmodule.c -o build/temp.aix-5.2-2.4
/build/src/numpy/core/src/umathmodule.o" failed with exit status 1

A closely related question is, how can I modify the Numpy setup.py and/or
distutils files to enable me to control the options with which cc_r is
invoked?  I inspected these files, but not being very expert in Python, I
could not figure this out.

Mark F. Morss
Principal Analyst, Market Risk
American Electric Power



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Numpy-discussion mailing list
Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion