numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] question about nonzero behavior in numarray

[Perry G. <pe...@st...>]

Todd pointed out to me that he's gotten some complaints about
the difference in how numarray nonzero behavior works.
In version 0.9 the use of an array for a truth value was
changed from a deprecation warning to an exception.

We deprecated it because the meaning of an array as a truth
value was ambiguous (or potentially confusing at best)
and we felt it was better to force users to be explicit
about what they wanted. Nevertheless, Numeric does allow that
and removing the capability to use it in that context breaks
some code. 

We can give some thought to reversing that decision, but since
we only hear from those that don't like the new behavior, we'd
like to get some sense if there are many that do like the new
behavior.

To clarify, what happens when:

x = zeros(2)
if x:
    print "true!"

I see 3 possibilities:

1) Keep it the way it is, no use of an array is permitted; an 
   exception is raised.
2) Print a warning message when it is used that way, and always
   keep it at that, i.e., warning message only.
3) Replicate Numeric behavior.

Personally, I prefer 1) since I think there is too much confusion
between whether false means an empty (len=0) array (like lists),
or an array that contains all zeros, and that it is worth breaking
backward compatibility on this (functions should be used to 
distinguish which meaning is desired). 

Perry

[Option 2 on the previous question wins hands down]

RE: [Numpy-discussion] type coercion question

[Nadav H. <na...@vi...>]

(3) seems to be reasonable since Int32-precision > Float32-precision

  Nadav.


-----Original Message-----
From:	Perry Greenfield [mailto:pe...@st...]
Sent:	Wed 31-Mar-04 17:39
To:	David M. Cooke; Todd Miller
Cc:	Gary Ruben; numpy-discussion
Subject:	[Numpy-discussion] type coercion question
David Cooke wrote:
> On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
> > On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > > > >>> cos(1)
> > > > > 0.54030227661132813
> > > > >
> > > > > gives a different result to cos(1.)
> > > <snip>
> > > > >  Do others think precision is being lost unnecessarily?
> > > >
> > > > No.  Do you have any suggestions?
> > >
> > > Yes. My problem is that numarray is not replicating the behaviour =
of
> > > the Python math module, which presumably coerces immediately to =
the
> > > float precision of the underlying platform. This is probably a
> > > float64 in Windows. If you don't explicitly specify that float32 =
is
> > > to be used, shouldn't the default be for a rank-0 value to
> > > immediately coerce ints to the same precision as the native Python
> > > float type on the underlying platform, since you know it will be
> > > coerced to that later and in the meantime you've lost precision
> > > because you've applied a function to a value of lower precision?
> > >
> >
> > I see your point and I'll talk it over with Perry unless he chimes =
in on
> > his own.  I'm ambivalent:  while it would be nice for the numarray
> > ufuncs to act as full equivalents to the Python math functions in =
scalar
> > cases,  numarray has to strike a balance between managing storage =
space
> > and maintaining precision and these two goals are in conflict.  I =
think
> > using an array package you have to be at least a little more aware =
of
> > storage concerns.
>
> I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-)
> Python float. The reason is that I have 'from numarray import *' for =
my
> interactive interpreter, as I use numarray a lot in that.
> I would rather not have to remember to always cast my args to ufuncs =
to
> arrays. This isn't a storage space problem, per se., the question is
> whether cos(1) and cos(1.0) should return the same answer.
>
> Having cos(1) cast 1 to a Float32 makes numarray much less useful as a
> calculator -- mistakes are more likely. I'll probably switch my
> interpreter back to Numeric so as not to make mistakes, or I'll write =
a
> wrapper (which I'll post if I do).
>
> Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is
> different, as there I've taken the conscious decision not to cast to =
an
> array. (And the fact that cos([1,1,1]) will return an array printed =
out
> with 'type=3DFloat32' is a clue to what I've done.)
>
Let me say that I see the last issue (whether cos([1,1,1]) and
cos([1.0,1.0,1.0]) are the same) is important. If cos([1,1,1])[0] !=3D
cos(1) I think that is just as bad. So the question is whether
ints scalars and arrays should automatically be promoted to
double floats when given to unary scientific functions (sqrt,
sin, cos, log, etc.). By the way, if an Int32 is so promoted,
I don't see why it isn't the same for Int8, Int16, etc.
But it appears that Numeric promotes Int32 to Float64 for cos(),
but promotes In16 to Float32 for cos().

I believe making this change to numarray is fairly easy (we'd have
to check). But is what most people want? Should we retain complete
compatibility with Numeric (which, as pointed out, treats different
kinds of integers differently)

Just to summarize what I see as acceptable choices:

1) ints promote to Float32 (current behavior)
2) ints promote to Float64
3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
behavior)

Perry






-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=3D1470&alloc_id=3D3638&op=3Dcli=
ck
_______________________________________________
Numpy-discussion mailing list
Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] type coercion question

[Colin J. W. <cj...@sy...>]

Rick White wrote:

>On Wed, 31 Mar 2004, Perry Greenfield wrote:
>
>  
>
>>Just to summarize what I see as acceptable choices:
>>
>>1) ints promote to Float32 (current behavior)
>>2) ints promote to Float64
>>3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
>>behavior)
>>    
>>
>
>I'm also +1 on option (2).
>

I'm also +1 on option (2).

Colin W.

Re: [Numpy-discussion] type coercion question

[Fernando P. <Fer...@co...>]

Chris Barker wrote:
> On Mar 31, 2004, at 7:39 AM, Perry Greenfield wrote:
> 
>>Just to summarize what I see as acceptable choices:
>>
>>1) ints promote to Float32 (current behavior)
>>2) ints promote to Float64
>>3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
>>behavior)
> 
> 
> +1 on (2)
> 
> I want math.cos(x) to give the same answer as numarray.cos( array( (x) 
> ) ) as much as possible. I like that Numarray does not do as much 
> auto-upcasting as Numeric, but if I want to control the size of my data 
> arrays, I'm not going to be passing an int into a floating point 
> function. i.e, I can always do:
> 
> cos( array( (x), Float32 ) )
> 
> Then I would want the result to be a Float32

+1 on (2) as well, based on a similar reasoning to Chris (who saved me the 
time of typing it up cleanly :)

Cheers,

f

Re: [Numpy-discussion] type coercion question

[Rick W. <rl...@st...>]

On Wed, 31 Mar 2004, Perry Greenfield wrote:

> Just to summarize what I see as acceptable choices:
>
> 1) ints promote to Float32 (current behavior)
> 2) ints promote to Float64
> 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
> behavior)

I'm also +1 on option (2).

I believe that this will not cause problems with unwanted promotions.  The
result of cos(1) is a scalar Python float, regardless of whether 1 is
converted to a Float32 or a Float64 before the cosine is evaluated.  Then
if that scalar is used with a Float32 numarray, it gets converted to a
Float32 and the result is Float32.  For example:

   >>> from numarray import *
   >>> cos(1)*ones(1,Float32)
   array([ 0.54030228], type=Float32)
   >>> cos(1.0)*ones(1,Float32)
   array([ 0.54030228], type=Float32)

So it seems to me there is no downside to converting integers to Float64
when the ufunc requires a float.  And I find it hard to believe that any
Numeric users are counting on behavior (3).
				Rick

Re: [Numpy-discussion] type coercion question

[Chris B. <Chr...@no...>]

On Mar 31, 2004, at 7:39 AM, Perry Greenfield wrote:
> Just to summarize what I see as acceptable choices:
>
> 1) ints promote to Float32 (current behavior)
> 2) ints promote to Float64
> 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
> behavior)

+1 on (2)

I want math.cos(x) to give the same answer as numarray.cos( array( (x) 
) ) as much as possible. I like that Numarray does not do as much 
auto-upcasting as Numeric, but if I want to control the size of my data 
arrays, I'm not going to be passing an int into a floating point 
function. i.e, I can always do:

cos( array( (x), Float32 ) )

Then I would want the result to be a Float32

-Chris


Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] type coercion question

[Perry G. <pe...@st...>]

David Cooke wrote:
> On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
> > On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > > > >>> cos(1)
> > > > > 0.54030227661132813
> > > > >
> > > > > gives a different result to cos(1.)
> > > <snip>
> > > > >  Do others think precision is being lost unnecessarily?
> > > >
> > > > No.  Do you have any suggestions?
> > >
> > > Yes. My problem is that numarray is not replicating the behaviour of
> > > the Python math module, which presumably coerces immediately to the
> > > float precision of the underlying platform. This is probably a
> > > float64 in Windows. If you don't explicitly specify that float32 is
> > > to be used, shouldn't the default be for a rank-0 value to
> > > immediately coerce ints to the same precision as the native Python
> > > float type on the underlying platform, since you know it will be
> > > coerced to that later and in the meantime you've lost precision
> > > because you've applied a function to a value of lower precision?
> > >
> >
> > I see your point and I'll talk it over with Perry unless he chimes in on
> > his own.  I'm ambivalent:  while it would be nice for the numarray
> > ufuncs to act as full equivalents to the Python math functions in scalar
> > cases,  numarray has to strike a balance between managing storage space
> > and maintaining precision and these two goals are in conflict.  I think
> > using an array package you have to be at least a little more aware of
> > storage concerns.
>
> I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-)
> Python float. The reason is that I have 'from numarray import *' for my
> interactive interpreter, as I use numarray a lot in that.
> I would rather not have to remember to always cast my args to ufuncs to
> arrays. This isn't a storage space problem, per se., the question is
> whether cos(1) and cos(1.0) should return the same answer.
>
> Having cos(1) cast 1 to a Float32 makes numarray much less useful as a
> calculator -- mistakes are more likely. I'll probably switch my
> interpreter back to Numeric so as not to make mistakes, or I'll write a
> wrapper (which I'll post if I do).
>
> Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is
> different, as there I've taken the conscious decision not to cast to an
> array. (And the fact that cos([1,1,1]) will return an array printed out
> with 'type=Float32' is a clue to what I've done.)
>
Let me say that I see the last issue (whether cos([1,1,1]) and
cos([1.0,1.0,1.0]) are the same) is important. If cos([1,1,1])[0] !=
cos(1) I think that is just as bad. So the question is whether
ints scalars and arrays should automatically be promoted to
double floats when given to unary scientific functions (sqrt,
sin, cos, log, etc.). By the way, if an Int32 is so promoted,
I don't see why it isn't the same for Int8, Int16, etc.
But it appears that Numeric promotes Int32 to Float64 for cos(),
but promotes In16 to Float32 for cos().

I believe making this change to numarray is fairly easy (we'd have
to check). But is what most people want? Should we retain complete
compatibility with Numeric (which, as pointed out, treats different
kinds of integers differently)

Just to summarize what I see as acceptable choices:

1) ints promote to Float32 (current behavior)
2) ints promote to Float64
3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
behavior)

Perry

Re: [Numpy-discussion] blitz++ and numarray

[Fernando P. <Fer...@co...>]

> Thanks. That is very helpful. I've incorporated this into my code, and
> managed to get a simple example with boost working (extracting an element
> of an array). See below.

Glad it helped.

> Do you have a similar function which converts a blitz array to a
> numarray/Numeric one? If not, I can easily concoct one for myself.

No, I don't have one.  But it's pretty straightforward to write one, the 
Numeric C API provides functions to create arrays from existing data.  You'll 
be surprised how conceptually similar Blitz and Numeric turn out to be (which 
is a good thing).

> I now need to try to figure out what this reference counting stuff is
> about. I read the official Python docs but am still currently very
> confused.

As I said, so far I've been able to bypass this issue by creating my arrays in 
python and 'filling' them in blitz.  But this is not always a realistic 
possibility, so you may indeed have to learn the reference counting management 
details.

Best,

f

Re: [Numpy-discussion] blitz++ and numarray

[Faheem M. <fa...@em...>]

On Mon, 29 Mar 2004, Fernando Perez wrote:

> The following should be enough to get you started:
>
> #include "Python.h"
> #include "Numeric/arrayobject.h"
> #include "blitz/array.h"
>
> using namespace std;
> using namespace blitz;
>
> // Convert a Numpy array to a blitz one, using the original's data (no copy)
> template<class T, int N>
> static Array<T,N> py_to_blitz(PyArrayObject* arr_obj)
> {
>      int T_size = sizeof(T);
>      TinyVector<int,N> shape(0);
>      TinyVector<int,N> strides(0);
>      int *arr_dimensions = arr_obj->dimensions;
>      int *arr_strides = arr_obj->strides;
>
>      for (int i=0;i<N;++i) {
>          shape[i]   = arr_dimensions[i];
>          strides[i] = arr_strides[i]/T_size;
>      }
>      return Array<T,N>((T*) arr_obj->data,shape,strides,neverDeleteData);
> }
>
> This is what I use for exactly the problem you are describing, and this code
> was pretty much lifted, with minor changes, from weave's auto-generated C++.
> What I do, to sidestep the memory management problems, is let python allocate
> all my Numeric arrays (using zeros() if I have no data).  I then use those
> inside my C++ code as blitz++ arrays via the above snippet.  Any changes made
> by the C++ code are automatically reflected in the Numeric arary, since the
> blitz object is using the Numeric data area.
>
> I hope this helps.  Let me know if you need more help, I can mail you complete
> example code.

Thanks. That is very helpful. I've incorporated this into my code, and
managed to get a simple example with boost working (extracting an element
of an array). See below.

Do you have a similar function which converts a blitz array to a
numarray/Numeric one? If not, I can easily concoct one for myself.

I now need to try to figure out what this reference counting stuff is
about. I read the official Python docs but am still currently very
confused.

                                                           Faheem.

***************************************************************************
#include "Python.h"
#include "Numeric/arrayobject.h"
#include <blitz/array.h>

using namespace std;
using namespace blitz;

template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj);

static PyObject * arrmod_elem(PyObject *self, PyObject *args);

static PyMethodDef arrmod_methods[] =
  {
    {"elem", (PyCFunction)arrmod_elem, METH_VARARGS, "Returns the trace of a two-dimensional array.\n"},
    {NULL, NULL, 0, NULL}
  };

PyMODINIT_FUNC initarrmod(void)
{
  (void) Py_InitModule3("arrmod", arrmod_methods, "Returns the Trace of a two-dimensional array.\n");
  import_array();
}

static PyObject * arrmod_elem(PyObject *self, PyObject *args)
{
  PyObject *input, *result;
  PyArrayObject *array;
  double el;
  int i, j;

  if (!PyArg_ParseTuple(args, "Oii", &input, &i, &j))
    return NULL;
  array = (PyArrayObject *)
    PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2);
  if (array == NULL)
    return NULL;

  Array<double,2> arr = py_to_blitz<double,2>(array);

  el = arr(i,j);
  result = Py_BuildValue("d",el);
  return result;
}

// Convert a Numpy array to a blitz one, using the original's data (no
// copy)
template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj)
{
  int T_size = sizeof(T);
  TinyVector<int,N> shape(0);
  TinyVector<int,N> strides(0);
  int *arr_dimensions = arr_obj->dimensions;
  int *arr_strides = arr_obj->strides;

  for (int i=0;i<N;++i)
    {
      shape[i]   = arr_dimensions[i];
      strides[i] = arr_strides[i]/T_size;
    }
  return Array<T,N>((T*) arr_obj->data,shape,strides,neverDeleteData);
}

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[David M. C. <co...@ph...>]

On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
> On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > > >>> cos(1)
> > > > 0.54030227661132813
> > > > 
> > > > gives a different result to cos(1.)
> > <snip>
> > > >  Do others think precision is being lost unnecessarily?
> > > 
> > > No.  Do you have any suggestions?
> > 
> > Yes. My problem is that numarray is not replicating the behaviour of
> > the Python math module, which presumably coerces immediately to the
> > float precision of the underlying platform. This is probably a
> > float64 in Windows. If you don't explicitly specify that float32 is
> > to be used, shouldn't the default be for a rank-0 value to
> > immediately coerce ints to the same precision as the native Python
> > float type on the underlying platform, since you know it will be
> > coerced to that later and in the meantime you've lost precision
> > because you've applied a function to a value of lower precision?
> > 
> 
> I see your point and I'll talk it over with Perry unless he chimes in on
> his own.  I'm ambivalent:  while it would be nice for the numarray
> ufuncs to act as full equivalents to the Python math functions in scalar
> cases,  numarray has to strike a balance between managing storage space
> and maintaining precision and these two goals are in conflict.  I think
> using an array package you have to be at least a little more aware of
> storage concerns.  

I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-)
Python float. The reason is that I have 'from numarray import *' for my
interactive interpreter, as I use numarray a lot in that.
I would rather not have to remember to always cast my args to ufuncs to
arrays. This isn't a storage space problem, per se., the question is
whether cos(1) and cos(1.0) should return the same answer.

Having cos(1) cast 1 to a Float32 makes numarray much less useful as a
calculator -- mistakes are more likely. I'll probably switch my
interpreter back to Numeric so as not to make mistakes, or I'll write a
wrapper (which I'll post if I do).

Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is
different, as there I've taken the conscious decision not to cast to an
array. (And the fact that cos([1,1,1]) will return an array printed out
with 'type=Float32' is a clue to what I've done.)

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand apossible bug

[Gary R. <ga...@em...>]

Thanks for pointing this out Alexander,
I admit that I hadn't tried any native Python container types to see how they behave. I was basing my comments on
Numeric's behaviour. When I have time, maybe I should raise a question on the Python developer's list. It seems like something that may have been overlooked.

regards,
Gary

----- Original Message -----
From: Alexander Schmolck <a.s...@gm...>
Date: Mon, 29 Mar 2004 13:52:20 +0100
To: "numpy-discussion" <num...@li...>
Subject: Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand apossible bug

> "Gary Ruben" <ga...@em...> writes:
> > In summary, I think te behaviour should be
> >>>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
> > [909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]
> >
> >>>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
> > ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
> 
> Such behavior would seem sensible, but I don't think it's what the builtin
> collections do (not sure why):
> 
> >>> n = 1.1
> >>> n
> 1.1000000000000001
> >>> [n]
> [1.1000000000000001]
> >>> str(n)
> '1.1'
> >>> str([n])
> '[1.1000000000000001]'
> 
> 
> 'as


-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm

Re: [Numpy-discussion] blitz++ and numarray

[Fernando P. <Fer...@co...>]

Faheem Mitha wrote:
> Hi,
> 
> I'm considering the use of the Blitz++ C++ library
> (http://www.oonumerics.org/blitz/) along with numarray/Numeric. I want
> to pass an array down to C++ and then manipulate it using Blitz++.

[snip]

> Also, weave.blitz() from SciPy looks relevant/useful, but this
> automatically generates its own C++ code, and I'd prefer to write my
> own.

The following should be enough to get you started:

#include "Python.h"
#include "Numeric/arrayobject.h"
#include "blitz/array.h"

using namespace std;
using namespace blitz;

// Convert a Numpy array to a blitz one, using the original's data (no copy)
template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj)
{
     int T_size = sizeof(T);
     TinyVector<int,N> shape(0);
     TinyVector<int,N> strides(0);
     int *arr_dimensions = arr_obj->dimensions;
     int *arr_strides = arr_obj->strides;

     for (int i=0;i<N;++i) {
         shape[i]   = arr_dimensions[i];
         strides[i] = arr_strides[i]/T_size;
     }
     return Array<T,N>((T*) arr_obj->data,shape,strides,neverDeleteData);
}

This is what I use for exactly the problem you are describing, and this code 
was pretty much lifted, with minor changes, from weave's auto-generated C++. 
What I do, to sidestep the memory management problems, is let python allocate 
all my Numeric arrays (using zeros() if I have no data).  I then use those 
inside my C++ code as blitz++ arrays via the above snippet.  Any changes made 
by the C++ code are automatically reflected in the Numeric arary, since the 
blitz object is using the Numeric data area.

I hope this helps.  Let me know if you need more help, I can mail you complete 
example code.

Cheers,

f

[Numpy-discussion] blitz++ and numarray

[Faheem M. <fa...@em...>]

Hi,

I'm considering the use of the Blitz++ C++ library
(http://www.oonumerics.org/blitz/) along with numarray/Numeric. I want
to pass an array down to C++ and then manipulate it using Blitz++.

My current thinking is to use the Numeric interface to pass down the
array, and then copy it to a blitz++ array object. This seems
straightforward except I am a little concerned about those reference
counting issues, which I don't properly understand (I'm a Python
beginner). I do get the impression that it is mainly an issue if I am
working with Python objects in C/C++ code, though, and I am mostly
going to try to avoid doing so by declaring C++ objects within the C++
code and working with those as much as possible.

Other possibilities include somehow using the Boost.Python package,
though I am not sure how. One advantage of this is that it apparently
takes care of the reference counting issue. However, I don't want to
learn yet another API.

Also, weave.blitz() from SciPy looks relevant/useful, but this
automatically generates its own C++ code, and I'd prefer to write my
own.

Can anyone suggested a good basic tutorial which discusses the issues
of reference counting in a C/C++ API context? I have tried reading the
official documentation and will continue to do so, but something
gentler would be useful.

Comments, suggestions gratefully appreciated. Please cc me on any
reply to fa...@em.... Thanks in advance.

                                                    Faheem.

Re: [Numpy-discussion] numarray C api examples

[Todd M. <jm...@st...>]

On Mon, 2004-03-29 at 11:25, Craig Rasmussen wrote:
> On Mar 27, 2004, at 2:49 PM, Todd Miller wrote:
> 
> > To install the examples, just install numarray.  The examples are then
> > found in the package numarray.examples.convolve.  If you want to write
> > your own extension,  my advice is to stick with the numpy compatible
> > interfaces.  A close second is the numarray "high level" interface.
> 
> Does numarray have the equivalent of PyArray_FromDimsAndData()?

numarray's Numeric compatibility layer has that function.  Because
numarray uses buffer objects, numarray winds up making a copy of the
data.  This behavior could be changed but hasn't been yet.  

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] numarray C api examples

[Craig R. <cra...@la...>]

On Mar 27, 2004, at 2:49 PM, Todd Miller wrote:

> To install the examples, just install numarray.  The examples are then
> found in the package numarray.examples.convolve.  If you want to write
> your own extension,  my advice is to stick with the numpy compatible
> interfaces.  A close second is the numarray "high level" interface.

Does numarray have the equivalent of PyArray_FromDimsAndData()?

This function is critical for the work I'm doing on Co-Array
Python (because it is essentially a distributed array module with
Co-Array syntax, it must use memory allocated by C) and for using
Fortran memory for Numeric or numarray arrays.

Regards,
Craig

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Alexander S. <a.s...@gm...>]

"Gary Ruben" <ga...@em...> writes:
> In summary, I think te behaviour should be
>>>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
> [909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]
>
>>>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
> ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])

Such behavior would seem sensible, but I don't think it's what the builtin
collections do (not sure why):

>>> n = 1.1
>>> n
1.1000000000000001
>>> [n]
[1.1000000000000001]
>>> str(n)
'1.1'
>>> str([n])
'[1.1000000000000001]'


'as

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Todd M. <jm...@st...>]

On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > >>> cos(1)
> > > 0.54030227661132813
> > > 
> > > gives a different result to cos(1.)
> <snip>
> > >  Do others think precision is being lost unnecessarily?
> > 
> > No.  Do you have any suggestions?
> 
> Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?
> 

I see your point and I'll talk it over with Perry unless he chimes in on
his own.  I'm ambivalent:  while it would be nice for the numarray
ufuncs to act as full equivalents to the Python math functions in scalar
cases,  numarray has to strike a balance between managing storage space
and maintaining precision and these two goals are in conflict.  I think
using an array package you have to be at least a little more aware of
storage concerns.  

> > > Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?
> > > 
> > 
> > Yes.  The basic look is exactly what I wanted:

Don't ask how, but I just noticed that my foot has nearly the same
dimensions of my mouth.

> > 
> > >>> a = numarray.objects.fromlist(range(10))
> > >>> str(a)
> > '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]'
> > >>> repr(a)
> > 'ObjectArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'
> > 
> > But I'm open to suggestions and recognize that the handling of
> > spaces/item sizing is weak (although it's not pronounced in this simple
> > example).  Do people want something else?
> > 
> > Regards,
> > Todd
> 
> Your example doesn't demonstrate the problem I'm talking about because for integers the __str__ and __repr__ methods are identical. However, it does show a difference between Numeric and numarray, which would display your str(a) as
> '[0  1  2  3  4  5  6  7  8  9 ]'
> 

Well, almost what I wanted.  This will change.

> If you make an ObjectArray containing objects whose __repr__ and __str__ methods are not the same, I think you need a way of displaying the contained objects via their type-specific __str__ method. Numeric did this by default, whereas numarray has changed that behaviour.

This will change too.

> I'll go back to my example again.
> In my example str() does not display the array elements in the same way that Numeric does.
> In my case, numarray does this:
> 
> >>> print ArrayOfErr([909., 802., 677., 585.], 1.0)
> [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
> 
> >>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
> [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
> 
> >>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
> ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
> 
> 
> ie. str() is not using the __str__ method I have defined for my Err class.
> Numeric correctly uses my class's Err.__str__ method and does this instead:
> 
> >>> print ArrayOfErr([909., 802., 677., 585.], 1.0)
> [909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]
> 
> In summary, I think te behaviour should be
> >>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
> [909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]
> 
> >>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
> ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
> 
> I hope I've clarified what I'm getting at, although I probably haven't :-)

Quite clear.   Stuff will change.

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Gary R. <ga...@em...>]

> > >>> cos(1)
> > 0.54030227661132813
> > 
> > gives a different result to cos(1.)
<snip>
> >  Do others think precision is being lost unnecessarily?
> 
> No.  Do you have any suggestions?

Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?

> > Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?
> > 
> 
> Yes.  The basic look is exactly what I wanted:
> 
> >>> a = numarray.objects.fromlist(range(10))
> >>> str(a)
> '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]'
> >>> repr(a)
> 'ObjectArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'
> 
> But I'm open to suggestions and recognize that the handling of
> spaces/item sizing is weak (although it's not pronounced in this simple
> example).  Do people want something else?
> 
> Regards,
> Todd

Your example doesn't demonstrate the problem I'm talking about because for integers the __str__ and __repr__ methods are identical. However, it does show a difference between Numeric and numarray, which would display your str(a) as
'[0  1  2  3  4  5  6  7  8  9 ]'

If you make an ObjectArray containing objects whose __repr__ and __str__ methods are not the same, I think you need a way of displaying the contained objects via their type-specific __str__ method. Numeric did this by default, whereas numarray has changed that behaviour.

I'll go back to my example again.
In my example str() does not display the array elements in the same way that Numeric does.
In my case, numarray does this:

>>> print ArrayOfErr([909., 802., 677., 585.], 1.0)
[Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]

>>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
[Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]

>>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])


ie. str() is not using the __str__ method I have defined for my Err class.
Numeric correctly uses my class's Err.__str__ method and does this instead:

>>> print ArrayOfErr([909., 802., 677., 585.], 1.0)
[909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]

In summary, I think te behaviour should be
>>> print str(ArrayOfErr([909., 802., 677., 585.], 1.0))
[909.0 +1.0/-1.0  802.0 +1.0/-1.0  677.0 +1.0/-1.0  585.0 +1.0/-1.0 ]

>>> print repr(ArrayOfErr([909., 802., 677., 585.], 1.0))
ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])

I hope I've clarified what I'm getting at, although I probably haven't :-)

Gary
-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Todd M. <jm...@st...>]

On Sun, 2004-03-28 at 19:46, Gary Ruben wrote:
> > I'm guessing,  but I think you would like to see:
> > 
> > >>> abs(-1.)
> > 1.0
> > >>> abs(1)
> > 1
> 
> Hi Todd,
> 
> Sorry if I wasn't clear. Yes, that's what I was expecting to see. Thanks for logging it and thanks for the explanation of why it's happening.
> 
> I'd also like to restate the other issues in the email to which that one referred, which haven't been commented on. The cos function result
> 
> >>> cos(1)
> 0.54030227661132813
> 
> gives a different result to cos(1.)
> Is this just because of numarray's stated departures from the way type coercion is done in Numeric? My guess is that numarray is casting the integer to a rank-0 Float32 array which is then coerced back to the native Python float type, which presumably is a float64 on the Windows platform.

This sounds correct.  (Thanks for explaining it!)

>  Do others think precision is being lost unnecessarily?

No.  Do you have any suggestions?

> Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?
> 

Yes.  The basic look is exactly what I wanted:

>>> a = numarray.objects.fromlist(range(10))
>>> str(a)
'[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]'
>>> repr(a)
'ObjectArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'

But I'm open to suggestions and recognize that the handling of
spaces/item sizing is weak (although it's not pronounced in this simple
example).  Do people want something else?

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Gary R. <ga...@em...>]

> I'm guessing,  but I think you would like to see:
> 
> >>> abs(-1.)
> 1.0
> >>> abs(1)
> 1

Hi Todd,

Sorry if I wasn't clear. Yes, that's what I was expecting to see. Thanks for logging it and thanks for the explanation of why it's happening.

I'd also like to restate the other issues in the email to which that one referred, which haven't been commented on. The cos function result

>>> cos(1)
0.54030227661132813

gives a different result to cos(1.)
Is this just because of numarray's stated departures from the way type coercion is done in Numeric? My guess is that numarray is casting the integer to a rank-0 Float32 array which is then coerced back to the native Python float type, which presumably is a float64 on the Windows platform. Do others think precision is being lost unnecessarily?

Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?

thanks,
Gary

-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm

Re: [Numpy-discussion] numarray C api examples

[Todd M. <jm...@st...>]

On Sun, 2004-03-28 at 10:54, Faheem Mitha wrote:
> On Sat, 27 Mar 2004, Todd Miller wrote:
> 
> > On Sat, 2004-03-27 at 12:35, Faheem Mitha wrote:
> > > The README in numarray-0.9/Examples refers to the non-existent file
> > > setup.py (at least in that directory). There are other files called
> > > setup.py elsewhere in the tarball, but they don't look like they are
> > > supposed to compile the examples.
> > >
> > > What am I missing?
> >
> > Nothing.  Those are dead comments in the README (they didn't survive the
> > "numarray re-packaging").  There is now a single setup.py for all
> > numarray packages in the root numarray directory.
> 
> Ok. In that case changing the README would be a good idea in order to
> avoid confusion.

Sure.

> > To install the examples, just install numarray.  The examples are then
> > found in the package numarray.examples.convolve.
> 
> Ok. I see the examples are in the Debian package
> python2.3-numarray-ext.

I don't use debian anymore so I'm not sure what goes on there.  I'm glad
some of the debian guys are making numarray available.

> 
> > If you want to write your own extension, my advice is to stick with
> > the numpy compatible interfaces.  A close second is the numarray
> > "high level" interface.
> 
> I see. You mean use
> 
> #include "arrayobject.h"
> 
> right?

Exactly.  Using the numpy compatible API also keeps you close to Numeric
in case small array performance ever becomes an issue for you and you
decide you want to switch back to Numeric.  It might also let others who
are using Numeric use your extensions even if you choose to use numarray
yourself.

> 
> BTW, the high level API example in convolve uses
> 
> #include "libnumarray.h"
> 
> even though the documentation refers to
> 
> #include "numarray.h"
> 

Thanks for the info.  Obviously this evolved over time and the docs
didn't.

> Which of these should be used? 

When in doubt,  go with what the example code is doing (in the source
distribution), so libnumarray.h.  If you see something questionable,
ask.

> Actually, it might be an idea to state
> explicitly in 12.3-12.6 which header files should be use. Possibly
> include them in the example code itself.

There is working example code in the source distribution.  At one point,
the example code in the manual was working as well; that's where it came
from.  

> 
> I'm considering just using the Numeric C API on the assumption that is
> will be easier to use (since I am a beginner). Is this assumption
> accurate?

I think they're both fairly easy,  but the Numeric API as the advantage
of backward compatibility.  

> Thanks for all the help.

Thanks for all the feedback.

Regards,
Todd

>                                                          Faheem.
> 
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] numarray C api examples

[Faheem M. <fa...@em...>]

On Sat, 27 Mar 2004, Todd Miller wrote:

> On Sat, 2004-03-27 at 12:35, Faheem Mitha wrote:
> > The README in numarray-0.9/Examples refers to the non-existent file
> > setup.py (at least in that directory). There are other files called
> > setup.py elsewhere in the tarball, but they don't look like they are
> > supposed to compile the examples.
> >
> > What am I missing?
>
> Nothing.  Those are dead comments in the README (they didn't survive the
> "numarray re-packaging").  There is now a single setup.py for all
> numarray packages in the root numarray directory.

Ok. In that case changing the README would be a good idea in order to
avoid confusion.

> To install the examples, just install numarray.  The examples are then
> found in the package numarray.examples.convolve.

Ok. I see the examples are in the Debian package
python2.3-numarray-ext.

> If you want to write your own extension, my advice is to stick with
> the numpy compatible interfaces.  A close second is the numarray
> "high level" interface.

I see. You mean use

#include "arrayobject.h"

right?

BTW, the high level API example in convolve uses

#include "libnumarray.h"

even though the documentation refers to

#include "numarray.h"

Which of these should be used? Actually, it might be an idea to state
explicitly in 12.3-12.6 which header files should be use. Possibly
include them in the example code itself.

I'm considering just using the Numeric C API on the assumption that is
will be easier to use (since I am a beginner). Is this assumption
accurate?

Thanks for all the help.
                                                         Faheem.

Re: [Numpy-discussion] indexing question

[Todd M. <jm...@st...>]

On Sat, 2004-03-27 at 23:47, Faheem Mitha wrote:
> On Sat, 27 Mar 2004, Todd Miller wrote:
> 
> > On Sat, 2004-03-27 at 19:27, Faheem Mitha wrote:
> > > On Sat, 27 Mar 2004, Todd Miller wrote:
> > >
> > > > Try foo[[0,2]].  Note that the array indexing syntax is a little picky:
> > > > foo[(0,2)] (tuple index) won't work,  just foo[[0,2]] (list index) or
> > > > foo[array([0,2])] (array index).
> > >
> > > Thanks. I see this extracts the relevant rows. How about columns?
> >
> > I think for columns you need to transpose the matrix or use take() with
> > the right axis=1.
> 
> I see. Thanks. I think that take() does what I need. BTW, this
> documentation example for take looks wrong, and gives an error for me.
> 
> *************************************************
> >>> a1 = array([10,20,30,40])
> >>> print a1[[3,5]]
> [40 60]
> >>> print take(a1,[3,5])
> [40 60]
> **************************************************
> 

Yes, you're right, that's an error.

> I get
> 
> >>> a1 = numarray.array([10,20,30,40])
> >>> a1
> array([10, 20, 30, 40])
> >>> numarray.take(a1,[3,5])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1776, in
> take
>     return array._take((indices,), outarr=outarr, clipmode=clipmode)
>   File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line
> 498, in _take
>     return ufunc._take(self, indices, **keywds)
>   File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1579, in
> __call__
>     result = self._doit(computation_mode, woutarr, cfunc, ufargs, 0)
>   File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1551, in
> _doit
>     apply(cfunc, ufargs)
> IndexError: Index[1,0]=5 out of range[4]
> 
> >>> a1[[3,5]]
> array([40, 40])

This too looks like an error.  Both usages should raise an exception...
and hopefully a prettier one in the future.

Thanks for the questions and feedback.

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug

[Todd M. <jm...@st...>]

On Sat, 2004-03-27 at 01:48, Gary Ruben wrote:
> Actually, this reproduces the problem:
> 
> >>> from numarray import *
> >>> from numarray.objects import *
> >>> abs(-1.)
> ObjectArray(1.0)
> >>> abs(-1)
> ObjectArray(1)
> 

I'm guessing,  but I think you would like to see:

>>> abs(-1.)
1.0
>>> abs(1)
1

If so,  this is a bug.  Two things are going on:

1)  numarray.objects redefines abs()

2) Internally, numarray has a concept of "rank-0" arrays which it uses
to represent scalars.  Apparently,  numarray.objects lacks the extra
code to filter rank-0 arrays out (convert back to scalars) as is done
with numerical arrays.

I logged this as a bug on Source Forge and it will be fixed for 1.0.

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] indexing question

[Faheem M. <fa...@em...>]

On Sat, 27 Mar 2004, Todd Miller wrote:

> On Sat, 2004-03-27 at 19:27, Faheem Mitha wrote:
> > On Sat, 27 Mar 2004, Todd Miller wrote:
> >
> > > Try foo[[0,2]].  Note that the array indexing syntax is a little picky:
> > > foo[(0,2)] (tuple index) won't work,  just foo[[0,2]] (list index) or
> > > foo[array([0,2])] (array index).
> >
> > Thanks. I see this extracts the relevant rows. How about columns?
>
> I think for columns you need to transpose the matrix or use take() with
> the right axis=1.

I see. Thanks. I think that take() does what I need. BTW, this
documentation example for take looks wrong, and gives an error for me.

*************************************************
>>> a1 = array([10,20,30,40])
>>> print a1[[3,5]]
[40 60]
>>> print take(a1,[3,5])
[40 60]
**************************************************

I get

>>> a1 = numarray.array([10,20,30,40])
>>> a1
array([10, 20, 30, 40])
>>> numarray.take(a1,[3,5])
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1776, in
take
    return array._take((indices,), outarr=outarr, clipmode=clipmode)
  File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line
498, in _take
    return ufunc._take(self, indices, **keywds)
  File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1579, in
__call__
    result = self._doit(computation_mode, woutarr, cfunc, ufargs, 0)
  File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 1551, in
_doit
    apply(cfunc, ufargs)
IndexError: Index[1,0]=5 out of range[4]

>>> a1[[3,5]]
array([40, 40])

                                                               Faheem.