numpy-discussion — Discussion list for all users of Numerical Python

RE: [Numpy-discussion] rank-0 arrays

[eric j. <er...@en...>]

Hey Konrad,

> 
> "eric jones" <er...@en...> writes:
> 
> > Reductions and indexing return different types based on the number
of
> > dimensions of the input array:
> >
> > >>> b = sum(a)
> > >>> l = len(b) # or whatever
> >
> > This code works happily if "a" is 2 or more dimensions, but will
fail if
> > it is 1d because the sum(a) will return a scalar in this case.  To
write
> 
> And it should fail, because a rank-0 array is not a sequence, so it
> doesn't have a length.
> 

I disagree. You should not have to write special code to check for a
specific case.  It breaks one of the beauties of Numeric -- i.e. you can
write generic code that handles arrays of any size and type.  Any method
that works on a 1 or more d array should also work on 0d arrays.  If you
ask for its shape, it returns a tuple.  If you ask for its size it
returns its length along its "first" axis.  This will always be 1.  It
allows for generic code.

On this note:
I do not see the benefit of making a scalar type object that is separate
for 0d arrays.  It seems to remove instead of enhance capabilities.
What does a scalar object buy that simply using 0d arrays for that
purpose does not?

> 
> But there are valid examples in which it would be nice if scalars
> were arrays (but probably if  *all* scalars supported array
operations,
> not just those that were generated by indexing from arrays):
> 
> - a.shape  should return () for a scalar (and (len(a),) for any
>            sequence type)
> 
> - a.astype(N.Float) should also work for scalars
> 
> Similarly, it would be nice if complex operations (real/imaginary
> part) would work on integers and floats.

Yes, this is needed.  And I think the argument for it is similar as
having len() work on 0d arrays.  It allows for generic code.

> 
> There's one more annoying difference between scalars and arrays of
> any rank which I think should be removed in numarray:
> 
>   >>> 3 % -2
>   -1
>   >>> array(3) % 2
>   1
>   >>> fmod(3, -2)
>   1.0
> 
> I.e. the mod operation uses fmod() for arrays, but different rules
> for standard Python numbers.

I think you meant, 

>   >>> array(3) % -2
>   1

That is unfortunate.  It would be nice to clean this up.

eric

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

"Perry Greenfield" <pe...@st...> writes:

> Is there any other compelling reason for new scalar types?

It solves the problem we are discussing (extracting array elements)
without introducing any incompatibility with existing code that uses
only the standard Python data types, and very little for code that
uses Float32 etc. I'd say it's the best compromise between
consistency and backwards compatibility.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

"eric jones" <er...@en...> writes:

> Reductions and indexing return different types based on the number of
> dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write

And it should fail, because a rank-0 array is not a sequence, so it
doesn't have a length.

But there are valid examples in which it would be nice if scalars
were arrays (but probably if  *all* scalars supported array operations,
not just those that were generated by indexing from arrays):

- a.shape  should return () for a scalar (and (len(a),) for any
           sequence type)

- a.astype(N.Float) should also work for scalars

Similarly, it would be nice if complex operations (real/imaginary
part) would work on integers and floats.

There's one more annoying difference between scalars and arrays of
any rank which I think should be removed in numarray:

  >>> 3 % -2
  -1
  >>> array(3) % 2
  1
  >>> fmod(3, -2)
  1.0

I.e. the mod operation uses fmod() for arrays, but different rules
for standard Python numbers.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] Tensor-like product with extra indices?

[Huaiyu Z. <hua...@ya...>]

On 13 Sep 2002, Konrad Hinsen wrote:
> 
> Untested, but the idea should work:
> 
>   def ravel_index(array, *indices):
>      offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
>      return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

Aagh!  :-)

> 
> > 2. Suppose the above function is called ravel_index and its
> >    inverse unravel_index.
> > 
> >    Given a list of matrices [A0, A1, ...] is there a function that
> >    calculates a matrix B such that
> > 
> >         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> > 
> >    where i = ravel_index(i0, i1, ...).
> 
> Not for all i, which I assume is what you want to do.

I end up using two for-loops, one over j and the other over [A0, ...].  
Hoping the speed problem will not be too bad later on.

Huaiyu

-- 
Huaiyu Zhu <hua...@ya...>

RE: [Numpy-discussion] rank-0 arrays

[Travis O. <oli...@ee...>]

> Hi Travis,
>
> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
>
> >>> x = Int8(22)
> >>> arr = arange(10, typecode = Int8)
> >>> arr[2]
> Int8(2)
>
> Or some other approach?

This is the gist of it.  Basically you extend the Python scalars to
include the single precision types (all the types Numeric supports).

Ultimately, in Python it would be nice if all of the scalars had the same
base class.

>
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

I would have to understand his reason better.  He could be right.  My
reason for rank-0 results as always been more of a type issue.  I'll have
to ask him.

Now I remmber an issue that makes me question the proposal:

Currently the length of a rank-0 array is 1, while the length of a scalar
raises an error --- this is a bad difference.

We could still implement rank-0 arrays as a separate (optimized) type
though (so it doesn't carry around the extra baggage of full-rank arrays).

Now-wanting-rank-0-arrays-all-the-time,

-Travis

Re: [Numpy-discussion] rank-0 arrays

[Tim H. <tim...@ie...>]

Thanks for the summary Eric.

However, I don't find this example particularly compelling. Probably because
I don't think len(b) should work if b is rank zero (or possibly return
None). Yeah, I know it's worked in NumPy for years, but that doesn't mean I
have to like it. I would favor stripping out as much of the weird special
casing of rank-0 arrays as possible in the transition to numarray.

In the particular example given below, isn't the return value meaningless if
b is rank-0. Or rather only meaningful if  "or whatever you need" happens to
be one. This all smells rather arbitrary to me. However, this isn't a
problem that I run into and I'll concede that it's possible that returning
rank[0] arrays and then treating the rank zeros arrays almost like shape
(1,) arrays most of the time may solve more problems than it causes, but I'd
be interested in seeing more realistic examples. I guess I'll go poke around
MA and see what I can see. Any other suggestions of what to look at?

-tim



> Hey Tim,
>
> Here is a short summary:
>
> Reductions and indexing return different types based on the number of
> dimensions of the input array:
>
> >>> b = sum(a)
> >>> l = len(b) # or whatever
>
> This code works happily if "a" is 2 or more dimensions, but will fail if
> it is 1d because the sum(a) will return a scalar in this case.  To write
> generic code, you have to put an if/else statement in to check whether b
> is a scalar or an array:
>
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
>
> or less verbose but still unpleasant:
>
> >>> b = asarray(sum(a))
> >>> l = len(b)
>
> eric
>
> > -----Original Message-----
> > From: num...@li... [mailto:numpy-
> > dis...@li...] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: num...@li...
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> >
> >
> >
> >
> > [Perry wrote about Travis's proposal]
> >
> > > refresh my memory about how this proposal would work. I've heard
> > > proposals to add new types to Python itself, but that seems out of
> > > the question. Are you talking about adding new scalar types as
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or a scalar
> > > (which was one of the drivers to produce rank-0 results).
> >
> > Hi Perry,
> >
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us as to the
> > basics
> > of it?.
> >
> > -tim
> >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Numpy-discussion mailing list
> > Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

RE: [Numpy-discussion] rank-0 arrays

[Perry G. <pe...@st...>]

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
On a different front, I'm not sure that this proposal is very useful
for numarray because of the new scalar/array coercion rules that
numarray has. Since the new scalar literals must still use 
type functions (e.g.,, Int8(2)) it doesn't help make expressions
any prettier (or do I misunderstand the proposal?). In returning
scalars from arrays, I don't see that converting the array type
to a Python scalar type helps much. With the exception of long
doubles (and the corresponding complex precision), the conversion
to the python scalar doesn't lose any precision and you can 
restore the exact array type value by assigning that scalar value
back to an array element. As mentioned, the only exception is for
long doubles; for that case, a new scalar type makes sense.

Is there any other compelling reason for new scalar types?

Perry

RE: [Numpy-discussion] rank-0 arrays

[Paul F D. <pa...@pf...>]

This is a perfect example of why the thing is so annoying. 
> -----Original Message-----
> From: num...@li... 
> [mailto:num...@li...] On 
> Behalf Of eric jones
> Sent: Friday, September 13, 2002 1:44 PM
> To: 'Tim Hochberg'; num...@li...
> Subject: RE: [Numpy-discussion] rank-0 arrays
> 
> 
> Hey Tim,
> 
> Here is a short summary:
> 
> Reductions and indexing return different types based on the 
> number of dimensions of the input array:
> 
> >>> b = sum(a)
> >>> l = len(b) # or whatever
> 
> This code works happily if "a" is 2 or more dimensions, but 
> will fail if it is 1d because the sum(a) will return a scalar 
> in this case.  To write generic code, you have to put an 
> if/else statement in to check whether b is a scalar or an array:
> 
> >>> b = sum(a)
> >>> if type(b) is ArrayType:
> ...     l = len(b)
> ... else:
> ...     l = 1 # or whatever you need
> 
> or less verbose but still unpleasant:  
> 
> >>> b = asarray(sum(a))
> >>> l = len(b)
> 
> eric
> 
> > -----Original Message-----
> > From: num...@li... [mailto:numpy- 
> > dis...@li...] On Behalf Of Tim Hochberg
> > Sent: Friday, September 13, 2002 3:18 PM
> > To: num...@li...
> > Subject: Re: [Numpy-discussion] rank-0 arrays
> > 
> > 
> > 
> > 
> > [Perry wrote about Travis's proposal]
> > 
> > > refresh my memory about how this proposal would work. I've heard 
> > > proposals to add new types to Python itself, but that 
> seems out of 
> > > the question. Are you talking about adding new scalar types as 
> > > module? E.g.
> > [snip]
> > > In any case, doesn't this still have the problem that Eric
> complained
> > > about, having to test for whether a result is an array or 
> a scalar 
> > > (which was one of the drivers to produce rank-0 results).
> > 
> > Hi Perry,
> > 
> > I like Travis's proposal the best of those I've seen so far, but I
> don't
> > recall the details of Eric's problem. Could you refresh us 
> as to the 
> > basics of it?.
> > 
> > -tim
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf 
> > _______________________________________________
> > Numpy-discussion mailing list Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

RE: [Numpy-discussion] rank-0 arrays

[Perry G. <pe...@st...>]

> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I don't
> recall the details of Eric's problem. Could you refresh us as to 
> the basics
> of it?.
> 
> -tim
> 
Rather than put keystrokes on Eric's fingertips :-) I think it would
be best if he (or Paul Dubois, who had similar concerns) gave
some examples. But I believe it concerned fairly generic
Numeric routines that had expressions that might or might not
result in scalars. Testing for both cases resulted in 
extra code (Paul said that MA had to deal with this a lot) and
that having to deal with this made the code error prone.

But I'd rather they explained it since they have very specific
experience with this issue. Perhaps they can give a couple
common examples.

Perry

RE: [Numpy-discussion] rank-0 arrays

[eric j. <er...@en...>]

Hey Tim,

Here is a short summary:

Reductions and indexing return different types based on the number of
dimensions of the input array:

>>> b = sum(a)
>>> l = len(b) # or whatever

This code works happily if "a" is 2 or more dimensions, but will fail if
it is 1d because the sum(a) will return a scalar in this case.  To write
generic code, you have to put an if/else statement in to check whether b
is a scalar or an array:

>>> b = sum(a)
>>> if type(b) is ArrayType:
...     l = len(b)
... else:
...     l = 1 # or whatever you need

or less verbose but still unpleasant:  

>>> b = asarray(sum(a))
>>> l = len(b)

eric

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Tim Hochberg
> Sent: Friday, September 13, 2002 3:18 PM
> To: num...@li...
> Subject: Re: [Numpy-discussion] rank-0 arrays
> 
> 
> 
> 
> [Perry wrote about Travis's proposal]
> 
> > refresh my memory about how this proposal would work. I've heard
> > proposals to add new types to Python itself, but that seems out of
> > the question. Are you talking about adding new scalar types as
> > module? E.g.
> [snip]
> > In any case, doesn't this still have the problem that Eric
complained
> > about, having to test for whether a result is an array or a scalar
> > (which was one of the drivers to produce rank-0 results).
> 
> Hi Perry,
> 
> I like Travis's proposal the best of those I've seen so far, but I
don't
> recall the details of Eric's problem. Could you refresh us as to the
> basics
> of it?.
> 
> -tim
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

Travis Oliphant <oli...@ee...> writes:

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree

Right. I still think this is the best solution in that it is the least
confusing. If I remember correctly, the main objection was that
implementing all those types was not as trivial as hoped, but I forgot
the details (and it wasn't me who tried).

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] rank-0 arrays

[Tim H. <tim...@ie...>]

[Perry wrote about Travis's proposal]

> refresh my memory about how this proposal would work. I've heard
> proposals to add new types to Python itself, but that seems out of
> the question. Are you talking about adding new scalar types as
> module? E.g.
[snip]
> In any case, doesn't this still have the problem that Eric complained
> about, having to test for whether a result is an array or a scalar
> (which was one of the drivers to produce rank-0 results).

Hi Perry,

I like Travis's proposal the best of those I've seen so far, but I don't
recall the details of Eric's problem. Could you refresh us as to the basics
of it?.

-tim

RE: [Numpy-discussion] rank-0 arrays

[Perry G. <pe...@st...>]

> If we just implemented Python scalars for the other precisions then much
> of this rank 0 array business would be solved as we could always return
> the Python scalar rather than an array for rank 0 arrays.
> 
> I believe Konrad made this suggestion many moons ago and I would agree
> with him.
> 
> -Travis Oliphant
> 
Hi Travis,

refresh my memory about how this proposal would work. I've heard
proposals to add new types to Python itself, but that seems out of
the question. Are you talking about adding new scalar types as 
module? E.g.

>>> x = Int8(22) 
>>> arr = arange(10, typecode = Int8)
>>> arr[2]
Int8(2)

Or some other approach? 

In any case, doesn't this still have the problem that Eric complained
about, having to test for whether a result is an array or a scalar
(which was one of the drivers to produce rank-0 results).

Thanks, Perry

Re: [Numpy-discussion] rank-0 arrays

[Travis O. <oli...@ee...>]

>
> To restate the issue: there was a question about whether
> an index to an array that identified a single element only
> (i.e., not a slice, nor an incomplete index, e.g. x[3] where
> x is two dimensional) should return a Python scalar or a
> rank-0 array. Currently Numeric is inconsistent on this point.
> One usually gets scalars, but on some occasions, rank-0 arrays
> are returned. Good arguments are to be had for either alternative.
>

I think we should implement Python scalars for the other types and then
eliminate rank-0 arrays completely.  I could have a student do this in a
few weeks if it was agreed on.

-Travis Oliphant

Re: [Numpy-discussion] rank-0 arrays

[Travis O. <oli...@ee...>]

> Two other issues come up trying to implement the "rank-0 experiment":
>
> 1.  What should be the behavior of subscripting a rank-0 array?
>
> a.  Return the scalar value                         (what numarray does
> now.  seems inconsistent)
+0.25

> b.  Raise an exception
+1

> c.  Return a copy  of the rank-0 array
-1

>
> 2.  What's a decent notation for .asScalar()?
>
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray
> does now)
-1

Numeric does this now for backward compatibility.  I don't think this
issue was discussed at length.


> b.  a[ <subscript_resulting_in_rank0> ]()                  (override
> __call__)
-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()
+1  just keep the current .toscalar()  notation.

Also int(a) and float(a) and complex(a) would work as well.

If we just implemented Python scalars for the other precisions then much
of this rank 0 array business would be solved as we could always return
the Python scalar rather than an array for rank 0 arrays.

I believe Konrad made this suggestion many moons ago and I would agree
with him.

-Travis Oliphant

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

Scott Gilbert <xs...@ya...> writes:

> > a.  Return the scalar value                         (what numarray does 
> > now.  seems inconsistent)
> 
> If you think of it as "dereferencing" instead of "indexing", I think this
> is most consistent.  It takes 3 elements to dereference a rank-3 array, so
> it should take 0 elements to dereference a rank-0 array.

That would be fine with me, but empty index brackets raise a
SyntaxError in Python.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] rank-0 arrays

[Scott G. <xs...@ya...>]

--- Todd Miller <jm...@st...> wrote:
> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)
>

If you think of it as "dereferencing" instead of "indexing", I think this
is most consistent.  It takes 3 elements to dereference a rank-3 array, so
it should take 0 elements to dereference a rank-0 array.


>
> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)
> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)
> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  
> 
> Any strong opinions?
> 

Since subscripting a rank-3 array:

  a[1, 2, 3] 

is very much like 

  a[(1, 2, 3)]

The __getitem__ receives the tuple (1, 2, 3) in both cases.  (Try it!)

So that would imply subscripting (dereferencing) a rank-0 array could be:

  a[] could be represented by a[()]

It's just unfortunate that Python doesn't currently recognize the a[] as a
valid syntax.




Cheers,
    -Scott






__________________________________________________
Do you Yahoo!?
Yahoo! News - Today's headlines
http://news.yahoo.com

Re: [Numpy-discussion] rank-0 arrays

[Pearu P. <pe...@ce...>]

On Fri, 13 Sep 2002, Todd Miller wrote:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray does 
> now.  seems inconsistent)

-0.5

> b.  Raise an exception

+1

> c.  Return a copy  of the rank-0 array

-1

> 2.  What's a decent notation for .asScalar()?
> 
> a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
> does now)

+-0, just be consistent.

> b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
> __call__)

-1

> c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

+0.5


Pearu

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

Todd Miller <jm...@st...> writes:

> Two other issues come up trying to implement the "rank-0 experiment":
> 
> 1.  What should be the behavior of subscripting a rank-0 array?
> 
> a.  Return the scalar value                         (what numarray
> does now.  seems inconsistent)
> b.  Raise an exception
> c.  Return a copy  of the rank-0 array

I am for b). A rank-0 array is not a sequence and has no elements,
so indexing shouldn't be allowed.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] rank-0 arrays

[Todd M. <jm...@st...>]

Two other issues come up trying to implement the "rank-0 experiment":

1.  What should be the behavior of subscripting a rank-0 array?

a.  Return the scalar value                         (what numarray does 
now.  seems inconsistent)
b.  Raise an exception
c.  Return a copy  of the rank-0 array

2.  What's a decent notation for .asScalar()?

a.  a[ <subscript_resulting_in_rank0> ][0]                (what numarray 
does now)
b.  a[ <subscript_resulting_in_rank0> ]()                  (override 
__call__)
c.  a[ <subscript_resulting_in_rank0> ].asScalar()  

Any strong opinions?

Todd

-- 
Todd Miller 			jm...@st...
STSCI / SSG

RE: [Numpy-discussion] rank-0 arrays

[Perry G. <pe...@st...>]

Konrad Hinsen writes:
>
> Pearu Peterson <pe...@ce...> writes:
>
> > I think it would be confusing if the result of repr would be `2' and not
> > `array(2)' because 2 and array(2) are not equivalent in all usages
> > but it should be clear from repr results as a first way to learn more
> > about the objects.
>
> I agree. There will already be some inevitable confusion with both
> rank-0 arrays and scalars around, with similar but not identical
> behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.
>
I also agree that having repr hide the fact that it is an array is
a bad thing. But do you find getting "array(2)" when indexing
a single item acceptable when working interactively? I agree that
displayhook is probably the best way of altering this behavior if
a user desires (perhaps we will provide a module function to do so, but
not enable it by default).

> >
> > Could array.__getitem_ and __getslice__ detect if their argument is
> > an array and skip using Python objects when iterating over indices?
>
> Of course they know that they are indexing an array, they are
> defined at the level of the array class/type. However, they cannot
> detect an iteration as opposed to a single item access.
>
I don't know that it is desirable either to have iteration
return different kinds of objects than explicit indexing, even
if possible. Do we really want

for val in x:
    ...

to use scalars while

for i in range(len(x)):
    val = x[i]
    ...

uses rank-0 arrays? I'm inclined not to have different behavior for
what seems like identical iteration.

> I don't know if this efficiency problem could be important in
> practice, probably only practice can tell. I have no idea how many
> single-item indexing operations into arrays occur in my code, this is
> not something I worried about when writing it.
>
> If there will be scalar and rank-0 array returning variants of
> indexing anyway, then I suppose that changing the index syntax
> to one or the other is not a big effort. So my suggestion is to
> make a test release and see what the reactions are.
>
That is our plan (hence the reference to "experimental"), but I
wanted to see if there were strong feelings on this before doing so.

Perry

Re: [Numpy-discussion] Tensor-like product with extra indices?

[Konrad H. <hi...@cn...>]

Huaiyu Zhu <hua...@ya...> writes:

> Two related questions:
> 
> 1. Suppose  B = ravel(A) and
>             B[i] corresponds to A[i0,i1,...].
> 
>    Are there named functions that map the indices
> 
>         (i0,i0,...) -> i
> 
>    and back?  This is assuming A.shape = (m0,m1,...) is known.

Untested, but the idea should work:

  def ravel_index(array, *indices):
     offsets = Numeric.multiply.accumulate(array.shape[1:][::-1])[::-1]
     return Numeric.sum(Numeric.array(indices[:-1])*offsets) + indices[-1]

> 2. Suppose the above function is called ravel_index and its
>    inverse unravel_index.
> 
>    Given a list of matrices [A0, A1, ...] is there a function that
>    calculates a matrix B such that
> 
>         B[i,j]  = A0[i0,j] * A1[i1,j] * ...
> 
>    where i = ravel_index(i0, i1, ...).

Not for all i, which I assume is what you want to do.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] rank-0 arrays

[Konrad H. <hi...@cn...>]

Pearu Peterson <pe...@ce...> writes:

> I think it would be confusing if the result of repr would be `2' and not
> `array(2)' because 2 and array(2) are not equivalent in all usages
> but it should be clear from repr results as a first way to learn more
> about the objects.

I agree. There will already be some inevitable confusion with both
rank-0 arrays and scalars around, with similar but not identical
behaviour. Rank-0 arrays shouldn't make it worse by using camouflage.

> > The second issue is an efficiency one. Currently numarray uses
> > Python objects for arrays. If we return rank-0 arrays for
> > single item indexing, then some naive uses of larger arrays
> > as sequences may lead to an enormous number of array objects
> > to be created. True, there will be equivalent means of doing
> > the same operation that won't result in massive object creations
> > (such as specifically converting an array to a list, which would 
> > be done much faster). Is this a serious problem?
> 
> Could array.__getitem_ and __getslice__ detect if their argument is
> an array and skip using Python objects when iterating over indices?

Of course they know that they are indexing an array, they are
defined at the level of the array class/type. However, they cannot
detect an iteration as opposed to a single item access.

I don't know if this efficiency problem could be important in
practice, probably only practice can tell. I have no idea how many
single-item indexing operations into arrays occur in my code, this is
not something I worried about when writing it.

If there will be scalar and rank-0 array returning variants of
indexing anyway, then I suppose that changing the index syntax
to one or the other is not a big effort. So my suggestion is to
make a test release and see what the reactions are.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

[Numpy-discussion] Tensor-like product with extra indices?

[Huaiyu Z. <hua...@ya...>]

Two related questions:

1. Suppose  B = ravel(A) and
            B[i] corresponds to A[i0,i1,...].

   Are there named functions that map the indices

        (i0,i0,...) -> i

   and back?  This is assuming A.shape = (m0,m1,...) is known.

2. Suppose the above function is called ravel_index and its
   inverse unravel_index.

   Given a list of matrices [A0, A1, ...] is there a function that
   calculates a matrix B such that

        B[i,j]  = A0[i0,j] * A1[i1,j] * ...

   where i = ravel_index(i0, i1, ...).

   What about the inverse function that turns B into C such that

        C[i0,i1,...,j] = A0[i0,j] * A1[i1,j] * ...


I've considered reshape, ravel, take, put, outerproct but couldn't come
up with a combination to do it without a for-loop on j and a for-loop on
[A0,A1,...].

The above two questions I need answers right now.  But I can imagine a
need for a more general operations in the future.  Given a sequence of
arrays [A0,A1,...], produce a tensor product by picking certain indices
from each array (like i0,i1 above), while keeping some other indices
fixed (like j above).  For example, the above could be written as

        B = tensor_like([A0,A1...], [(0,None,None,...), ...,
                                (None,....,None,0), (1,1,...1)])

As another example, if len(A.shape)==3 then

        transpose(A) == tensor_like([A], [(2,), (1,), (0,)])


Huaiyu

Re: [Numpy-discussion] rank-0 arrays

[Pearu P. <pe...@ce...>]

On Thu, 12 Sep 2002, Perry Greenfield wrote:

> If we return rank-0 arrays, what should repr return for rank-0
> arrays. My initial impression is that the following is highly
> undesirable for a interactive session, but maybe it is just me:
> 
> >>> x = arange(10)
> >>> x[2]
> array(2)
>
> We, of course, could arrange __repr__ to return "2" instead,
> in other words print the simple scalar for all cases of rank-0
> arrays. This would yield the expected output in the above
> example. Nevertheless, isn't it violating the intent of repr?
> Are there other examples where Python uses repr in a similar,
> misleading manner? But perhaps most feel that returning array(2)
> is perfectly acceptable and won't annoy users. I am curious
> about what people think about this.

I think it would be confusing if the result of repr would be `2' and not
`array(2)' because 2 and array(2) are not equivalent in all usages
but it should be clear from repr results as a first way to learn more
about the objects.

For example, if using array(2) as an index in Python native objects, then
TypeError is raised (as expected). In interactive session the quickest way
to check the type of a variable is just type its name and press enter:

>>> i
array(2)

Now, if repr(array(2)) returns '2', then one firstly assumes that
`i' is an integer. However, this would be very confusing if one sees
>>> some_list[i]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: sequence index must be integer
>>> i
2

So, I think that repr(array(2)) should return 'array(2)'. And users can
always change this behaviour locally by using sys.displayhook. Though, I
would recommend using tools like ipython for interactive sessions.

Btw, note that during an interactive session it would be *seemingly*
desired if also repr(string) would return str(string). For example,
when viewing documentation in interactive sessions. Wouldn't it be nice to
have

>>> sys.displayhook.__doc__
displayhook(object) -> None

Print an object to sys.stdout and also save it in __builtin__._

>>>

instead of the current behaviour:

>>> sys.displayhook.__doc__
'displayhook(object) -> None\n\nPrint an object to sys.stdout and also
save it in __builtin__._\n'
>>>


> The second issue is an efficiency one. Currently numarray uses
> Python objects for arrays. If we return rank-0 arrays for
> single item indexing, then some naive uses of larger arrays
> as sequences may lead to an enormous number of array objects
> to be created. True, there will be equivalent means of doing
> the same operation that won't result in massive object creations
> (such as specifically converting an array to a list, which would 
> be done much faster). Is this a serious problem?

Could array.__getitem_ and __getslice__ detect if their argument is
an array and skip using Python objects when iterating over indices?
If this is technically possible then it is not a good reason to drop
returning rank-0 arrays. The actual implementation may come later, though.

> If people still want rank-0 arrays, what should repr do?

Always return 'array(...)'.

You can also ask from python-dev for advice if numarray is considered to
be included to Python library in future. I am sure that repr issue will be
brought up if repr==str for 0-rank arrays.

Pearu