numpy-discussion

[Numpy-discussion] s are mutable scalars

[Huaiyu Z. <hua...@ya...>]

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu

Re: [Numpy-discussion] rank-0 arrays are mutable scalars

[Huaiyu Z. <hua...@ya...>]

Oops, the subject line was somehow cut off.  Please use this one if you 
follow up. - Huaiyu

On Tue, 17 Sep 2002, Huaiyu Zhu wrote:

Some of the choices between rank-0 arrays and new scalar types
might be resolved by enumerating the properties desired of them.

Most properties of rank-0 arrays could be fixed by consistency
requirements alone, using operations that reduce array dimensions.

Let a = ones((2,3,4))
b = sum(a)
c = sum(b)
d = sum(c)

Property 1: the shape of an array is a tuple of integers
	a.shape == (2, 3, 4)
	b.shape == (3, 4)
	c.shape == (4,)
	d.shape == ()
	
Property 2: rank(a) == len(a.shape)
	rank(a) == 3 == len(a.shape)
	rank(b) == 2 == len(b.shape)
	rank(c) == 1 == len(c.shape)
	rank(d) == 0 == len(d.shape)

Property 3: len(a) == a.shape[0]
	len(a) == 2 == a.shape[0]
	len(b) == 3 == b.shape[0]
	len(c) == 4 == c.shape[0]
	len(d) == Exception == d.shape[0]

	# Currently the last is wrong?

Property 4: size(a) == product(a.shape)
	size(a) == 24 == product(a.shape)
	size(b) == 12 == product(b.shape)
	size(c) == 4 == product(c.shape)
	size(d) == 1 == product(d.shape)

	# Currently the last is wrong

Property 5: rank-0 array behaves as mutable numbers when used as value
	array(2)  is similar to 2
	array(2.0)  is similar to 2.0
	array(2j)  is similar to 2j

	# This is a summary of many concrete properties.

Property 6: Indexing reduces rank. Slicing preserves rank.
	a[:,:,:].shape = (2, 3, 4)
	a[1,:,:].shape = (3, 4)
	a[1,1,:].shape = (4,)
	a[1,1,1].shape = ()

Property 7: Indexing by tuple of ints gives scalar.
	a[1,1,1] == 1
	b[1,1] == 2
	c[1,] == 6
	d[()] == 24

	# So rank-0 array indexed by empty tuple should be scalar.
        # Currently the last is wrong

Property 8: Indexing by tuple of slices gives array.
	a[:,:,:] == ones((2,3,4))
	b[:,:] == ones((3,4)) * 2
	c[:] == ones((,4)) * 6
	d[()] == ones(()) * 24

	# So rank-0 array indexed by empty tuple should be rank-0 array.
        # Currently the last is wrong

Property 9: Indexing as lvalues
	a[1,1,1] = 2
	b[1,1] = 2
	c[1,] = 2
	d[()] = 2

Property 10: Indexing and slicing as lvalues
	a[:,:,:] = ones((2, 3, 4))
	a[1,:,:] = ones((3, 4))
	a[1,1,:] = ones((4,))
	a[1,1,1] = ones(())
	
	# But the last is wrong.


Conclusion 1: rank-0 arrays are equivalent to scalars.
	See properties 7 and 8.

Conclusion 2: rank-0 arrays are mutable.
	See property 9.

Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
	should not be defined.

	See conclusion 1 and properties 1, 2, 3, 4.

Conclusion 4: A missing axis is similar to having dimension 1.
	See property 4.

Conclusion 5: rank-0 int arrays should be allowed to act as indices.
	See property 5.

Conclusion 6: rank-0 arrays should not be hashable except by object id.
	See conclusion 2.


Discussions:

  - These properties correspond to the current implementation quite well,
	except a few rough edges.

  - Mutable scalars are useful in their own rights.

  - Is there substantial difference in overhead between rank-0 arrays and
    scalars?

  - How to write literal values?  array(1) is too many characters.

  - For rank-1 and rank-0 arrays, Python notation distinguishes:

	c[1] vs c[1,]
	d[] vs d[()]

    Should these be used to handle semantic difference between indexing 
    and slicing?  Should d[] be syntactically allowed?

Hope these observations help.

Huaiyu

Re: [Numpy-discussion] s are mutable scalars

[Travis O. <oli...@ee...>]

>
> Some of the choices between rank-0 arrays and new scalar types
> might be resolved by enumerating the properties desired of them.
>

These are helpful observations.

> Most properties of rank-0 arrays could be fixed by consistency
> requirements alone, using operations that reduce array dimensions.
>
> Let a = ones((2,3,4))
> b = sum(a)
> c = sum(b)
> d = sum(c)
>
> Property 1: the shape of an array is a tuple of integers
> 	a.shape == (2, 3, 4)
> 	b.shape == (3, 4)
> 	c.shape == (4,)
> 	d.shape == ()
>
> Property 2: rank(a) == len(a.shape)
> 	rank(a) == 3 == len(a.shape)
> 	rank(b) == 2 == len(b.shape)
> 	rank(c) == 1 == len(c.shape)
> 	rank(d) == 0 == len(d.shape)
>
> Property 3: len(a) == a.shape[0]
> 	len(a) == 2 == a.shape[0]
> 	len(b) == 3 == b.shape[0]
> 	len(c) == 4 == c.shape[0]
> 	len(d) == Exception == d.shape[0]
>
> 	# Currently the last is wrong?

Agreed, but this is because d is an integer and out of Numerics control.
This is a case for returning 0d arrays rather than Python scalars.

>
> Property 4: size(a) == product(a.shape)
> 	size(a) == 24 == product(a.shape)
> 	size(b) == 12 == product(b.shape)
> 	size(c) == 4 == product(c.shape)
> 	size(d) == 1 == product(d.shape)
>
> 	# Currently the last is wrong

I disagree that this is wrong.  This works as described for me.

>
> Property 5: rank-0 array behaves as mutable numbers when used as value
> 	array(2)  is similar to 2
> 	array(2.0)  is similar to 2.0
> 	array(2j)  is similar to 2j
>
> 	# This is a summary of many concrete properties.
>
> Property 6: Indexing reduces rank. Slicing preserves rank.
> 	a[:,:,:].shape = (2, 3, 4)
> 	a[1,:,:].shape = (3, 4)
> 	a[1,1,:].shape = (4,)
> 	a[1,1,1].shape = ()
>
> Property 7: Indexing by tuple of ints gives scalar.
> 	a[1,1,1] == 1
> 	b[1,1] == 2
> 	c[1,] == 6
> 	d[()] == 24
>
> 	# So rank-0 array indexed by empty tuple should be scalar.
>         # Currently the last is wrong

Not sure about this property, but interesting.

>
> Property 8: Indexing by tuple of slices gives array.
> 	a[:,:,:] == ones((2,3,4))
> 	b[:,:] == ones((3,4)) * 2
> 	c[:] == ones((,4)) * 6
> 	d[()] == ones(()) * 24
>
> 	# So rank-0 array indexed by empty tuple should be rank-0 array.
>         # Currently the last is wrong

Not sure about this one either.
>
> Property 9: Indexing as lvalues
> 	a[1,1,1] = 2
> 	b[1,1] = 2
> 	c[1,] = 2
> 	d[()] = 2
>
> Property 10: Indexing and slicing as lvalues
> 	a[:,:,:] = ones((2, 3, 4))
> 	a[1,:,:] = ones((3, 4))
> 	a[1,1,:] = ones((4,))
> 	a[1,1,1] = ones(())
>
> 	# But the last is wrong.
>
>
> Conclusion 1: rank-0 arrays are equivalent to scalars.
> 	See properties 7 and 8.
>
> Conclusion 2: rank-0 arrays are mutable.
> 	See property 9.
>
> Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar)
> 	should not be defined.

Why is this?  I thought you argued the other way for len(scalar).  Of
course, one solution is that we could overwrite the len() function and
allow it to work for scalars.

>
> 	See conclusion 1 and properties 1, 2, 3, 4.
>
> Conclusion 4: A missing axis is similar to having dimension 1.
> 	See property 4.
>
> Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> 	See property 5.

Can't do this for lists and other builtin sequences.
>
> Conclusion 6: rank-0 arrays should not be hashable except by object id.
> 	See conclusion 2.
>
>
> Discussions:
>
>   - These properties correspond to the current implementation quite well,
> 	except a few rough edges.
>
>   - Mutable scalars are useful in their own rights.
>
>   - Is there substantial difference in overhead between rank-0 arrays and
>     scalars?

Yes.

>
>   - How to write literal values?  array(1) is too many characters.
>
>   - For rank-1 and rank-0 arrays, Python notation distinguishes:
>
> 	c[1] vs c[1,]
> 	d[] vs d[()]
>
>     Should these be used to handle semantic difference between indexing
>     and slicing?  Should d[] be syntactically allowed?
>
> Hope these observations help.

Thanks for the observations.

-Travis O.

Re: [Numpy-discussion] rank-0 arrays are mutable scalars

[Huaiyu Z. <hua...@ya...>]

On Tue, 17 Sep 2002, Travis Oliphant wrote:

> > 	len(d) == Exception == d.shape[0]
> >
> > 	# Currently the last is wrong?
> 
> Agreed, but this is because d is an integer and out of Numerics control.
> This is a case for returning 0d arrays rather than Python scalars.

That is one problem.  It can be removed by using shape(d).  

More fundamentally, though, len(d) == shape(d)[0] == ()[0] => IndexError.
I think Konrad made this point a few days back.

> > 	size(d) == 1 == product(d.shape)
> >
> > 	# Currently the last is wrong
> 
> I disagree that this is wrong.  This works as described for me.

Right.  Change d.shape to shape(d) here (and in several other places).

> Why is this?  I thought you argued the other way for len(scalar).  Of
> course, one solution is that we could overwrite the len() function and
> allow it to work for scalars.

Raising exception is the correct behavior, not a problem to be solved.

> >
> > Conclusion 5: rank-0 int arrays should be allowed to act as indices.
> > 	See property 5.
> 
> Can't do this for lists and other builtin sequences.

If numarray defines a consistent set of behaviors for integer types that
is intuitively understandable, it might not be difficult to persuade core
Python to check against an abstract integer type.

> >   - Is there substantial difference in overhead between rank-0 arrays and
> >     scalars?
> 
> Yes.

That would be one major problem.  


However, after giving this some more thoughts, I'm starting to doubt the 
analogy I made.  The problem is that in the end there is still a need to 
index an array and obtain a good old immutable scalar.  So

- What notation should be used for this purpose?  We can use c[0] to get
  immutable scalars and c[0,] for rank-0 arrays / mutable scalars.  But 
  what about other ranks?  Python does not allow distinctions based on  
  a[1,1,1] versus a[(1,1,1)] or d[] versus d[()].

- This weakens the argument that rank-0 arrays are scalars, since that 
  argument is essentially based on sum(c) and c[0] being of the same type.

Huaiyu

[Numpy-discussion] Conversion numarray/Numeric objects

[Aureli S. F. <au...@ip...>]

Hi,

I have developed till now all my numerical stuff using Numeric objects.

I want to use some modules using numarray. I have tested some 
conversion functions on Python, e.g.

numarray.array(Numeric_object.tolist())

but it seems to me a bit slow (specially for large arrays as images). 
Is there any more ways of doing this conversion?
Should I write my own conversion in C?
Which advices could you give me in order to make an smoother transition?

Furthermore I found the web site about numarray/Numeric 
incompatibilities very interesting. However I did not understand 
quite well the paragraph:

>Numeric arrays have some public attributes. Numarray arrays
>have none. All changes to an array's state must be made
>through accessor methods. Specifically, Numeric has the following
>attributes:
>
>Numeric        numarray accessor method(s)
>Attribute
>
>shape      --> shape()  (i.e., specify new shape through method arg instead
>of
>                          assigning to shape attribute)
>flat       --> flat()
>real       --> real()   (set capability not implemented yet, but will be)
>imag       --> imag()   (ditto)
>savespace  --> not used, no equivalent functionality (xxx check this)

since interactively typing a.shape() or a.shape((8,8)) raise an 
exception (Tuple object not callable). Am I misunderstanding 
something?

Thanks in advance.
Best regards,
Aureli

-- 
#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail: au...@ip...
fon: +49 30 39006-143
fax: +49 30 3917517
web: http://vision.fhg.de/~aureli/web-aureli_en.html
#################################

Re: [Numpy-discussion] Conversion numarray/Numeric objects

[Todd M. <jm...@st...>]

Aureli Soria Frisch wrote:

> Hi,
>
>
> I have developed till now all my numerical stuff using Numeric objects.
>
>
> I want to use some modules using numarray. I have tested some 
> conversion functions on Python, e.g.
>
>
> numarray.array(Numeric_object.tolist())
>
Much faster,  but less general, is:

numarray.fromstring(Numeric_object.tostring(), type=XXXX)

>
> but it seems to me a bit slow (specially for large arrays as images). 
> Is there any more ways of doing this conversion?
>
> Should I write my own conversion in C?
>
It depends on how soon you need it to be fast.    numarray 
fromlist/tolist should be implemented in C for the next release, which 
should occur within a couple months, perhaps sooner.  If you want to 
write a C extension for numarray now, the best way to do it is to use 
the Numeric compatability layer,   which essentially uses a subset of 
the Numeric C-API,  and is not likely to change.

> Which advices could you give me in order to make an smoother transition?
>
>
> Furthermore I found the web site about numarray/Numeric 
> incompatibilities very interesting. However I did not understand quite 
> well the paragraph:
>
I think that webpage is out of date.

>
> since interactively typing a.shape() or a.shape((8,8)) raise an 
> exception (Tuple object not callable). Am I misunderstanding something?
>
Actually, for python 2.2 and up,  numarray has all of those attributes 
in a Numeric compatible, Python properties based form.   So ".shape" 
get/sets the shape in numarray, just as in Numeric:

 >>> import numarray
 >>> a = numarray.arange(10)
 >>> a.shape
(10,)
 >>> a.shape = (2,5)
 >>> a.shape
(2,5)

Because of this,  the expression a.shape() that you mentioned tries to 
"call" the shape tuple returned by .shape, resulting in an exception.

Todd

-- 
Todd Miller 			jm...@st...
STSCI / SSB