numpy-discussion — Discussion list for all users of Numerical Python

RE: [Numpy-discussion] adding a .M attribute to the array.

[Pearu P. <pe...@ce...>]

On Wed, 6 Mar 2002, Pearu Peterson wrote:

> Actually, library functions probably use asarray for arguments, this
> function could also clean up the asmatrix bit.

Ok, ignore this remark. Library functions actually would use this bit
for triggering different operations, eg. sin(a), sin(a.m).

Pearu

RE: [Numpy-discussion] adding a .M attribute to the array.

[Pearu P. <pe...@ce...>]

On Wed, 6 Mar 2002, Perry Greenfield wrote:

> This is a clever idea that reminds me of  something we were considering
> for something else (exactly what I can't quite remember :-). But like
> all such schemes it still does produce an object, and a user might
> infer (reasonably or unreasonably) that if they can type
> 
> x = a.m * b
> 
> Then they can treat a.m as an array, e.g.,
> 
> x = a.m

Yes, I was also thinking about the same issue. If we could somehow
confince users that .m attribute comes only with operations, e.g.

   a  .m*  b

then it should be safe... Note that in order to use .m feature, an user
must read about it somewhere, say, from tutorial. And there should be
noted explicitely where _not_ to use .m feature, that is, in assignments
and in function arguments, in order to avoid side any unwanted 
side effects. Actually, library functions probably use asarray for
arguments, this function could also clean up the asmatrix bit.

Pearu

RE: [Numpy-discussion] Adding a flag to allow integer array access and masking

[Travis O. <oli...@ee...>]

> Like Greg I'm wary of having many different interpretations
> for indexing behavior (I'm not even that crazy about having
> numarray handle boolean index arrays differently than the others
> --something we haven't implemented yet, and perhaps we shouldn't).
>

You may be wary, but there are already multiple ways people think about
using integers to index arrays.  I'm trying to suggest a facility that
allows several different interpretations of array access.

> Before discussing the merits of this, shouldn't we take the attitude
> that absence of feedback is not necessarily equivalent to approval,
> particularly for something that affects the public interface of
> the module? I would feel better about this if I saw several
> affirming the need for such features rather than few openly
> opposing it.
>

I do have this view.  I'm not changing anything, right now.

Well, I affirm that this is one of the drawbacks of Numeric as compared
with other array-oriented environments.  We definitely need a way to index
an array using integers and masks.

I guess if nobody else feels this way, then I'm alone in my discomfort.

> But if one were to do something like this, I would use a different kind
> of object than 0d arrays, e.g., an instance of a class defined for just
> that purpose.

We could do that as well.

> You would really want to make sure that no data could
> mistakenly be interpreted as a flag, even if the chances were remote.
> I would also not use an underscore as the beginning of the name.

I'm not particularly wedded to _I notation, it was just a start.

> Maybe
> I'm wrong about this, but I've come to take that to mean its a private
> variable that should not be used by users of the module, and that usage
> would confuse that. Finally, the name of the flag should be descriptive
> (e.g. MaskInd).
>
> But there could be better alternatives. As an example,
>
> x[nonzero(maskarray)] instead of x[maskarray, MaskInd]

I've thought about that, too, it would work if nonzero returned some class
that stored away (but didn't copy) the maskarray info.


-Travis

Re: [Numpy-discussion] adding a .M attribute to the array.

[Andrew P. L. <bs...@al...>]

Well, I believe that it solves the wrong problem.  What I really want are
Matrix objects that stay as Matrix objects even through their associated
functions.  And Array objects that stay as array objects.

Why add any characters or casts at all when the objects can stay in their
original type?

Please correct me if I'm missing something here.

-a

On Tue, 5 Mar 2002, Travis Oliphant wrote:

>
> Recently there has been discussion on the list about the awkwardness of
> matrix syntax when using Numeric Python.
>
> Matrix expressions can be awkard to express in Numeric which is a negative
> mark on an otherwise excellent computing environment.
>
> Currently part of the problem can be solved by working with Matrix objects
> explicitly:
>
> a = Matrix.Matrix("[1 2 3; 4 5 6]")     # Notice the strings.
>
> However, most operations return arrays which have to be recast to matrices
> using at best a character with parenthesis:
>
> M = Matrix.Matrix
>
> M(sin(a)) * M(cos(a)).T
>
> The suggestion was made to add ".M" as an attribute of arrays which returns a
> matrix.  Thus, the code above can be written:
>
> sin(a).M * cos(a).M.T
>
> While some aesthestic simplicity is obtained, the big advantage is in
> consistency.  Somebody else may decide that
>
> P = Matrix.Matrix  is a better choice.  But, if we establish that
>
> .M  always returns a matrix for arrays < 2d, then we gain consistency.
>
> I've made this change and am ready to commit the change to the Numeric tree,
> unless there are strong objections.   I know some people do not like the
> proliferation of attributes, but in this case the notational convenience it
> affords to otherwise overly burdened syntax and the consistency it allows
> Numeric to deal with Matrix equations may be worth it.
>
> What do you think?
>
> -Travis Oliphant
>
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

RE: [Numpy-discussion] adding a .M attribute to the array.

[Perry G. <pe...@st...>]

Konrad Hinsen writes:

> > discussed (and never resolved) a couple years ago. Suppose I do this:
> > 
> > x = a.M * libfunc(b.M, c.M)
> > 
> > where libfunc is a 3rd party module written in Python that was written
> > assuming that operators were elementwise operators. It may silently
> 
> Then you are calling a routine with wrong arguments - that can happen
> in Python all the time.
> 
> From my point of view, arrays and matrices are two entirely different
> things. A function written for matrix objects cannot be expected to
> work with array objects, and vice versa. Matrix operations should
> return matrix objects, and array operations should return array
> objects.
> 
> What arrays and matrices have in common is not semantics, but
> implementation. That is something that implementors should profit
> from, but users shouldn't even need to know about.
> 
> The discussion about matrices has focused on matrix multiplication as
> the main difference between the two objects. I suppose this was
> motivated by comparisons to Matlab and similar environments, which do
> not have the notion of data types and thus cannot properly distinguish
> between matrices and arrays. I don't see why should follow this
> limited approach.
> 
> A matrix object should not only do matrix multiplication properly, but
> also provide methods such as diagonalization, application of functions
> as matrix functions, etc. That would be much more than syntactic
> sugar, it would be a real implementation of the mathematical concept
> "matrix".
> 
> Seen from this point of view, it is not at all clear why an array
> should have an attribute that is an "equivalent" matrix, as no such
> equivalence exists in general (only for 2D arrays).
> 
Not an unreasonable position. Are you also arguing that the two types
should know about each other and raise an exception if there is an
attempt to mix them in operations?

Perry

RE: [Numpy-discussion] adding a .M attribute to the array.

[Perry G. <pe...@st...>]

Pearu writes:

>
> Here is one suggestion that is based on the observation that all we need
> is an easy way to tell that the following operation should be applied
> as a matrix operation. So, the suggestion is to provide an attribute or a
> member function that returns an array (note, not a Matrix instance) that
> has a bit, called it asmatrix, set true but _only_ temporally. The bit is
> cleaned on every operation. And before applying an operation, the
> corresponding method (currently there seem to be only four relevant
> methods: __mul__, __pow__ and their r-versions) checks if either of
> the operants has asmatrix bit true, then performs the corresponding matrix
> operation, otherwise the default element-wise operation. And before
> returning, it cleans up asmatrix bit.
>
> For the sake of an example, let .m be Numeric array attribute that when
> accessed sets asmatrix=1 and returns the array.
> Examples:
>
>   a * b       - is element-wise multiplication
>   a.m * b, a * b.m   - are matrix multiplications, the resulting
>                        array, as well a and b, have asmatrix=0
>   a.m ** -1   - is matrix inverse
>   sin(a)      - element-wise sin
>   sin(a.m)    - matrix sin
>
> To summarize the main ideas:
>  * array has asmatrix bit that most of the time is false.
>  * there is a way to set the asmatrix bit true, either by .m or .M
>    attributes or .m(), .M(), .. methods that return the same array.
>  * __mul__, __pow__, etc. methods check if either operant has asmatrix
>    true, then performs the corresponding matrix operation, otherwise
>    the corresponding element-wise operation.
>  * all operations clean asmatrix bit.
>
> So, what do you think?
>
> 	Pearu
>
This is a clever idea that reminds me of  something we were considering
for something else (exactly what I can't quite remember :-). But like
all such schemes it still does produce an object, and a user might
infer (reasonably or unreasonably) that if they can type

x = a.m * b

Then they can treat a.m as an array, e.g.,

x = a.m

In that case, the special case behavior still becomes camouflaged,
when x is used later, albeit only to bite you once. It is clear
what the operation does when the attribute is used in the expression,
but if it isn't, there is still room for confusion. I like the
idea, but I'll have to think about whether the downside outweighs
the benefits.

Perry

RE: [Numpy-discussion] adding a .M attribute to the array.

[Pearu P. <pe...@ce...>]

On Wed, 6 Mar 2002, Paul F Dubois wrote:

> If you have never looked at MA, please examine source file
> Packages/MA/Lib/MA.py before commenting. This file is fairly complex and

You mean those who never used MA should take a day off to read 2000 line
code in order to understand the implications of using MA and give a
comment? ;-)

Indeed, I have never used MA but as a first look it does not look too
promising regarding performance: e.g. there seem to be lots of python code
involved to apply a simple multiplication of arrays. Could someone more
familiar with MA give a comment on performance issues, especially, keeping
in mind number crunchers?

Pearu

RE: [Numpy-discussion] adding a .M attribute to the array.

[Pearu P. <pe...@ce...>]

On Wed, 6 Mar 2002, Perry Greenfield wrote:

> Other suggestions?

Here is one suggestion that is based on the observation that all we need
is an easy way to tell that the following operation should be applied
as a matrix operation. So, the suggestion is to provide an attribute or a
member function that returns an array (note, not a Matrix instance) that
has a bit, called it asmatrix, set true but _only_ temporally. The bit is
cleaned on every operation. And before applying an operation, the
corresponding method (currently there seem to be only four relevant
methods: __mul__, __pow__ and their r-versions) checks if either of 
the operants has asmatrix bit true, then performs the corresponding matrix
operation, otherwise the default element-wise operation. And before
returning, it cleans up asmatrix bit.

For the sake of an example, let .m be Numeric array attribute that when
accessed sets asmatrix=1 and returns the array.
Examples:

  a * b       - is element-wise multiplication
  a.m * b, a * b.m   - are matrix multiplications, the resulting
                       array, as well a and b, have asmatrix=0
  a.m ** -1   - is matrix inverse
  sin(a)      - element-wise sin
  sin(a.m)    - matrix sin

To summarize the main ideas:
 * array has asmatrix bit that most of the time is false.
 * there is a way to set the asmatrix bit true, either by .m or .M
   attributes or .m(), .M(), .. methods that return the same array.
 * __mul__, __pow__, etc. methods check if either operant has asmatrix
   true, then performs the corresponding matrix operation, otherwise
   the corresponding element-wise operation.
 * all operations clean asmatrix bit.

So, what do you think?

	Pearu

RE: [Numpy-discussion] adding a .M attribute to the array.

[Paul F D. <pa...@pf...>]

I believe the correct solution is a major upgrade to Matrix.py along the
lines of what is done in MA; that is, to craft an object that uses
Numeric for its implementation but which defines all its own operators
in a manner that is semantically sensible for the type of object it is. 

Such an upgrade could subsequently be improved by using different
underlying software for various operations, or even more sophisticated
changes such as using a transposed attribute to lazily evaluate
transposes in a cleaner way than Numeric does it. Also an upgrade to
Numarray would then be virtually painless.

If you have never looked at MA, please examine source file
Packages/MA/Lib/MA.py before commenting. This file is fairly complex and
the required changes to Matrix.py would be considerably simpler; but you
can verify that it is fairly straightforward to do. On my project we
have done something similar to create a "climate data variable" object.

Such a design includes an "exit" function to allow the instance to
cheaply view itself as the underlying Numeric array. (In MA, this is
"filled", which makes a Numeric array by replacing missing values, but
if there are no missing values returns the underlying Numeric array).

I'm willing to do this for the community but it would have a side
effect; if anyone has been doing "from Matrix import *" they would
suddenly get a lot more names imported that would conflict with any
imported from Numeric.

[Numpy-discussion] Final prep for 21.0

[Paul F D. <pa...@pf...>]

I have committed to CVS that which I expect to become 21.0. The final
set of changes is a revision to the way we use distutils to make RPMs. I
am unqualified to test these but the submitter (Vermeulen) did.

Please update your CVS and test this version. Developers, please make no
further commits until I make the release. I will make the release this
weekend unless I receive advice to the contrary from testers.

Re: [Numpy-discussion] adding a .M attribute to the array.

[Konrad H. <hi...@cn...>]

"Perry Greenfield" <pe...@st...> writes:

> discussed (and never resolved) a couple years ago. Suppose I do this:
> 
> x = a.M * libfunc(b.M, c.M)
> 
> where libfunc is a 3rd party module written in Python that was written
> assuming that operators were elementwise operators. It may silently

Then you are calling a routine with wrong arguments - that can happen
in Python all the time.

From my point of view, arrays and matrices are two entirely different
things. A function written for matrix objects cannot be expected to
work with array objects, and vice versa. Matrix operations should
return matrix objects, and array operations should return array
objects.

What arrays and matrices have in common is not semantics, but
implementation. That is something that implementors should profit
from, but users shouldn't even need to know about.

The discussion about matrices has focused on matrix multiplication as
the main difference between the two objects. I suppose this was
motivated by comparisons to Matlab and similar environments, which do
not have the notion of data types and thus cannot properly distinguish
between matrices and arrays. I don't see why should follow this
limited approach.

A matrix object should not only do matrix multiplication properly, but
also provide methods such as diagonalization, application of functions
as matrix functions, etc. That would be much more than syntactic
sugar, it would be a real implementation of the mathematical concept
"matrix".

Seen from this point of view, it is not at all clear why an array
should have an attribute that is an "equivalent" matrix, as no such
equivalence exists in general (only for 2D arrays).

> Conceivably there will be modules useful for both kinds of objects. Do

I don't think so. The only analogous operations between arrays and
matrices are addition, subtraction, negation, and multiplication with
a scalar, and those would use the same syntax anyway.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] adding a .M attribute to the array.

[Todd A. P. Ph.D. <tp...@ac...>]

I often (perhaps inappropriately) fall into the "silent user"
category.  However, many of those in this conversation have put
significant effort into python development and the least I can do is
offer a comment from the standpoint of someone who uses python and
Numeric extensively.  Perhaps I am stepping into the middle of a
conversation here -- I hope I have read all the relevant material.
People may "like" Matlab syntax because it requires less typing or
because it pleases them aesthetically.  I personally feel that explicit
function based operators (like transpose()) are very clear and
unambiguous.  While I understand the desire to have the code and the
"math" look similar I think, in general, this is leads to the same
kind of difficulty one has with notation in mathematics -- Notation
that works well in some fields is extremely cumbersome in others.  I
don't expect it to look like an equation. I find orderly, predictable
behavior that doesn't send me to the source code too often to figure
out what is happening very helpful.  Treating 1d or 2d arrays as
matrices is admittedly *very* useful in some applications but
cumbersome in others.  This problem is reminiscent of the "clash"
between the PIL and Numeric modules or between the C-language
row-major matrix storage format and the (in my opinion) better thought
out FORTRAN column-major matrix storage format.  These differences
place limitations on the potential for synergistic profits in the
project.  It is my personal experience/opinion that "convenience"
methods are best added in a specific application that is not intended
to be released generally.

-Todd







* Perry Greenfield (pe...@st...) wrote:
> Travis Oliphant writes:
> >
> >
> > .M  always returns a matrix for arrays < 2d, then we gain consistency.
> >
> > I've made this change and am ready to commit the change to the
> > Numeric tree,
> > unless there are strong objections.   I know some people do not like the
> > proliferation of attributes, but in this case the notational
> > convenience it
> > affords to otherwise overly burdened syntax and the consistency it allows
> > Numeric to deal with Matrix equations may be worth it.
> >
> > What do you think?
> >
> > -Travis Oliphant
> >
> >
> I'd have to agree with Konrad and Paul on this one. While it makes simple
> matrix expressions clearer, it opens a whole can of worms that were
> discussed (and never resolved) a couple years ago. Suppose I do this:
> 
> x = a.M * libfunc(b.M, c.M)
> 
> where libfunc is a 3rd party module written in Python that was written
> assuming that operators were elementwise operators. It may silently
> do a matrix multiplication (depending on the shapes) instead of the
> intended elementwise multiplication. Yet the usage above looks just as
> legitimate as
> 
> x = a.M * b.M
> 
> In other words, it raises the issues of having incompatible modules, some
> written with Numeric objects in mind, others with Matrix objects in mind.
> Conceivably there will be modules useful for both kinds of objects. Do
> we need to support two kinds? How do we deal with this?
> 
> This is still a problem if we don't allow the .M attribute but still have
> widespread usage of a array object with different behavior for operations.
> Unlike masked arrays, whose basic behavior is unchanged for "good" data,
> the behavior for identical data is completely different.
> 
> I wish I had a good answer for this. I don't remember all of the past
> suggestions, but it looks like one of the following solutions is needed:
> 
> 1) Campaign for new operators in Python (various proposals to this affect.
>    This is probably best from the Numeric point of view (maybe not from
>    Python in general though).
> 2) Allow different array classes with different behavior, but come up with
>    conventions and utilities for library developers to produce versions of
>    arrays compatible with the convention assumed for the module (and convert
>    back to the input type for output values). This doesn't prevent all
>    opportunities for confusion and errors however. It also puts a stronger
>    burden on library developers.
> 3) Do nothing and deal with the resulting mess. Perhaps the two camps have
>    little need for each other's tools and it won't be much of a problem.
>    Do option 2 retroactively if it is a problem.
> 
> Other suggestions?
> 
> Perry
> 
> 
> 
> Perry
> 
> 
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Adding a flag to allow integer array access and masking

[Travis O. <oli...@ee...>]

> Please, no.   There are many short names that you could use that would
> avoid overloading the [] operator.   Especially in Python, where
> one cannot trivially decide the type of a variable,
> the behavior should change as little as possible as the type
> of each variable changes.

Right now, what I'm suggesting fails with an error.  Everyone talks about
things not changing when types change but this is actually almost never
the case.

There is almost always a different behavior if the objects have different
types.

Are you opposed to anything going inside the [] operator to help indicate
how the objects inside should be interpreted

 >
> Here, the indexing operation changes completely if you change the
> last index from an int to an array.  That means you have to
> execute the code to understand it -- one can't just look and
> assume from local syntax.

No, only a very specific kind of array.  Currently, such a change gives
you an error.  And if the array was in the wrong place it would also give
you an error.

Your concerns seem motivated by not really understanding the suggested
change.  Could you provide me with other examples that show precisely what
you mean?

> Besides, you know some idiot is going to eventually write
> code that looks like this:
>
> def access(a, b, x):
> 	return a[b, x]  # I think that a must be a 2-D array...
>
> # 1000 lines later...
> access(a, _I)	# Whoops all my assumptions were wrong...

I have no idea, what your concern is here.  This would result in an error
currently and under the scheme I suggested.

-Travis

RE: [Numpy-discussion] adding a .M attribute to the array.

[Perry G. <pe...@st...>]

Travis Oliphant writes:
>
>
> .M  always returns a matrix for arrays < 2d, then we gain consistency.
>
> I've made this change and am ready to commit the change to the
> Numeric tree,
> unless there are strong objections.   I know some people do not like the
> proliferation of attributes, but in this case the notational
> convenience it
> affords to otherwise overly burdened syntax and the consistency it allows
> Numeric to deal with Matrix equations may be worth it.
>
> What do you think?
>
> -Travis Oliphant
>
>
I'd have to agree with Konrad and Paul on this one. While it makes simple
matrix expressions clearer, it opens a whole can of worms that were
discussed (and never resolved) a couple years ago. Suppose I do this:

x = a.M * libfunc(b.M, c.M)

where libfunc is a 3rd party module written in Python that was written
assuming that operators were elementwise operators. It may silently
do a matrix multiplication (depending on the shapes) instead of the
intended elementwise multiplication. Yet the usage above looks just as
legitimate as

x = a.M * b.M

In other words, it raises the issues of having incompatible modules, some
written with Numeric objects in mind, others with Matrix objects in mind.
Conceivably there will be modules useful for both kinds of objects. Do
we need to support two kinds? How do we deal with this?

This is still a problem if we don't allow the .M attribute but still have
widespread usage of a array object with different behavior for operations.
Unlike masked arrays, whose basic behavior is unchanged for "good" data,
the behavior for identical data is completely different.

I wish I had a good answer for this. I don't remember all of the past
suggestions, but it looks like one of the following solutions is needed:

1) Campaign for new operators in Python (various proposals to this affect.
   This is probably best from the Numeric point of view (maybe not from
   Python in general though).
2) Allow different array classes with different behavior, but come up with
   conventions and utilities for library developers to produce versions of
   arrays compatible with the convention assumed for the module (and convert
   back to the input type for output values). This doesn't prevent all
   opportunities for confusion and errors however. It also puts a stronger
   burden on library developers.
3) Do nothing and deal with the resulting mess. Perhaps the two camps have
   little need for each other's tools and it won't be much of a problem.
   Do option 2 retroactively if it is a problem.

Other suggestions?

Perry



Perry

[Numpy-discussion] Re: problem with integer type array object

[Jin W. P. <jw...@ae...>]

Guess, I didn't read the DOC carefully.
I just found out that Python int is equivalent to a C long.
And PyArray_INT doesn't have a corresponding Python scalar type.
Sorry for a 'dumb' question. 

-- 
+----------------------------------------+
| Jin Woo Park (jw...@ae...)|
| Research Assistant,Dept.Aerospace Eng. |
| Seoul National University, Korea       |
+----------------------------------------+

[Numpy-discussion] problem with integer type array object

[Jin W. P. <jw...@ae...>]

I was working on an external module build in C 
where a function returns a 2-D integer array.
However, when I imported module, I found a
strange behavior concerning the type of the element
of the array.
This is what basically happens:


static PyObject* foo(PyObject* arg, PyObject* args) {
    PyObject* a;
    int dims[2]={2,2};
    a = PyArray_FromDims(2,dims,PyArray_INT);
    return a;
}

in python, 
>>> a=foo()
>>> type(a[0,0])
<type 'array'>

I expected the type 'int'. 
If you make a Numpy array in python, then you get the expected
type of 'int'.
>>> b=array([[0,0],[0,0]])
>>> type(b[0,0])
<type 'int'>

Is this somehow intended? 
Thanks for any info,

-- 
+----------------------------------------+
| Jin Woo Park (jw...@ae...)|
| Research Assistant,Dept.Aerospace Eng. |
| Seoul National University, Korea       |
+----------------------------------------+

RE: [Numpy-discussion] Adding a flag to allow integer array access and masking

[Perry G. <pe...@st...>]

Travis Oliphant writes:
> 
> I have not heard any feedback back on my proposal to add a final 
> object to 
> the extended slice syntax to current Numeric to allow for 
> unambiguous index 
> and mask-array access.  
> 
> As a modification to the proposal, suppose we just check to see 
> if the last 
> argument (of at least two) is a 0d array of type signed byte 
> (currently this 
> is illegal and will raise an error).   This number would be a 
> flag indicating 
> how to interpret the previous objects.  Of course these numbers would be 
> hidden from the user who would write:
> 
> a[index_array, _I] = <values>
> b = a[index_array, _I]
> 
> or 
> 
> a[mask_array, _M] = <values>
> b = a[mask_array, _M]
> 
> where _M is a  0d signed byte array indicating that the 
> mask_array should be 
> interpreted as a mask while  _I is a 0d signed byte array 
> indicating that the 
> index_array should be interpreted as a integers into the 
> flattened version of 
> a. 
> 
> Other indexing schemes could be envisioned as well
> 
> a[a1,a2,a3,_X] could be the cross product of the integer arrays 
> a1, a2, and 
> a3 for example.
> 
> or 
> 
> a[a1, a2, a3, _Z] could select elements from a by "zipping" the 
> sequences a1, 
> a2, and a3 together to form a list of tuples to grab from a.
> 
> Comments?
> 
Like Greg I'm wary of having many different interpretations
for indexing behavior (I'm not even that crazy about having
numarray handle boolean index arrays differently than the others
--something we haven't implemented yet, and perhaps we shouldn't).

Before discussing the merits of this, shouldn't we take the attitude
that absence of feedback is not necessarily equivalent to approval,
particularly for something that affects the public interface of
the module? I would feel better about this if I saw several 
affirming the need for such features rather than few openly 
opposing it.

But if one were to do something like this, I would use a different kind
of object than 0d arrays, e.g., an instance of a class defined for just
that purpose. You would really want to make sure that no data could
mistakenly be interpreted as a flag, even if the chances were remote.
I would also not use an underscore as the beginning of the name. Maybe
I'm wrong about this, but I've come to take that to mean its a private
variable that should not be used by users of the module, and that usage
would confuse that. Finally, the name of the flag should be descriptive
(e.g. MaskInd).

But there could be better alternatives. As an example, 

x[nonzero(maskarray)] instead of x[maskarray, MaskInd] 

(Yes, it does generate a temporary so that is a drawback)

Perry

Re: [Numpy-discussion] Adding a flag to allow integer array access and masking

[Greg K. <gp...@be...>]

Please, no.   There are many short names that you could use that would
avoid overloading the [] operator.   Especially in Python, where
one cannot trivially decide the type of a variable,
the behavior should change as little as possible as the type
of each variable changes.

Here, the indexing operation changes completely if you change the
last index from an int to an array.  That means you have to
execute the code to understand it -- one can't just look and
assume from local syntax.


Besides, you know some idiot is going to eventually write
code that looks like this:

def access(a, b, x):
	return a[b, x]  # I think that a must be a 2-D array...

# 1000 lines later...
access(a, _I)	# Whoops all my assumptions were wrong...




> From: Travis Oliphant <oli...@ie...>
> Subject: [Numpy-discussion] Adding a flag to allow integer array access and masking
> 
> 
> I have not heard any feedback back on my proposal to add a final object to 
> the extended slice syntax to current Numeric to allow for unambiguous index 
> and mask-array access.  
> ...hidden from the user who would write:
> 
> a[index_array, _I] = <values>
> b = a[index_array, _I]
> 
> or 
> 
> a[mask_array, _M] = <values>
> b = a[mask_array, _M]
> 
> where _M is a  0d signed byte array indicating that the mask_array should be 
> interpreted as a mask while  _I is a 0d signed byte array indicating that the 
> index_array should be interpreted as a integers into the flattened version of 
> a. 
> 
> Other indexing schemes could be envisioned as well...

Re: [Numpy-discussion] Numerical Python and LAPACK on 64-bit machines

[Konrad H. <hi...@cn...>]

Pearu Peterson <pe...@ce...> writes:

> Having wrapped a lot of Fortran codes to Python, I agree, that Numerical
> Python should use 'int', instead of, 'long'. Though I have little
> influence to make this change to happen in Numeric but just agreeing with
> you.

Even without the Fortran aspect, I'd prefer 'int' for integer arrays
in general. There may be applications that need 64-bit integers, but
any portable application wouldn't rely on them anyway. 64-bit arrays
take up more memory, and when you pickle them you cannot read those
files on 32-bit machines.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] Numerical Python and LAPACK on 64-bit machines

[Pearu P. <pe...@ce...>]

Hi,

On Wed, 6 Mar 2002, Roman Geus wrote:

> So, what really needs to be changed (at least for this machine) is how
> Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers.
> So this means using 'int' instead of 'long int'. 

Having wrapped a lot of Fortran codes to Python, I agree, that Numerical
Python should use 'int', instead of, 'long'. Though I have little
influence to make this change to happen in Numeric but just agreeing with
you.

Pearu

Re: [Numpy-discussion] Numerical Python and LAPACK on 64-bit machines

[Roman G. <ge...@in...>]

Roy Dragseth wrote:
> 
> On Wednesday 06 March 2002 09:50 am, Roman Geus wrote:
> > Dear Numerical Python user and developers
> >
> > I ran into the following problem:
> >
> > The python application I'm developing uses Numerical Python and other
> > C modules that call LAPACK. My application runs well on 32bit
> > architectures:
> >
> > When I tried to run the application on a HP-UX 64bit machine the
> > application produced bus errors. After a long debugging session I
> > found out that Fortran integers are still 32bit wide on this
> > machine. Therefore also the HP LAPACK library has to called using
> > 32bit integers. Numerical Python however codes Fortran integers as C
> > 'long int' variables, which are 64bit wide on this machine.
> 
> Have you tried the +i8 flag for the HP fortran compiler?  It converts all
> fortran integers to 8bytes entities.
> 
> Regards,
> Roy.

I think this wouldn't help. The optimized BLAS/LAPACK library supplied
by HP expects 32bit integers and other software incorporated into my
python application (e.g. SuperLU) calls BLAS/LAPACK library using 32bit
integers (C 'int' type).

So, what really needs to be changed (at least for this machine) is how
Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers.
So this means using 'int' instead of 'long int'. 

Regards,
Roman

Re: [Numpy-discussion] Numerical Python and LAPACK on 64-bit machines

[Roy D. <Roy...@cc...>]

On Wednesday 06 March 2002 09:50 am, Roman Geus wrote:
> Dear Numerical Python user and developers
>
> I ran into the following problem:
>
> The python application I'm developing uses Numerical Python and other
> C modules that call LAPACK. My application runs well on 32bit
> architectures:
>
> When I tried to run the application on a HP-UX 64bit machine the
> application produced bus errors. After a long debugging session I
> found out that Fortran integers are still 32bit wide on this
> machine. Therefore also the HP LAPACK library has to called using
> 32bit integers. Numerical Python however codes Fortran integers as C
> 'long int' variables, which are 64bit wide on this machine.

Have you tried the +i8 flag for the HP fortran compiler?  It converts all 
fortran integers to 8bytes entities.

Regards,
Roy.

[Numpy-discussion] Numerical Python and LAPACK on 64-bit machines

[Roman G. <ge...@in...>]

Dear Numerical Python user and developers

I ran into the following problem:

The python application I'm developing uses Numerical Python and other
C modules that call LAPACK. My application runs well on 32bit
architectures:

When I tried to run the application on a HP-UX 64bit machine the
application produced bus errors. After a long debugging session I
found out that Fortran integers are still 32bit wide on this
machine. Therefore also the HP LAPACK library has to called using
32bit integers. Numerical Python however codes Fortran integers as C
'long int' variables, which are 64bit wide on this machine.

To make my application run on the HP-UX 64bit machine, I had to change
all 'long int' to 'int' variables in Src/lapack_litemodule.c, which
is a rather painful hack (see end of message for an example).

My question is: 

Should Fortran integers not be coded as 'int' instead of 'long int' in
Numerical Python? This way, it would still work on all 32 bit machines
and also on the 64-bit machines I know.

Would this work on all 64-bit machines?

Thanks for your comments/help.

-- Roman Geus



E.g. the lapack_lite_dgetrf() function now looks like this:


static PyObject *lapack_lite_dgetrf(PyObject *self, PyObject *args)
{
int  lapack_lite_status__;
int m;
int n;
PyObject *a;
int lda;
PyObject *ipiv;
int info;

 int i; 
 int *ipiv_int;
 int ipiv_len;

TRY(PyArg_ParseTuple(args,"iiOiOi",&m,&n,&a,&lda,&ipiv,&info));

TRY(lapack_lite_CheckObject(a,PyArray_DOUBLE,"a","PyArray_DOUBLE","dgetrf")); 
TRY(lapack_lite_CheckObject(ipiv,PyArray_LONG,"ipiv","PyArray_LONG","dgetrf"));  
 ipiv_len = m < n ? m : n;
 ipiv_int = (int *)malloc(ipiv_len * sizeof(int));
 assert(ipiv_int);
 for (i = 0; i < ipiv_len; i ++)
   ipiv_int[i] = LDATA(ipiv)[i];

#if defined(NO_APPEND_FORTRAN)
lapack_lite_status__ = dgetrf(&m,&n,DDATA(a),&lda,ipiv_int,&info);
#else
lapack_lite_status__ = dgetrf_(&m,&n,DDATA(a),&lda,ipiv_int,&info);
#endif

 for (i = 0; i < ipiv_len; i ++)
   LDATA(ipiv)[i] = ipiv_int[i];

 free(ipiv);

return
Py_BuildValue("{s:l,s:l,s:l,s:l,s:l}","dgetrf_",(long)lapack_lite_status__,"m",(long)m,"n",(long)n,"lda",(long)lda,"info",(long)info);
}

Re: [Numpy-discussion] adding a .M attribute to the array.

[Konrad H. <hi...@cn...>]

Travis Oliphant <oli...@ie...> writes:

> I've made this change and am ready to commit the change to the Numeric tree, 
> unless there are strong objections.   I know some people do not like the 
> proliferation of attributes, but in this case the notational convenience it 

At the risk of sounding unconstructively negative, I think this is a
misuse of attributes. For someone used to read standard Python code,
where attributes are, well, attributes, code using this notation is
just weird.

Personally, consistent notation is more important than short notation.
The Pythonesque solution to this problem, in my opinion, is separate
matrix and array objects (which can and should of course share
implementation code) plus explicit constructors to convert between the
two. I am a bit worried that kludges such as fake attributes set
bad precedents for the future.

One of the main reasons why I like Python is its clean syntax and its
simple object model. This kind of notation messes up both of them.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] adding a .M attribute to the array.

[Pearu P. <pe...@ce...>]

Hi!

On Tue, 5 Mar 2002, Travis Oliphant wrote:

> The suggestion was made to add ".M" as an attribute of arrays which returns a 
> matrix.  Thus, the code above can be written:
> 
> sin(a).M * cos(a).M.T
> 
> While some aesthestic simplicity is obtained, the big advantage is in 
> consistency.
 
> I've made this change and am ready to commit the change to the Numeric tree, 
> unless there are strong objections.   I know some people do not like the 
> proliferation of attributes, but in this case the notational convenience it 
> affords to otherwise overly burdened syntax and the consistency it allows 
> Numeric to deal with Matrix equations may be worth it.
> 
> What do you think?

Would it be possible to use own Matrix classes instead of what is in
Matrix.py? I gather that there must be some setter method in Numeric for
that:

Numeric.set_matrix_factory(MyMatrixClass)

with a requirement that MyMatrixClass must be a subclass of Matrix.Matrix.

I think it would be a very important feature as users can define their own
matrix operations, for example, using their own BLAS routines to
speed up operations with matrices (yes, I am thinking of SciPy specific
Matrix class).

Thanks,
	Pearu