You can subscribe to this list here.
2000 
_{Jan}
(8) 
_{Feb}
(49) 
_{Mar}
(48) 
_{Apr}
(28) 
_{May}
(37) 
_{Jun}
(28) 
_{Jul}
(16) 
_{Aug}
(16) 
_{Sep}
(44) 
_{Oct}
(61) 
_{Nov}
(31) 
_{Dec}
(24) 

2001 
_{Jan}
(56) 
_{Feb}
(54) 
_{Mar}
(41) 
_{Apr}
(71) 
_{May}
(48) 
_{Jun}
(32) 
_{Jul}
(53) 
_{Aug}
(91) 
_{Sep}
(56) 
_{Oct}
(33) 
_{Nov}
(81) 
_{Dec}
(54) 
2002 
_{Jan}
(72) 
_{Feb}
(37) 
_{Mar}
(126) 
_{Apr}
(62) 
_{May}
(34) 
_{Jun}
(124) 
_{Jul}
(36) 
_{Aug}
(34) 
_{Sep}
(60) 
_{Oct}
(37) 
_{Nov}
(23) 
_{Dec}
(104) 
2003 
_{Jan}
(110) 
_{Feb}
(73) 
_{Mar}
(42) 
_{Apr}
(8) 
_{May}
(76) 
_{Jun}
(14) 
_{Jul}
(52) 
_{Aug}
(26) 
_{Sep}
(108) 
_{Oct}
(82) 
_{Nov}
(89) 
_{Dec}
(94) 
2004 
_{Jan}
(117) 
_{Feb}
(86) 
_{Mar}
(75) 
_{Apr}
(55) 
_{May}
(75) 
_{Jun}
(160) 
_{Jul}
(152) 
_{Aug}
(86) 
_{Sep}
(75) 
_{Oct}
(134) 
_{Nov}
(62) 
_{Dec}
(60) 
2005 
_{Jan}
(187) 
_{Feb}
(318) 
_{Mar}
(296) 
_{Apr}
(205) 
_{May}
(84) 
_{Jun}
(63) 
_{Jul}
(122) 
_{Aug}
(59) 
_{Sep}
(66) 
_{Oct}
(148) 
_{Nov}
(120) 
_{Dec}
(70) 
2006 
_{Jan}
(460) 
_{Feb}
(683) 
_{Mar}
(589) 
_{Apr}
(559) 
_{May}
(445) 
_{Jun}
(712) 
_{Jul}
(815) 
_{Aug}
(663) 
_{Sep}
(559) 
_{Oct}
(930) 
_{Nov}
(373) 
_{Dec}

S  M  T  W  T  F  S 

1

2

3
(1) 
4

5

6

7

8

9
(3) 
10

11
(1) 
12

13
(22) 
14
(1) 
15
(5) 
16
(9) 
17
(4) 
18
(2) 
19
(2) 
20
(2) 
21
(1) 
22

23

24
(3) 
25

26
(2) 
27
(1) 
28

29

30
(1) 





From: Zaur Shibzoukhov <szport@fr...>  20020930 09:10:21

I propose to add two function to multiarray module in Numeric: tofile(array, openedfile) array = fromfile(opendfile, dimension, typecode = "l") Now I use these functions in my module. Here is an implementation from module: ###############################################################################3 #include "Python.h" #define PY_ARRAY_UNIQUE_SYMBOL PyArray_arrayfile #include "Numeric/arrayobject.h" #define MAX_DIMS 40 static PyObject * PyArray_FromFile(PyObject *self, PyObject *args) { PyObject *f, *dimobj, *temp; PyArray_Descr *descr; PyArrayObject *ret = NULL; int i, n, dim[40], nt; char *typecode = "l"; FILE *fp; int type_num, N = 1; size_t nread; if (!PyArg_ParseTuple(args, "O!O!s#:fromfile", \ &PyFile_Type, &f, &PyTuple_Type, &dimobj, &typecode, &nt)) return NULL; fp = PyFile_AsFile(f); if (fp == NULL) { PyErr_SetString(PyExc_TypeError, "1st argument must be open file"); return NULL; } n = PyTuple_Size(dimobj); if (n > MAX_DIMS) { PyErr_SetString(PyExc_TypeError, "dimension is too large"); return NULL; } if (n > 0) { for (i = 0; i < n; i++) { temp = PyTuple_GetItem(dimobj, i); dim[i] = (int)PyInt_AsLong(temp); } descr = PyArray_DescrFromType(*typecode); type_num = descr > type_num; ret = (PyArrayObject *)PyArray_FromDims(n, dim, type_num); memcpy(ret>data, descr>zero, N*descr>elsize); N = 1; for (i = 0; i < n; i++) N *= dim[i]; nread = fread((char *)ret>data, descr>elsize, N, fp); if (nread < (size_t)N) { PyErr_SetString(PyExc_EOFError, "not enough items in file"); return NULL; } } return (PyObject *)ret; } static char PyArray_FromFile_doc [] = "fromfile(f, dimension, typecode = \'l\')\n\ \n\ Create array from open file f with given dimension and typecode.\n\ Note, that file must be open in binary mode."; static PyObject * PyArray_ToFile(PyObject *self, PyObject *args) { PyObject *f; PyArrayObject *A; int N; FILE *fp; if (!PyArg_ParseTuple(args, "O!O!:tofile", &PyArray_Type, &A, &PyFile_Type, &f)) return NULL; fp = PyFile_AsFile(f); if (fp == NULL) { PyErr_SetString(PyExc_TypeError, "arg must be open file"); return NULL; } N = PyArray_SIZE(A); if (N > 0) { if (fwrite(A>data, A>descr>elsize, N, fp) != (size_t)N) { PyErr_SetFromErrno(PyExc_IOError); clearerr(fp); return NULL; } } Py_INCREF(Py_None); return Py_None; } static char PyArray_ToFile_doc [] = "tofile(array, f)\n\ \n\ Write array to file open f.\n\ Note, that file must be open in binary mode."; static PyMethodDef arrayfile_module_methods[] = { {"fromfile", PyArray_FromFile, 1, PyArray_FromFile_doc}, {"tofile", PyArray_ToFile, 1, PyArray_ToFile_doc}, {NULL, NULL} /* sentinel */ }; ############################################################################ Zaur Shibzoukhov 
From: Leonardo Santagada <retype@te...>  20020927 17:16:05

confirm 498554 
From: Mathew Yeates <mathew@co...>  20020926 23:12:54

Hmmm, aparently, I can cast a long array to a double array but not to a float array. Check out the following code from arrayobject.c (Numeric 22.0) [this is a switch on the type being casted from] case PyArray_LONG: /*Shouldn't allow Longs to be cast to Ints, not safe on 64bit machines!*/ return (totype >= PyArray_INT) && (totype != PyArray_FLOAT); This makes absolutely no sense to me. Is this a bug? Mathew  M47h3w `/34735 
From: Mathew Yeates <mathew@co...>  20020926 00:23:42

The following code produces an error  a=array(([1,2,3,4,5],[10,9,8,7,6])).astype(Float32) cond=array(([0,0,0,0,0],[1,1,1,1,1]) b=array(([0,0,0,0,0],[0,0,0,0,0])) choose(cond,(a,b)) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: Array can not be safely cast to required type  I suppose I might need to cast b to Float32. But, I have a function which is passed a and b as args. Ideally, I would like a function which casts a and b to the highest compatible type since "choose" doesnt do it. How can I do this simply? Mathew  M47h3w `/34735 
From: Perry Greenfield <perry@st...>  20020924 21:32:31

Konrad Hinsen writes: > "Perry Greenfield" <perry@...> writes: > > > Questions: > > > > 1) given 2, is there still a desire for .reduce() to return > > rank0 arrays (if not, we have .areduce() which is intented to return > > arrays always). > > > > 2) whichever is the "returns arrays always" reduce method, should the > > endpoint be rank0 arrays or rank1 len1 arrays? > > I don't really see an application where a reduction operation yielding > rank1 or higher arrays would be useful. It would be a special case, > not useful for generic programming. So my answer to 2) is rank0. > What I am wondering is what behavior suits "generic" programming more. Eric Jones and Paul Dubois have given examples where having to deal with entities that may be scalars or arrays is a pain. But, having said that, the behavior that Eric wanted was not consistent with what many thought rank0 arrays should have (i.e., len(rank0)=1, rank0[0] = value). The proposal to generate rank1 len1 arrays was made since len() of these arrays is = 1, and indexing with [0] does work. So for the kinds of examples he gave, rank1 len1 arrays appear to allow for more generic code. But I'm not trying to speak for everyone; that's why I'm asking for opinions. Do you have examples where you find rank0 arrays make for more generic code in your cases when len() and indexing on these does not have the behavior Eric wanted? May I see a couple? > As for 1), if indexing doesn't return rank0 arrays, then standard > reduction shouldn't either. We would then have a system in which > rank0 arrays are "expert only" stuff, most users would never see > them, and they could safely be ignored in tutorials. > That's my inclination, but I think that the question of whether there should be some reduce mechanism that returns arrays always is still a valid one. I can see that there are good uses for that (or at least a function to cast a scalar to a rank1 len1 array if it isn't already an array, like what array() does except that it now generates rank0 arrays). Perry 
From: Konrad Hinsen <hinsen@cn...>  20020924 21:18:37

"Perry Greenfield" <perry@...> writes: > Questions: > > 1) given 2, is there still a desire for .reduce() to return > rank0 arrays (if not, we have .areduce() which is intented to return > arrays always). > > 2) whichever is the "returns arrays always" reduce method, should the > endpoint be rank0 arrays or rank1 len1 arrays? I don't really see an application where a reduction operation yielding rank1 or higher arrays would be useful. It would be a special case, not useful for generic programming. So my answer to 2) is rank0. As for 1), if indexing doesn't return rank0 arrays, then standard reduction shouldn't either. We would then have a system in which rank0 arrays are "expert only" stuff, most users would never see them, and they could safely be ignored in tutorials. Konrad.   Konrad Hinsen  EMail: hinsen@... Centre de Biophysique Moleculaire (CNRS)  Tel.: +332.38.25.56.24 Rue Charles Sadron  Fax: +332.38.63.15.17 45071 Orleans Cedex 2  Deutsch/Esperanto/English/ France  Nederlands/Francais  
From: Perry Greenfield <perry@st...>  20020924 20:39:51

[I posted this almost a week ago, but apparently an email problem prevented it from actually getting posted!] I think there has been sufficient discussion about rank0 arrays to make some decisions about how numarray will handle them. [If you don't want to wade through the rationale, jump to the end where there is a short summary of what we plan to do and what we have questions about] ******************************************************************** First I'd like to take a stab at summarizing the case made for rank0 arrays in general and adding some of my own comments regarding these points. 1) rank0 arrays are a useful mechanism to avoid having binary operations with scalars cause unintended promotion of other arrays to larger numeric types (e.g. 2*arange(10, typecode=Int16) results in an Int32 result). *** For numarray this is a nonissue because the coercion rules prevent scalars from increasing the type of an array if the scalar is the same kind of number (e.g., Int, Float, Complex) as the array. 2) rank0 arrays preserve the type information instead of converting scalars to Python scalars. *** This seems of limited value. With only a couple possible exceptions in the future (and none now), Python scalars are effectively the largest type available so no information is lost. One can convert to and from Python scalars and not lose any information. The possible future exceptions are long doubles and UInt32 (supported in Numeric, but not numarray yetbut frankly, I'm not yet sure how important UInt32 is at the moment). It is possible that Python scalars may move up in size so this may or may not become an issue in the future. By itself, it does not appear to be a compelling reason. 3) rank0 arrays allow controlling exceptions (e.g. divide by zero) in a way different from how Python handles them (exception always) *** This is a valid concern...maybe. I was more impressed by it initially, but it occurred to me that most expressions that involve a scalar exception (NaN, dividebyzero, overflow, etc.) generally corrupt everything, unlike exceptions with arrays where only a few values may be tainted. Unless one is only interested in ignoring some scalar results in a scalar expression as part of a larger computation, it seems of very limited use to ignore, or warn on scalar exceptions. In any event, this is really of no relevance to the use of rank0 for indexed results or reduction operations. 4) Using rank0 arrays in place of scalars would promote more generic programming. This was really the last point of real contention as far as I was concerned. In the end, it really came down to seeing good examples of how lacking this caused code to be much worse than it could be with rank0 arrays. There really are two cases being discussed: whether indexing a single item ("complete dereferencing" in Scott Gilbert's description) returns rank0, and whether reduction operations return rank0. *** indexing returning rank0. Amazingly enough no one was able to provide even one real code example of where rank0 returns from indexing was a problem (this includes MA as far as I can tell). In the end, this has been much ado about nothing. Henceforth, numarray will return Python scalars when arrays are indexed (as it does currently). *** reduction operations. There are good examples of where reduction operations returning rank0 are made simpler. However, the situation is muddied quite a bit by other issues which I will discuss below. This is an area that deserves a bit more discussion in general. But before I tackle that, there is a point about rank0 arrays that needs to be made which I think is in some respects is an obvious point, but somehow got lost in much of the discussions. Even if it were true that rank0 arrays made for much simpler, generic code, they are far less useful than might appear in simplifying code. Why? Because even if array operations (whether indexing, reduction or other functions) were entirely consistent about never returning scalars, it is a general fact that most Numeric/numarray code must be prepared to handle Python scalars thrown at it in place of arrays by the user. Since Python scalars can come leaking into your code at many points, consistency in Numeric/numarray in avoiding Python scalars really doesn't solve the issue. I would hazard a guess that the great majority of the conditional code that exist today would not be eliminated because of this (e.g., this appears to be the case for MA) Reduction operations: There is a good case to be made that reduction operations should result in rank0 arrays rather than scalars (after all, they are reducing dimensions), but not everyone agrees that is what should be done. But before deciding what is to be done there, some problems with rank0 arrays should be discussed. I think Konrad and Huaiyu have made very powerful arguments about how certain operations like indexing, attributes and such should or shouldn't work. In particular, some current Numeric behaviors should be changed. Indexing by 0 should not work (instead Scott Gilbert's suggestion of indexing with an empty tuple sounds right, if a bit syntatically clumsy due to Python not accepting an empty index). len should not return 1, etc. So even if we return rank0 values in reduction operations, this appears to still cause problems with some of the examples given by Eric that depend on len(rank0) = 1. What should be done about that? One possibility is to use different numarray functions designed to help write generic code (e.g., an alternate function to len). But there is one aspect to this that ought to be pointed out. Some have asked for rank0 arrays that can be indexed with 0 and produce len of 1. There is such an object that does this and it is a rank1 len1 array. One alternative is to have reduction operations have as their endpoint a rank1 len1 array rather than a rank0 array. The rank0 endpoint is more justified conceptually, but apparently less practical. If a reduction on a 1d arrray always produced a 1d array, then one can always be guaranteed that it can be indexed, and that len works on it. The drawback is that it can never be used as a scalar directly as rank0 arrays could be. I think this is a case where you can't have it both ways. If you want a scalarlike object, then some operations that work on higher rank arrays won't work on it (or shouldn't). If you want something where these operations do work, don't expect to use it where a scalar is expected unless you index it. Is there any interest in this alternate approach to reductions? We plan to have two reduction methods available, one that results in scalars, and one in arrays. The main question is which one .reduce maps to, and what the endpoint is for the method that always returns arrays is. *********************************************************************** SUMMARY *********************************************************************** 1) Indexing returns Python scalars in numarray. No rank0 arrays are ever returned from indexing. 2) rank0 arrays will be supported, but len(), and indexing will not work as they do in Numeric. In particular, to get a scalar, one will have to index with an empty tuple (e.g., x[()], x[0] will raise an exception), len() will return None. Questions: 1) given 2, is there still a desire for .reduce() to return rank0 arrays (if not, we have .areduce() which is intented to return arrays always). 2) whichever is the "returns arrays always" reduce method, should the endpoint be rank0 arrays or rank1 len1 arrays? 
From: Rob <rob@py...>  20020921 00:40:14

Konrad Hinsen wrote: > <snip> What I like about Numeric stems from what I like about Python the language does not get in my way of creating algorithms. I do not have to constantly refer to some reference book while programming. The idea of the Numeric Python EM Project (URL below) was to put EM code into a form where it could be more easily understood. Python and Numeric Python were the perfect choice for that task. As for Mathematica and other commercial software, I have no use for them. Python is free. Sincerely, Rob.   The Numeric Python EM Project http://www.pythonemproject.com 
From: Travis N. Vaught <travis@en...>  20020920 17:54:24

Thought I'd reply to all since I'm including some links that are possibly interesting to the list: > Original Message > From: numpydiscussionadmin@... > [mailto:numpydiscussionadmin@...]On Behalf Of Konrad > Hinsen > Sent: Friday, September 20, 2002 12:04 PM > To: numpydiscussion@... > Subject: [Numpydiscussion] A message from Cameron Laird > > > I send this on behalf of Cameron Laird <claird@...>. > Please reply to him, not to me. > > I have at least a couple of assignments from magazines such > as IBM's developerWorks to report on matters that involve > Numeric. I'd welcome contact from anyone here who wants to > publicize his or her work with Python and Numeric. > The SciPy Toolkit (http://www.scipy.org) attacks Matlab's functionality headonproviding much of the functional interface symantics that Matlab provides and much much more. It is a stated goal of many in the SciPy development community to eliminate the need for Matlab altogether by providing a tool that offers _all_ the functionality and better performance (among many other things) in an opensource, truly objectoriented package. > I have a particular interest in advantages Python and Numeric > enjoy over such alternatives as Mathematica, IDL, SAS/IML, > MATLAB, and so on, all of which are more narrowly targeted at > the kinds of scientific and engineering problems tackled by > contributors to this mailing list. What does Python do for > you that the commercial products don't? > > I suspect that many of you will mention, in one form or > another, Python's aptness for programming "in the large". > Do you have specific examples of how this is clumsy in > MATLAB, Mathematica, and so on? > > Have you tried to interface MATLAB and so on to hardware > instrumentation or other external data sources? > The recent SciPy '02 workshop (http://www.scipy.org/site_content/scipy02) had some presented material that tangentially addressed interfacing Matlab and Mathematica (perhaps even in a bof, I've slept since then). You might try the scipyuser mailing list as well. (Presentations: http://www.scipy.org/site_content/scipy02/presentations , Mailing list: http://www.scipy.org/site_content/MailList) > How do the scientists and engineers (as opposed to the > "informaticians" or software developers) on your teams > accept Python, compared to IDL and friends? Do scientists > at your site program? > > Is there anything Python's missing in its competition with > MATLAB and so on? > The aforementioned workshop included a survey that addressed these issues as well. You can look at the aggregated results by following the link below (it's a little raw in its layout, let me know if there is a question about how to interpret some things). http://www.scipy.org/site_content/scipy02/survey_results.htm (note: some questions were 'pick one' and others were 'pick all that apply'the total number of surveys turned in was 36, I think.) > > Cameron Laird <Cameron@...> +1 281 996 8546 FAX > http://phaseit.net/claird/misc.writing/publications.html > 
From: Konrad Hinsen <hinsen@cn...>  20020920 17:04:58

I send this on behalf of Cameron Laird <claird@...>. Please reply to him, not to me. I have at least a couple of assignments from magazines such as IBM's developerWorks to report on matters that involve Numeric. I'd welcome contact from anyone here who wants to publicize his or her work with Python and Numeric. I have a particular interest in advantages Python and Numeric enjoy over such alternatives as Mathematica, IDL, SAS/IML, MATLAB, and so on, all of which are more narrowly targeted at the kinds of scientific and engineering problems tackled by contributors to this mailing list. What does Python do for you that the commercial products don't? I suspect that many of you will mention, in one form or another, Python's aptness for programming "in the large". Do you have specific examples of how this is clumsy in MATLAB, Mathematica, and so on? Have you tried to interface MATLAB and so on to hardware instrumentation or other external data sources? How do the scientists and engineers (as opposed to the "informaticians" or software developers) on your teams accept Python, compared to IDL and friends? Do scientists at your site program? Is there anything Python's missing in its competition with MATLAB and so on? Cameron Laird <Cameron@...> +1 281 996 8546 FAX http://phaseit.net/claird/misc.writing/publications.html 
From: Todd Miller <jmiller@st...>  20020919 16:48:19

Aureli Soria Frisch wrote: > Hi, > > > I have developed till now all my numerical stuff using Numeric objects. > > > I want to use some modules using numarray. I have tested some > conversion functions on Python, e.g. > > > numarray.array(Numeric_object.tolist()) > Much faster, but less general, is: numarray.fromstring(Numeric_object.tostring(), type=XXXX) > > but it seems to me a bit slow (specially for large arrays as images). > Is there any more ways of doing this conversion? > > Should I write my own conversion in C? > It depends on how soon you need it to be fast. numarray fromlist/tolist should be implemented in C for the next release, which should occur within a couple months, perhaps sooner. If you want to write a C extension for numarray now, the best way to do it is to use the Numeric compatability layer, which essentially uses a subset of the Numeric CAPI, and is not likely to change. > Which advices could you give me in order to make an smoother transition? > > > Furthermore I found the web site about numarray/Numeric > incompatibilities very interesting. However I did not understand quite > well the paragraph: > I think that webpage is out of date. > > since interactively typing a.shape() or a.shape((8,8)) raise an > exception (Tuple object not callable). Am I misunderstanding something? > Actually, for python 2.2 and up, numarray has all of those attributes in a Numeric compatible, Python properties based form. So ".shape" get/sets the shape in numarray, just as in Numeric: >>> import numarray >>> a = numarray.arange(10) >>> a.shape (10,) >>> a.shape = (2,5) >>> a.shape (2,5) Because of this, the expression a.shape() that you mentioned tries to "call" the shape tuple returned by .shape, resulting in an exception. Todd  Todd Miller jmiller@... STSCI / SSB 
From: Aureli Soria Frisch <aureli@ip...>  20020919 16:17:52

Hi, I have developed till now all my numerical stuff using Numeric objects. I want to use some modules using numarray. I have tested some conversion functions on Python, e.g. numarray.array(Numeric_object.tolist()) but it seems to me a bit slow (specially for large arrays as images). Is there any more ways of doing this conversion? Should I write my own conversion in C? Which advices could you give me in order to make an smoother transition? Furthermore I found the web site about numarray/Numeric incompatibilities very interesting. However I did not understand quite well the paragraph: >Numeric arrays have some public attributes. Numarray arrays >have none. All changes to an array's state must be made >through accessor methods. Specifically, Numeric has the following >attributes: > >Numeric numarray accessor method(s) >Attribute > >shape > shape() (i.e., specify new shape through method arg instead >of > assigning to shape attribute) >flat > flat() >real > real() (set capability not implemented yet, but will be) >imag > imag() (ditto) >savespace > not used, no equivalent functionality (xxx check this) since interactively typing a.shape() or a.shape((8,8)) raise an exception (Tuple object not callable). Am I misunderstanding something? Thanks in advance. Best regards, Aureli  ################################# Aureli Soria Frisch Fraunhofer IPK Dept. Pattern Recognition post: Pascalstr. 89, 10587 Berlin, Germany email: aureli@... fon: +49 30 39006143 fax: +49 30 3917517 web: http://vision.fhg.de/~aureli/webaureli_en.html ################################# 
From: Ralf Juengling <juenglin@in...>  20020918 08:15:11

On Mon, 20020916 at 22:11, Tim Hochberg wrote: >=20 > Can someone refresh my memory as to why some properties of NumArrays that > are conceptually attributes are accessed as atributes (a.shape, a.rank), > while others are accessed through functions (a.type, a.iscontiguous,...). Yes, I'd like to understand as well; there is even a third. There are attributes, e.g. a.shape vs methods, e.g. a.type() vs functions e.g. len(a). Regards, Ralf >=20 > Thanks, >=20 > tim >=20 >=20 >=20 >=20 >  > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpydiscussion mailing list > Numpydiscussion@... > https://lists.sourceforge.net/lists/listinfo/numpydiscussion =20  Ralf J=FCngling Institut f=FCr Informatik  Lehrstuhl f. Mustererkennung & Bildverarbeitung GeorgesK=F6hlerAllee Geb=E4ude 52 Tel: +49(0)7612038215 79110 Freiburg Fax: +49(0)7612038262  
From: Huaiyu Zhu <huaiyu_zhu@ya...>  20020918 07:02:29

On Tue, 17 Sep 2002, Travis Oliphant wrote: > > len(d) == Exception == d.shape[0] > > > > # Currently the last is wrong? > > Agreed, but this is because d is an integer and out of Numerics control. > This is a case for returning 0d arrays rather than Python scalars. That is one problem. It can be removed by using shape(d). More fundamentally, though, len(d) == shape(d)[0] == ()[0] => IndexError. I think Konrad made this point a few days back. > > size(d) == 1 == product(d.shape) > > > > # Currently the last is wrong > > I disagree that this is wrong. This works as described for me. Right. Change d.shape to shape(d) here (and in several other places). > Why is this? I thought you argued the other way for len(scalar). Of > course, one solution is that we could overwrite the len() function and > allow it to work for scalars. Raising exception is the correct behavior, not a problem to be solved. > > > > Conclusion 5: rank0 int arrays should be allowed to act as indices. > > See property 5. > > Can't do this for lists and other builtin sequences. If numarray defines a consistent set of behaviors for integer types that is intuitively understandable, it might not be difficult to persuade core Python to check against an abstract integer type. > >  Is there substantial difference in overhead between rank0 arrays and > > scalars? > > Yes. That would be one major problem. However, after giving this some more thoughts, I'm starting to doubt the analogy I made. The problem is that in the end there is still a need to index an array and obtain a good old immutable scalar. So  What notation should be used for this purpose? We can use c[0] to get immutable scalars and c[0,] for rank0 arrays / mutable scalars. But what about other ranks? Python does not allow distinctions based on a[1,1,1] versus a[(1,1,1)] or d[] versus d[()].  This weakens the argument that rank0 arrays are scalars, since that argument is essentially based on sum(c) and c[0] being of the same type. Huaiyu 
From: Travis Oliphant <oliphant@ee...>  20020917 16:09:33

> > Some of the choices between rank0 arrays and new scalar types > might be resolved by enumerating the properties desired of them. > These are helpful observations. > Most properties of rank0 arrays could be fixed by consistency > requirements alone, using operations that reduce array dimensions. > > Let a = ones((2,3,4)) > b = sum(a) > c = sum(b) > d = sum(c) > > Property 1: the shape of an array is a tuple of integers > a.shape == (2, 3, 4) > b.shape == (3, 4) > c.shape == (4,) > d.shape == () > > Property 2: rank(a) == len(a.shape) > rank(a) == 3 == len(a.shape) > rank(b) == 2 == len(b.shape) > rank(c) == 1 == len(c.shape) > rank(d) == 0 == len(d.shape) > > Property 3: len(a) == a.shape[0] > len(a) == 2 == a.shape[0] > len(b) == 3 == b.shape[0] > len(c) == 4 == c.shape[0] > len(d) == Exception == d.shape[0] > > # Currently the last is wrong? Agreed, but this is because d is an integer and out of Numerics control. This is a case for returning 0d arrays rather than Python scalars. > > Property 4: size(a) == product(a.shape) > size(a) == 24 == product(a.shape) > size(b) == 12 == product(b.shape) > size(c) == 4 == product(c.shape) > size(d) == 1 == product(d.shape) > > # Currently the last is wrong I disagree that this is wrong. This works as described for me. > > Property 5: rank0 array behaves as mutable numbers when used as value > array(2) is similar to 2 > array(2.0) is similar to 2.0 > array(2j) is similar to 2j > > # This is a summary of many concrete properties. > > Property 6: Indexing reduces rank. Slicing preserves rank. > a[:,:,:].shape = (2, 3, 4) > a[1,:,:].shape = (3, 4) > a[1,1,:].shape = (4,) > a[1,1,1].shape = () > > Property 7: Indexing by tuple of ints gives scalar. > a[1,1,1] == 1 > b[1,1] == 2 > c[1,] == 6 > d[()] == 24 > > # So rank0 array indexed by empty tuple should be scalar. > # Currently the last is wrong Not sure about this property, but interesting. > > Property 8: Indexing by tuple of slices gives array. > a[:,:,:] == ones((2,3,4)) > b[:,:] == ones((3,4)) * 2 > c[:] == ones((,4)) * 6 > d[()] == ones(()) * 24 > > # So rank0 array indexed by empty tuple should be rank0 array. > # Currently the last is wrong Not sure about this one either. > > Property 9: Indexing as lvalues > a[1,1,1] = 2 > b[1,1] = 2 > c[1,] = 2 > d[()] = 2 > > Property 10: Indexing and slicing as lvalues > a[:,:,:] = ones((2, 3, 4)) > a[1,:,:] = ones((3, 4)) > a[1,1,:] = ones((4,)) > a[1,1,1] = ones(()) > > # But the last is wrong. > > > Conclusion 1: rank0 arrays are equivalent to scalars. > See properties 7 and 8. > > Conclusion 2: rank0 arrays are mutable. > See property 9. > > Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) > should not be defined. Why is this? I thought you argued the other way for len(scalar). Of course, one solution is that we could overwrite the len() function and allow it to work for scalars. > > See conclusion 1 and properties 1, 2, 3, 4. > > Conclusion 4: A missing axis is similar to having dimension 1. > See property 4. > > Conclusion 5: rank0 int arrays should be allowed to act as indices. > See property 5. Can't do this for lists and other builtin sequences. > > Conclusion 6: rank0 arrays should not be hashable except by object id. > See conclusion 2. > > > Discussions: > >  These properties correspond to the current implementation quite well, > except a few rough edges. > >  Mutable scalars are useful in their own rights. > >  Is there substantial difference in overhead between rank0 arrays and > scalars? Yes. > >  How to write literal values? array(1) is too many characters. > >  For rank1 and rank0 arrays, Python notation distinguishes: > > c[1] vs c[1,] > d[] vs d[()] > > Should these be used to handle semantic difference between indexing > and slicing? Should d[] be syntactically allowed? > > Hope these observations help. Thanks for the observations. Travis O. 
From: Huaiyu Zhu <huaiyu_zhu@ya...>  20020917 07:11:41

Oops, the subject line was somehow cut off. Please use this one if you follow up.  Huaiyu On Tue, 17 Sep 2002, Huaiyu Zhu wrote: Some of the choices between rank0 arrays and new scalar types might be resolved by enumerating the properties desired of them. Most properties of rank0 arrays could be fixed by consistency requirements alone, using operations that reduce array dimensions. Let a = ones((2,3,4)) b = sum(a) c = sum(b) d = sum(c) Property 1: the shape of an array is a tuple of integers a.shape == (2, 3, 4) b.shape == (3, 4) c.shape == (4,) d.shape == () Property 2: rank(a) == len(a.shape) rank(a) == 3 == len(a.shape) rank(b) == 2 == len(b.shape) rank(c) == 1 == len(c.shape) rank(d) == 0 == len(d.shape) Property 3: len(a) == a.shape[0] len(a) == 2 == a.shape[0] len(b) == 3 == b.shape[0] len(c) == 4 == c.shape[0] len(d) == Exception == d.shape[0] # Currently the last is wrong? Property 4: size(a) == product(a.shape) size(a) == 24 == product(a.shape) size(b) == 12 == product(b.shape) size(c) == 4 == product(c.shape) size(d) == 1 == product(d.shape) # Currently the last is wrong Property 5: rank0 array behaves as mutable numbers when used as value array(2) is similar to 2 array(2.0) is similar to 2.0 array(2j) is similar to 2j # This is a summary of many concrete properties. Property 6: Indexing reduces rank. Slicing preserves rank. a[:,:,:].shape = (2, 3, 4) a[1,:,:].shape = (3, 4) a[1,1,:].shape = (4,) a[1,1,1].shape = () Property 7: Indexing by tuple of ints gives scalar. a[1,1,1] == 1 b[1,1] == 2 c[1,] == 6 d[()] == 24 # So rank0 array indexed by empty tuple should be scalar. # Currently the last is wrong Property 8: Indexing by tuple of slices gives array. a[:,:,:] == ones((2,3,4)) b[:,:] == ones((3,4)) * 2 c[:] == ones((,4)) * 6 d[()] == ones(()) * 24 # So rank0 array indexed by empty tuple should be rank0 array. # Currently the last is wrong Property 9: Indexing as lvalues a[1,1,1] = 2 b[1,1] = 2 c[1,] = 2 d[()] = 2 Property 10: Indexing and slicing as lvalues a[:,:,:] = ones((2, 3, 4)) a[1,:,:] = ones((3, 4)) a[1,1,:] = ones((4,)) a[1,1,1] = ones(()) # But the last is wrong. Conclusion 1: rank0 arrays are equivalent to scalars. See properties 7 and 8. Conclusion 2: rank0 arrays are mutable. See property 9. Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) should not be defined. See conclusion 1 and properties 1, 2, 3, 4. Conclusion 4: A missing axis is similar to having dimension 1. See property 4. Conclusion 5: rank0 int arrays should be allowed to act as indices. See property 5. Conclusion 6: rank0 arrays should not be hashable except by object id. See conclusion 2. Discussions:  These properties correspond to the current implementation quite well, except a few rough edges.  Mutable scalars are useful in their own rights.  Is there substantial difference in overhead between rank0 arrays and scalars?  How to write literal values? array(1) is too many characters.  For rank1 and rank0 arrays, Python notation distinguishes: c[1] vs c[1,] d[] vs d[()] Should these be used to handle semantic difference between indexing and slicing? Should d[] be syntactically allowed? Hope these observations help. Huaiyu 
From: Huaiyu Zhu <huaiyu_zhu@ya...>  20020917 07:04:02

Some of the choices between rank0 arrays and new scalar types might be resolved by enumerating the properties desired of them. Most properties of rank0 arrays could be fixed by consistency requirements alone, using operations that reduce array dimensions. Let a = ones((2,3,4)) b = sum(a) c = sum(b) d = sum(c) Property 1: the shape of an array is a tuple of integers a.shape == (2, 3, 4) b.shape == (3, 4) c.shape == (4,) d.shape == () Property 2: rank(a) == len(a.shape) rank(a) == 3 == len(a.shape) rank(b) == 2 == len(b.shape) rank(c) == 1 == len(c.shape) rank(d) == 0 == len(d.shape) Property 3: len(a) == a.shape[0] len(a) == 2 == a.shape[0] len(b) == 3 == b.shape[0] len(c) == 4 == c.shape[0] len(d) == Exception == d.shape[0] # Currently the last is wrong? Property 4: size(a) == product(a.shape) size(a) == 24 == product(a.shape) size(b) == 12 == product(b.shape) size(c) == 4 == product(c.shape) size(d) == 1 == product(d.shape) # Currently the last is wrong Property 5: rank0 array behaves as mutable numbers when used as value array(2) is similar to 2 array(2.0) is similar to 2.0 array(2j) is similar to 2j # This is a summary of many concrete properties. Property 6: Indexing reduces rank. Slicing preserves rank. a[:,:,:].shape = (2, 3, 4) a[1,:,:].shape = (3, 4) a[1,1,:].shape = (4,) a[1,1,1].shape = () Property 7: Indexing by tuple of ints gives scalar. a[1,1,1] == 1 b[1,1] == 2 c[1,] == 6 d[()] == 24 # So rank0 array indexed by empty tuple should be scalar. # Currently the last is wrong Property 8: Indexing by tuple of slices gives array. a[:,:,:] == ones((2,3,4)) b[:,:] == ones((3,4)) * 2 c[:] == ones((,4)) * 6 d[()] == ones(()) * 24 # So rank0 array indexed by empty tuple should be rank0 array. # Currently the last is wrong Property 9: Indexing as lvalues a[1,1,1] = 2 b[1,1] = 2 c[1,] = 2 d[()] = 2 Property 10: Indexing and slicing as lvalues a[:,:,:] = ones((2, 3, 4)) a[1,:,:] = ones((3, 4)) a[1,1,:] = ones((4,)) a[1,1,1] = ones(()) # But the last is wrong. Conclusion 1: rank0 arrays are equivalent to scalars. See properties 7 and 8. Conclusion 2: rank0 arrays are mutable. See property 9. Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) should not be defined. See conclusion 1 and properties 1, 2, 3, 4. Conclusion 4: A missing axis is similar to having dimension 1. See property 4. Conclusion 5: rank0 int arrays should be allowed to act as indices. See property 5. Conclusion 6: rank0 arrays should not be hashable except by object id. See conclusion 2. Discussions:  These properties correspond to the current implementation quite well, except a few rough edges.  Mutable scalars are useful in their own rights.  Is there substantial difference in overhead between rank0 arrays and scalars?  How to write literal values? array(1) is too many characters.  For rank1 and rank0 arrays, Python notation distinguishes: c[1] vs c[1,] d[] vs d[()] Should these be used to handle semantic difference between indexing and slicing? Should d[] be syntactically allowed? Hope these observations help. Huaiyu 
From: Huaiyu Zhu <huaiyu_zhu@ya...>  20020917 06:39:39

There is a bug in the dimensionreducing functions (sum, product, alltrue, sometrue) in numarray: The outside swapaxes should take into account that one axis is reduced. The expected behavior should be: >>> from numarray import * >>> a = ones((2,3,4)) >>> sum(a, axis=0).shape (3, 4) >>> sum(a, axis=1).shape (2, 4) >>> sum(a, axis=2).shape (2, 3) A patch is attached below. Huaiyu *** numarray.py~ Tue Jul 2 11:12:24 2002  numarray.py Mon Sep 16 23:17:27 2002 *************** *** 995,997 **** else: ! return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis)  995,997  else: ! return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis1) *************** *** 1014,1016 **** return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)), ! 0, axis)  1014,1016  return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)), ! 0, axis1) *************** *** 1034,1036 **** return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)), ! 0, axis) def sometrue(a, axis=0):  1034,1036  return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)), ! 0, axis1) def sometrue(a, axis=0): *************** *** 1043,1045 **** return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)), ! 0, axis)  1043,1045  return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)), ! 0, axis1) 
From: Pearu Peterson <pearu@ce...>  20020916 23:09:26

On Mon, 16 Sep 2002, Travis Oliphant wrote: > > Frankly, I have no idea what the implimentation details would be, but > > could we get rid of rank0 arrays altogether? I have always simply found > > them strange and confusing... What are they really neccesary for > > (besides holding scalar values of different precision that standard > > Pyton scalars)? > > With new coercion rules this becomes a possibility. Arguments against it > are that special rank0 arrays behave as more consistent numbers with the > rest of Numeric than Python scalars. In other words they have a length > and a shape and one can right Ndimensional code that works the same even > when the result is a scalar. In addition, rank0 arrays are mutable while Python scalars are not. Mutability is sometimes useful (e.g. when emulating C or Fortran calls in Python) but often also evil due to its unpythonic side effect. Pearu 
From: Travis Oliphant <oliphant@ee...>  20020916 22:50:52

> > Travis Oliphant wrote: > > > This is the gist of it. Basically you extend the Python scalars to > > include the single precision types (all the types Numeric supports). > > Would they be recognised as scalars by Python? In particular, could you > use one as an index? Personally, this is what has bit me in the past: I > could use A[3,2] as an index if A was type "Int" but not if it was > "Int16" for example. > > In any case, the type of A[3,2] should NOT depend on the precision of > the numbers stored in A. > > Frankly, I have no idea what the implimentation details would be, but > could we get rid of rank0 arrays altogether? I have always simply found > them strange and confusing... What are they really neccesary for > (besides holding scalar values of different precision that standard > Pyton scalars)? With new coercion rules this becomes a possibility. Arguments against it are that special rank0 arrays behave as more consistent numbers with the rest of Numeric than Python scalars. In other words they have a length and a shape and one can right Ndimensional code that works the same even when the result is a scalar. Another advantage of having a Numeric scalar is that we can control the behavior of floating point operations better. e.g. if only Python scalars were available and sum(a) returned 0, then 1 / sum(a) would behave as Python behaves (always raises error). while with our own scalars 1 / sum(a) could potentially behave however the user wanted. Travis 
From: Perry Greenfield <perry@st...>  20020916 21:47:54

Konrad Hinsen writes: > "Chris Barker" <Chris.Barker@...> writes: > > > use one as an index? Personally, this is what has bit me in the past: I > > At least up to Python 1.5, no, indices have to be of integer type. I > don't know if that condition was extended in later versions. > > > could use A[3,2] as an index if A was type "Int" but not if it was > > "Int16" for example. > > Ehmmm... Are you sure that is the right example? The restriction is on > the type of the index, not on the type of the array. > I think Chris is referring to the fact that Numeric returns a rank0 array for A[3,2] if A is of type Int16, and that value cannot be used as in index in Python sequences (at least for now; nothing technical prevents it from being implemented to accept object that have an __int__ method). This is the odd inconsistency Numeric has now. If A were a 1d Int16 array then A[2] would not be a rank0 array, nor is A[3,2] a rank0 array if A is of type Int32 (apparently because a Python scalar type suffices). Why it gives rank0 for 2d and not 1d I have no idea. > 
From: Tim Hochberg <tim.hochberg@ie...>  20020916 20:42:51

[Me (Tim Hochberg)] > > Can someone refresh my memory as to why some properties of > > NumArrays that are conceptually attributes are accessed as > > atributes (a.shape, a.rank), while others are accessed > > through functions (a.type, a.iscontiguous,...). [Paul F Dubois] > Because that is what Numeric did, and as explained in the position > statement on the numpy website, we are not changing things unless we > have a compelling reason. Fair enough, but what about attributes new to numarray (rank, type, maybe others). Is the preference then for function syntax over attribute syntax? I'd prefer the latter, but I'm mostly trying to figure out what the pattern is. tim 
From: Paul F Dubois <paul@pf...>  20020916 20:26:57

Because that is what Numeric did, and as explained in the position statement on the numpy website, we are not changing things unless we have a compelling reason. Numeric did some things like a.shape as properties rather than pairs of get/set functions; there was no special reason for it except a feeling that it was easier to type or looked more mathematical. E.g., x.imaginary. I think the driving reason for shape was interactive use: x.shape=(4,5) However we got here, we aren't going to debate changing any of this. The scalar issue is on the table because it causes actual trouble for real people rather than philosophical inconsistency. > > > Can someone refresh my memory as to why some properties of > NumArrays that are conceptually attributes are accessed as > atributes (a.shape, a.rank), while others are accessed > through functions (a.type, a.iscontiguous,...). > > Thanks, > > tim 
From: Tim Hochberg <tim.hochberg@ie...>  20020916 20:14:04

Can someone refresh my memory as to why some properties of NumArrays that are conceptually attributes are accessed as atributes (a.shape, a.rank), while others are accessed through functions (a.type, a.iscontiguous,...). Thanks, tim 
From: Tim Hochberg <tim.hochberg@ie...>  20020916 19:38:06

From: "Konrad Hinsen" <hinsen@...> > > Travis Oliphant wrote: > > Would they be recognised as scalars by Python? In particular, could you > > There is no "scalar" category in Python. New scalar datatypes would be > types with the same behaviour as the existing Python scalar types, but > different from the. That means that explicitly typechecking code would > not accept them, but everything else would. > > > use one as an index? Personally, this is what has bit me in the past: I > > At least up to Python 1.5, no, indices have to be of integer type. I > don't know if that condition was extended in later versions. In python 2.2 (I'm not sure about 2.0,2.1), you can subclass int and the resultant class can be used as an index. So I think this could be done. > > Frankly, I have no idea what the implimentation details would be, but > > could we get rid of rank0 arrays altogether? I have always simply found > > If we introduce additional scalars, yes. Given that numarray has changed its coercion rules, is this still necessary? From http://stsdas.stsci.edu/numarray/Userguide.html: start quote ************* Type Coercion ************* In expressions involving only arrays, the normal coercion rules apply (i.e., the same as Numeric). However, the same rules do not apply to binary operations between arrays and Python scalars in certain cases. If the kind of number is the same for the array and scalar (e.g., both are integer types or both are float types), then the type of the output is determined by the precision of the array, not the scalar. Some examples will best illustrate: Scalar type * Array type Numeric result type numarray result type Int Int16 Int32 Int16 Int Int8 Int32 Int8 Float Int8 Float64 Float64 Float Float32 Float64 Float32 The change in the rules was made so that it would be easy to preserve the precision of arrays in expressions involving scalars. Previous solutions with Numeric were either quite awkward (using a function to create a rank0 array from a scalar with the desired type) or surprising (the savespace attribute, that never allowed type coercion). The problem arises because Python has a limited selection of scalar types. This appears to be the best solution though it admittedy may surprise some who are used to the classical type coercion model. end quote > Whether or not we would want > to get rid of them is of course another question. Agreed. It's possible that rank zero arrays would be useful in certain unusual situations, although I can't think of any good examples. Regardless of that choice, if indexing never returns rank0 arrays most people will never have to deal with them. tim 