You can subscribe to this list here.
2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
From: Perry G. <pe...@st...> - 2002-09-24 21:32:31
|
Konrad Hinsen writes: > "Perry Greenfield" <pe...@st...> writes: > > > Questions: > > > > 1) given 2, is there still a desire for .reduce() to return > > rank-0 arrays (if not, we have .areduce() which is intented to return > > arrays always). > > > > 2) whichever is the "returns arrays always" reduce method, should the > > endpoint be rank-0 arrays or rank-1 len-1 arrays? > > I don't really see an application where a reduction operation yielding > rank-1 or higher arrays would be useful. It would be a special case, > not useful for generic programming. So my answer to 2) is rank-0. > What I am wondering is what behavior suits "generic" programming more. Eric Jones and Paul Dubois have given examples where having to deal with entities that may be scalars or arrays is a pain. But, having said that, the behavior that Eric wanted was not consistent with what many thought rank-0 arrays should have (i.e., len(rank-0)=1, rank-0[0] = value). The proposal to generate rank-1 len-1 arrays was made since len() of these arrays is = 1, and indexing with [0] does work. So for the kinds of examples he gave, rank-1 len-1 arrays appear to allow for more generic code. But I'm not trying to speak for everyone; that's why I'm asking for opinions. Do you have examples where you find rank-0 arrays make for more generic code in your cases when len() and indexing on these does not have the behavior Eric wanted? May I see a couple? > As for 1), if indexing doesn't return rank-0 arrays, then standard > reduction shouldn't either. We would then have a system in which > rank-0 arrays are "expert only" stuff, most users would never see > them, and they could safely be ignored in tutorials. > That's my inclination, but I think that the question of whether there should be some reduce mechanism that returns arrays always is still a valid one. I can see that there are good uses for that (or at least a function to cast a scalar to a rank-1 len-1 array if it isn't already an array, like what array() does except that it now generates rank-0 arrays). Perry |
From: Konrad H. <hi...@cn...> - 2002-09-24 21:18:37
|
"Perry Greenfield" <pe...@st...> writes: > Questions: > > 1) given 2, is there still a desire for .reduce() to return > rank-0 arrays (if not, we have .areduce() which is intented to return > arrays always). > > 2) whichever is the "returns arrays always" reduce method, should the > endpoint be rank-0 arrays or rank-1 len-1 arrays? I don't really see an application where a reduction operation yielding rank-1 or higher arrays would be useful. It would be a special case, not useful for generic programming. So my answer to 2) is rank-0. As for 1), if indexing doesn't return rank-0 arrays, then standard reduction shouldn't either. We would then have a system in which rank-0 arrays are "expert only" stuff, most users would never see them, and they could safely be ignored in tutorials. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hi...@cn... Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- |
From: Perry G. <pe...@st...> - 2002-09-24 20:39:51
|
[I posted this almost a week ago, but apparently an email problem prevented it from actually getting posted!] I think there has been sufficient discussion about rank-0 arrays to make some decisions about how numarray will handle them. [If you don't want to wade through the rationale, jump to the end where there is a short summary of what we plan to do and what we have questions about] ******************************************************************** First I'd like to take a stab at summarizing the case made for rank-0 arrays in general and adding some of my own comments regarding these points. 1) rank-0 arrays are a useful mechanism to avoid having binary operations with scalars cause unintended promotion of other arrays to larger numeric types (e.g. 2*arange(10, typecode=Int16) results in an Int32 result). *** For numarray this is a non-issue because the coercion rules prevent scalars from increasing the type of an array if the scalar is the same kind of number (e.g., Int, Float, Complex) as the array. 2) rank-0 arrays preserve the type information instead of converting scalars to Python scalars. *** This seems of limited value. With only a couple possible exceptions in the future (and none now), Python scalars are effectively the largest type available so no information is lost. One can convert to and from Python scalars and not lose any information. The possible future exceptions are long doubles and UInt32 (supported in Numeric, but not numarray yet--but frankly, I'm not yet sure how important UInt32 is at the moment). It is possible that Python scalars may move up in size so this may or may not become an issue in the future. By itself, it does not appear to be a compelling reason. 3) rank-0 arrays allow controlling exceptions (e.g. divide by zero) in a way different from how Python handles them (exception always) *** This is a valid concern...maybe. I was more impressed by it initially, but it occurred to me that most expressions that involve a scalar exception (NaN, divide-by-zero, overflow, etc.) generally corrupt everything, unlike exceptions with arrays where only a few values may be tainted. Unless one is only interested in ignoring some scalar results in a scalar expression as part of a larger computation, it seems of very limited use to ignore, or warn on scalar exceptions. In any event, this is really of no relevance to the use of rank-0 for indexed results or reduction operations. 4) Using rank-0 arrays in place of scalars would promote more generic programming. This was really the last point of real contention as far as I was concerned. In the end, it really came down to seeing good examples of how lacking this caused code to be much worse than it could be with rank-0 arrays. There really are two cases being discussed: whether indexing a single item ("complete dereferencing" in Scott Gilbert's description) returns rank-0, and whether reduction operations return rank-0. *** indexing returning rank-0. Amazingly enough no one was able to provide even one real code example of where rank-0 returns from indexing was a problem (this includes MA as far as I can tell). In the end, this has been much ado about nothing. Henceforth, numarray will return Python scalars when arrays are indexed (as it does currently). *** reduction operations. There are good examples of where reduction operations returning rank-0 are made simpler. However, the situation is muddied quite a bit by other issues which I will discuss below. This is an area that deserves a bit more discussion in general. But before I tackle that, there is a point about rank-0 arrays that needs to be made which I think is in some respects is an obvious point, but somehow got lost in much of the discussions. Even if it were true that rank-0 arrays made for much simpler, generic code, they are far less useful than might appear in simplifying code. Why? Because even if array operations (whether indexing, reduction or other functions) were entirely consistent about never returning scalars, it is a general fact that most Numeric/numarray code must be prepared to handle Python scalars thrown at it in place of arrays by the user. Since Python scalars can come leaking into your code at many points, consistency in Numeric/numarray in avoiding Python scalars really doesn't solve the issue. I would hazard a guess that the great majority of the conditional code that exist today would not be eliminated because of this (e.g., this appears to be the case for MA) Reduction operations: There is a good case to be made that reduction operations should result in rank-0 arrays rather than scalars (after all, they are reducing dimensions), but not everyone agrees that is what should be done. But before deciding what is to be done there, some problems with rank-0 arrays should be discussed. I think Konrad and Huaiyu have made very powerful arguments about how certain operations like indexing, attributes and such should or shouldn't work. In particular, some current Numeric behaviors should be changed. Indexing by 0 should not work (instead Scott Gilbert's suggestion of indexing with an empty tuple sounds right, if a bit syntatically clumsy due to Python not accepting an empty index). len should not return 1, etc. So even if we return rank-0 values in reduction operations, this appears to still cause problems with some of the examples given by Eric that depend on len(rank-0) = 1. What should be done about that? One possibility is to use different numarray functions designed to help write generic code (e.g., an alternate function to len). But there is one aspect to this that ought to be pointed out. Some have asked for rank-0 arrays that can be indexed with 0 and produce len of 1. There is such an object that does this and it is a rank-1 len-1 array. One alternative is to have reduction operations have as their endpoint a rank-1 len-1 array rather than a rank-0 array. The rank-0 endpoint is more justified conceptually, but apparently less practical. If a reduction on a 1-d arrray always produced a 1-d array, then one can always be guaranteed that it can be indexed, and that len works on it. The drawback is that it can never be used as a scalar directly as rank-0 arrays could be. I think this is a case where you can't have it both ways. If you want a scalar-like object, then some operations that work on higher rank arrays won't work on it (or shouldn't). If you want something where these operations do work, don't expect to use it where a scalar is expected unless you index it. Is there any interest in this alternate approach to reductions? We plan to have two reduction methods available, one that results in scalars, and one in arrays. The main question is which one .reduce maps to, and what the endpoint is for the method that always returns arrays is. *********************************************************************** SUMMARY *********************************************************************** 1) Indexing returns Python scalars in numarray. No rank-0 arrays are ever returned from indexing. 2) rank-0 arrays will be supported, but len(), and indexing will not work as they do in Numeric. In particular, to get a scalar, one will have to index with an empty tuple (e.g., x[()], x[0] will raise an exception), len() will return None. Questions: 1) given 2, is there still a desire for .reduce() to return rank-0 arrays (if not, we have .areduce() which is intented to return arrays always). 2) whichever is the "returns arrays always" reduce method, should the endpoint be rank-0 arrays or rank-1 len-1 arrays? |
From: Rob <ro...@py...> - 2002-09-21 00:40:14
|
Konrad Hinsen wrote: > <snip> What I like about Numeric stems from what I like about Python- the language does not get in my way of creating algorithms. I do not have to constantly refer to some reference book while programming. The idea of the Numeric Python EM Project (URL below) was to put EM code into a form where it could be more easily understood. Python and Numeric Python were the perfect choice for that task. As for Mathematica and other commercial software, I have no use for them. Python is free. Sincerely, Rob. -- ----------------------------- The Numeric Python EM Project www.pythonemproject.com |
From: Travis N. V. <tr...@en...> - 2002-09-20 17:54:24
|
Thought I'd reply to all since I'm including some links that are possibly interesting to the list: > -----Original Message----- > From: num...@li... > [mailto:num...@li...]On Behalf Of Konrad > Hinsen > Sent: Friday, September 20, 2002 12:04 PM > To: num...@li... > Subject: [Numpy-discussion] A message from Cameron Laird > > > I send this on behalf of Cameron Laird <cl...@ph...>. > Please reply to him, not to me. > > I have at least a couple of assignments from magazines such > as IBM's developerWorks to report on matters that involve > Numeric. I'd welcome contact from anyone here who wants to > publicize his or her work with Python and Numeric. > The SciPy Toolkit (http://www.scipy.org) attacks Matlab's functionality head-on--providing much of the functional interface symantics that Matlab provides and much much more. It is a stated goal of many in the SciPy development community to eliminate the need for Matlab altogether by providing a tool that offers _all_ the functionality and better performance (among many other things) in an open-source, truly object-oriented package. > I have a particular interest in advantages Python and Numeric > enjoy over such alternatives as Mathematica, IDL, SAS/IML, > MATLAB, and so on, all of which are more narrowly targeted at > the kinds of scientific and engineering problems tackled by > contributors to this mailing list. What does Python do for > you that the commercial products don't? > > I suspect that many of you will mention, in one form or > another, Python's aptness for programming "in the large". > Do you have specific examples of how this is clumsy in > MATLAB, Mathematica, and so on? > > Have you tried to interface MATLAB and so on to hardware > instrumentation or other external data sources? > The recent SciPy '02 workshop (http://www.scipy.org/site_content/scipy02) had some presented material that tangentially addressed interfacing Matlab and Mathematica (perhaps even in a bof, I've slept since then). You might try the scipy-user mailing list as well. (Presentations: http://www.scipy.org/site_content/scipy02/presentations , Mailing list: http://www.scipy.org/site_content/MailList) > How do the scientists and engineers (as opposed to the > "informaticians" or software developers) on your teams > accept Python, compared to IDL and friends? Do scientists > at your site program? > > Is there anything Python's missing in its competition with > MATLAB and so on? > The aforementioned workshop included a survey that addressed these issues as well. You can look at the aggregated results by following the link below (it's a little raw in its layout, let me know if there is a question about how to interpret some things). http://www.scipy.org/site_content/scipy02/survey_results.htm (note: some questions were 'pick one' and others were 'pick all that apply'--the total number of surveys turned in was 36, I think.) > > Cameron Laird <Ca...@La...> +1 281 996 8546 FAX > http://phaseit.net/claird/misc.writing/publications.html > |
From: Konrad H. <hi...@cn...> - 2002-09-20 17:04:58
|
I send this on behalf of Cameron Laird <cl...@ph...>. Please reply to him, not to me. I have at least a couple of assignments from magazines such as IBM's developerWorks to report on matters that involve Numeric. I'd welcome contact from anyone here who wants to publicize his or her work with Python and Numeric. I have a particular interest in advantages Python and Numeric enjoy over such alternatives as Mathematica, IDL, SAS/IML, MATLAB, and so on, all of which are more narrowly targeted at the kinds of scientific and engineering problems tackled by contributors to this mailing list. What does Python do for you that the commercial products don't? I suspect that many of you will mention, in one form or another, Python's aptness for programming "in the large". Do you have specific examples of how this is clumsy in MATLAB, Mathematica, and so on? Have you tried to interface MATLAB and so on to hardware instrumentation or other external data sources? How do the scientists and engineers (as opposed to the "informaticians" or software developers) on your teams accept Python, compared to IDL and friends? Do scientists at your site program? Is there anything Python's missing in its competition with MATLAB and so on? Cameron Laird <Ca...@La...> +1 281 996 8546 FAX http://phaseit.net/claird/misc.writing/publications.html |
From: Todd M. <jm...@st...> - 2002-09-19 16:48:19
|
Aureli Soria Frisch wrote: > Hi, > > > I have developed till now all my numerical stuff using Numeric objects. > > > I want to use some modules using numarray. I have tested some > conversion functions on Python, e.g. > > > numarray.array(Numeric_object.tolist()) > Much faster, but less general, is: numarray.fromstring(Numeric_object.tostring(), type=XXXX) > > but it seems to me a bit slow (specially for large arrays as images). > Is there any more ways of doing this conversion? > > Should I write my own conversion in C? > It depends on how soon you need it to be fast. numarray fromlist/tolist should be implemented in C for the next release, which should occur within a couple months, perhaps sooner. If you want to write a C extension for numarray now, the best way to do it is to use the Numeric compatability layer, which essentially uses a subset of the Numeric C-API, and is not likely to change. > Which advices could you give me in order to make an smoother transition? > > > Furthermore I found the web site about numarray/Numeric > incompatibilities very interesting. However I did not understand quite > well the paragraph: > I think that webpage is out of date. > > since interactively typing a.shape() or a.shape((8,8)) raise an > exception (Tuple object not callable). Am I misunderstanding something? > Actually, for python 2.2 and up, numarray has all of those attributes in a Numeric compatible, Python properties based form. So ".shape" get/sets the shape in numarray, just as in Numeric: >>> import numarray >>> a = numarray.arange(10) >>> a.shape (10,) >>> a.shape = (2,5) >>> a.shape (2,5) Because of this, the expression a.shape() that you mentioned tries to "call" the shape tuple returned by .shape, resulting in an exception. Todd -- Todd Miller jm...@st... STSCI / SSB |
From: Aureli S. F. <au...@ip...> - 2002-09-19 16:17:52
|
Hi, I have developed till now all my numerical stuff using Numeric objects. I want to use some modules using numarray. I have tested some conversion functions on Python, e.g. numarray.array(Numeric_object.tolist()) but it seems to me a bit slow (specially for large arrays as images). Is there any more ways of doing this conversion? Should I write my own conversion in C? Which advices could you give me in order to make an smoother transition? Furthermore I found the web site about numarray/Numeric incompatibilities very interesting. However I did not understand quite well the paragraph: >Numeric arrays have some public attributes. Numarray arrays >have none. All changes to an array's state must be made >through accessor methods. Specifically, Numeric has the following >attributes: > >Numeric numarray accessor method(s) >Attribute > >shape --> shape() (i.e., specify new shape through method arg instead >of > assigning to shape attribute) >flat --> flat() >real --> real() (set capability not implemented yet, but will be) >imag --> imag() (ditto) >savespace --> not used, no equivalent functionality (xxx check this) since interactively typing a.shape() or a.shape((8,8)) raise an exception (Tuple object not callable). Am I misunderstanding something? Thanks in advance. Best regards, Aureli -- ################################# Aureli Soria Frisch Fraunhofer IPK Dept. Pattern Recognition post: Pascalstr. 8-9, 10587 Berlin, Germany e-mail: au...@ip... fon: +49 30 39006-143 fax: +49 30 3917517 web: http://vision.fhg.de/~aureli/web-aureli_en.html ################################# |
From: Ralf J. <jue...@in...> - 2002-09-18 08:15:11
|
On Mon, 2002-09-16 at 22:11, Tim Hochberg wrote: >=20 > Can someone refresh my memory as to why some properties of NumArrays that > are conceptually attributes are accessed as atributes (a.shape, a.rank), > while others are accessed through functions (a.type, a.iscontiguous,...). Yes, I'd like to understand as well; there is even a third. There are attributes, e.g. a.shape vs methods, e.g. a.type() vs functions e.g. len(a). Regards, Ralf >=20 > Thanks, >=20 > -tim >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion --=20 ------------------------------------------------------------------------- Ralf J=FCngling Institut f=FCr Informatik - Lehrstuhl f. Mustererkennung & Bildverarbeitung Georges-K=F6hler-Allee Geb=E4ude 52 Tel: +49-(0)761-203-8215 79110 Freiburg Fax: +49-(0)761-203-8262 ------------------------------------------------------------------------- |
From: Huaiyu Z. <hua...@ya...> - 2002-09-18 07:02:29
|
On Tue, 17 Sep 2002, Travis Oliphant wrote: > > len(d) == Exception == d.shape[0] > > > > # Currently the last is wrong? > > Agreed, but this is because d is an integer and out of Numerics control. > This is a case for returning 0d arrays rather than Python scalars. That is one problem. It can be removed by using shape(d). More fundamentally, though, len(d) == shape(d)[0] == ()[0] => IndexError. I think Konrad made this point a few days back. > > size(d) == 1 == product(d.shape) > > > > # Currently the last is wrong > > I disagree that this is wrong. This works as described for me. Right. Change d.shape to shape(d) here (and in several other places). > Why is this? I thought you argued the other way for len(scalar). Of > course, one solution is that we could overwrite the len() function and > allow it to work for scalars. Raising exception is the correct behavior, not a problem to be solved. > > > > Conclusion 5: rank-0 int arrays should be allowed to act as indices. > > See property 5. > > Can't do this for lists and other builtin sequences. If numarray defines a consistent set of behaviors for integer types that is intuitively understandable, it might not be difficult to persuade core Python to check against an abstract integer type. > > - Is there substantial difference in overhead between rank-0 arrays and > > scalars? > > Yes. That would be one major problem. However, after giving this some more thoughts, I'm starting to doubt the analogy I made. The problem is that in the end there is still a need to index an array and obtain a good old immutable scalar. So - What notation should be used for this purpose? We can use c[0] to get immutable scalars and c[0,] for rank-0 arrays / mutable scalars. But what about other ranks? Python does not allow distinctions based on a[1,1,1] versus a[(1,1,1)] or d[] versus d[()]. - This weakens the argument that rank-0 arrays are scalars, since that argument is essentially based on sum(c) and c[0] being of the same type. Huaiyu |
From: Travis O. <oli...@ee...> - 2002-09-17 16:09:33
|
> > Some of the choices between rank-0 arrays and new scalar types > might be resolved by enumerating the properties desired of them. > These are helpful observations. > Most properties of rank-0 arrays could be fixed by consistency > requirements alone, using operations that reduce array dimensions. > > Let a = ones((2,3,4)) > b = sum(a) > c = sum(b) > d = sum(c) > > Property 1: the shape of an array is a tuple of integers > a.shape == (2, 3, 4) > b.shape == (3, 4) > c.shape == (4,) > d.shape == () > > Property 2: rank(a) == len(a.shape) > rank(a) == 3 == len(a.shape) > rank(b) == 2 == len(b.shape) > rank(c) == 1 == len(c.shape) > rank(d) == 0 == len(d.shape) > > Property 3: len(a) == a.shape[0] > len(a) == 2 == a.shape[0] > len(b) == 3 == b.shape[0] > len(c) == 4 == c.shape[0] > len(d) == Exception == d.shape[0] > > # Currently the last is wrong? Agreed, but this is because d is an integer and out of Numerics control. This is a case for returning 0d arrays rather than Python scalars. > > Property 4: size(a) == product(a.shape) > size(a) == 24 == product(a.shape) > size(b) == 12 == product(b.shape) > size(c) == 4 == product(c.shape) > size(d) == 1 == product(d.shape) > > # Currently the last is wrong I disagree that this is wrong. This works as described for me. > > Property 5: rank-0 array behaves as mutable numbers when used as value > array(2) is similar to 2 > array(2.0) is similar to 2.0 > array(2j) is similar to 2j > > # This is a summary of many concrete properties. > > Property 6: Indexing reduces rank. Slicing preserves rank. > a[:,:,:].shape = (2, 3, 4) > a[1,:,:].shape = (3, 4) > a[1,1,:].shape = (4,) > a[1,1,1].shape = () > > Property 7: Indexing by tuple of ints gives scalar. > a[1,1,1] == 1 > b[1,1] == 2 > c[1,] == 6 > d[()] == 24 > > # So rank-0 array indexed by empty tuple should be scalar. > # Currently the last is wrong Not sure about this property, but interesting. > > Property 8: Indexing by tuple of slices gives array. > a[:,:,:] == ones((2,3,4)) > b[:,:] == ones((3,4)) * 2 > c[:] == ones((,4)) * 6 > d[()] == ones(()) * 24 > > # So rank-0 array indexed by empty tuple should be rank-0 array. > # Currently the last is wrong Not sure about this one either. > > Property 9: Indexing as lvalues > a[1,1,1] = 2 > b[1,1] = 2 > c[1,] = 2 > d[()] = 2 > > Property 10: Indexing and slicing as lvalues > a[:,:,:] = ones((2, 3, 4)) > a[1,:,:] = ones((3, 4)) > a[1,1,:] = ones((4,)) > a[1,1,1] = ones(()) > > # But the last is wrong. > > > Conclusion 1: rank-0 arrays are equivalent to scalars. > See properties 7 and 8. > > Conclusion 2: rank-0 arrays are mutable. > See property 9. > > Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) > should not be defined. Why is this? I thought you argued the other way for len(scalar). Of course, one solution is that we could overwrite the len() function and allow it to work for scalars. > > See conclusion 1 and properties 1, 2, 3, 4. > > Conclusion 4: A missing axis is similar to having dimension 1. > See property 4. > > Conclusion 5: rank-0 int arrays should be allowed to act as indices. > See property 5. Can't do this for lists and other builtin sequences. > > Conclusion 6: rank-0 arrays should not be hashable except by object id. > See conclusion 2. > > > Discussions: > > - These properties correspond to the current implementation quite well, > except a few rough edges. > > - Mutable scalars are useful in their own rights. > > - Is there substantial difference in overhead between rank-0 arrays and > scalars? Yes. > > - How to write literal values? array(1) is too many characters. > > - For rank-1 and rank-0 arrays, Python notation distinguishes: > > c[1] vs c[1,] > d[] vs d[()] > > Should these be used to handle semantic difference between indexing > and slicing? Should d[] be syntactically allowed? > > Hope these observations help. Thanks for the observations. -Travis O. |
From: Huaiyu Z. <hua...@ya...> - 2002-09-17 07:11:41
|
Oops, the subject line was somehow cut off. Please use this one if you follow up. - Huaiyu On Tue, 17 Sep 2002, Huaiyu Zhu wrote: Some of the choices between rank-0 arrays and new scalar types might be resolved by enumerating the properties desired of them. Most properties of rank-0 arrays could be fixed by consistency requirements alone, using operations that reduce array dimensions. Let a = ones((2,3,4)) b = sum(a) c = sum(b) d = sum(c) Property 1: the shape of an array is a tuple of integers a.shape == (2, 3, 4) b.shape == (3, 4) c.shape == (4,) d.shape == () Property 2: rank(a) == len(a.shape) rank(a) == 3 == len(a.shape) rank(b) == 2 == len(b.shape) rank(c) == 1 == len(c.shape) rank(d) == 0 == len(d.shape) Property 3: len(a) == a.shape[0] len(a) == 2 == a.shape[0] len(b) == 3 == b.shape[0] len(c) == 4 == c.shape[0] len(d) == Exception == d.shape[0] # Currently the last is wrong? Property 4: size(a) == product(a.shape) size(a) == 24 == product(a.shape) size(b) == 12 == product(b.shape) size(c) == 4 == product(c.shape) size(d) == 1 == product(d.shape) # Currently the last is wrong Property 5: rank-0 array behaves as mutable numbers when used as value array(2) is similar to 2 array(2.0) is similar to 2.0 array(2j) is similar to 2j # This is a summary of many concrete properties. Property 6: Indexing reduces rank. Slicing preserves rank. a[:,:,:].shape = (2, 3, 4) a[1,:,:].shape = (3, 4) a[1,1,:].shape = (4,) a[1,1,1].shape = () Property 7: Indexing by tuple of ints gives scalar. a[1,1,1] == 1 b[1,1] == 2 c[1,] == 6 d[()] == 24 # So rank-0 array indexed by empty tuple should be scalar. # Currently the last is wrong Property 8: Indexing by tuple of slices gives array. a[:,:,:] == ones((2,3,4)) b[:,:] == ones((3,4)) * 2 c[:] == ones((,4)) * 6 d[()] == ones(()) * 24 # So rank-0 array indexed by empty tuple should be rank-0 array. # Currently the last is wrong Property 9: Indexing as lvalues a[1,1,1] = 2 b[1,1] = 2 c[1,] = 2 d[()] = 2 Property 10: Indexing and slicing as lvalues a[:,:,:] = ones((2, 3, 4)) a[1,:,:] = ones((3, 4)) a[1,1,:] = ones((4,)) a[1,1,1] = ones(()) # But the last is wrong. Conclusion 1: rank-0 arrays are equivalent to scalars. See properties 7 and 8. Conclusion 2: rank-0 arrays are mutable. See property 9. Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) should not be defined. See conclusion 1 and properties 1, 2, 3, 4. Conclusion 4: A missing axis is similar to having dimension 1. See property 4. Conclusion 5: rank-0 int arrays should be allowed to act as indices. See property 5. Conclusion 6: rank-0 arrays should not be hashable except by object id. See conclusion 2. Discussions: - These properties correspond to the current implementation quite well, except a few rough edges. - Mutable scalars are useful in their own rights. - Is there substantial difference in overhead between rank-0 arrays and scalars? - How to write literal values? array(1) is too many characters. - For rank-1 and rank-0 arrays, Python notation distinguishes: c[1] vs c[1,] d[] vs d[()] Should these be used to handle semantic difference between indexing and slicing? Should d[] be syntactically allowed? Hope these observations help. Huaiyu |
From: Huaiyu Z. <hua...@ya...> - 2002-09-17 07:04:02
|
Some of the choices between rank-0 arrays and new scalar types might be resolved by enumerating the properties desired of them. Most properties of rank-0 arrays could be fixed by consistency requirements alone, using operations that reduce array dimensions. Let a = ones((2,3,4)) b = sum(a) c = sum(b) d = sum(c) Property 1: the shape of an array is a tuple of integers a.shape == (2, 3, 4) b.shape == (3, 4) c.shape == (4,) d.shape == () Property 2: rank(a) == len(a.shape) rank(a) == 3 == len(a.shape) rank(b) == 2 == len(b.shape) rank(c) == 1 == len(c.shape) rank(d) == 0 == len(d.shape) Property 3: len(a) == a.shape[0] len(a) == 2 == a.shape[0] len(b) == 3 == b.shape[0] len(c) == 4 == c.shape[0] len(d) == Exception == d.shape[0] # Currently the last is wrong? Property 4: size(a) == product(a.shape) size(a) == 24 == product(a.shape) size(b) == 12 == product(b.shape) size(c) == 4 == product(c.shape) size(d) == 1 == product(d.shape) # Currently the last is wrong Property 5: rank-0 array behaves as mutable numbers when used as value array(2) is similar to 2 array(2.0) is similar to 2.0 array(2j) is similar to 2j # This is a summary of many concrete properties. Property 6: Indexing reduces rank. Slicing preserves rank. a[:,:,:].shape = (2, 3, 4) a[1,:,:].shape = (3, 4) a[1,1,:].shape = (4,) a[1,1,1].shape = () Property 7: Indexing by tuple of ints gives scalar. a[1,1,1] == 1 b[1,1] == 2 c[1,] == 6 d[()] == 24 # So rank-0 array indexed by empty tuple should be scalar. # Currently the last is wrong Property 8: Indexing by tuple of slices gives array. a[:,:,:] == ones((2,3,4)) b[:,:] == ones((3,4)) * 2 c[:] == ones((,4)) * 6 d[()] == ones(()) * 24 # So rank-0 array indexed by empty tuple should be rank-0 array. # Currently the last is wrong Property 9: Indexing as lvalues a[1,1,1] = 2 b[1,1] = 2 c[1,] = 2 d[()] = 2 Property 10: Indexing and slicing as lvalues a[:,:,:] = ones((2, 3, 4)) a[1,:,:] = ones((3, 4)) a[1,1,:] = ones((4,)) a[1,1,1] = ones(()) # But the last is wrong. Conclusion 1: rank-0 arrays are equivalent to scalars. See properties 7 and 8. Conclusion 2: rank-0 arrays are mutable. See property 9. Conclusion 3: shape(scalar), size(scalar) are all defined, but len(scalar) should not be defined. See conclusion 1 and properties 1, 2, 3, 4. Conclusion 4: A missing axis is similar to having dimension 1. See property 4. Conclusion 5: rank-0 int arrays should be allowed to act as indices. See property 5. Conclusion 6: rank-0 arrays should not be hashable except by object id. See conclusion 2. Discussions: - These properties correspond to the current implementation quite well, except a few rough edges. - Mutable scalars are useful in their own rights. - Is there substantial difference in overhead between rank-0 arrays and scalars? - How to write literal values? array(1) is too many characters. - For rank-1 and rank-0 arrays, Python notation distinguishes: c[1] vs c[1,] d[] vs d[()] Should these be used to handle semantic difference between indexing and slicing? Should d[] be syntactically allowed? Hope these observations help. Huaiyu |
From: Huaiyu Z. <hua...@ya...> - 2002-09-17 06:39:39
|
There is a bug in the dimension-reducing functions (sum, product, alltrue, sometrue) in numarray: The outside swapaxes should take into account that one axis is reduced. The expected behavior should be: >>> from numarray import * >>> a = ones((2,3,4)) >>> sum(a, axis=0).shape (3, 4) >>> sum(a, axis=1).shape (2, 4) >>> sum(a, axis=2).shape (2, 3) A patch is attached below. Huaiyu *** numarray.py~ Tue Jul 2 11:12:24 2002 --- numarray.py Mon Sep 16 23:17:27 2002 *************** *** 995,997 **** else: ! return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis) --- 995,997 ---- else: ! return swapaxes(ufunc.add.reduce(swapaxes(a, 0, axis)), 0, axis-1) *************** *** 1014,1016 **** return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)), ! 0, axis) --- 1014,1016 ---- return swapaxes(ufunc.multiply.reduce(swapaxes(a, 0, axis)), ! 0, axis-1) *************** *** 1034,1036 **** return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)), ! 0, axis) def sometrue(a, axis=0): --- 1034,1036 ---- return swapaxes(ufunc.logical_and.reduce(swapaxes(a, 0, axis)), ! 0, axis-1) def sometrue(a, axis=0): *************** *** 1043,1045 **** return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)), ! 0, axis) --- 1043,1045 ---- return swapaxes(ufunc.logical_or.reduce(swapaxes(a, 0, axis)), ! 0, axis-1) |
From: Pearu P. <pe...@ce...> - 2002-09-16 23:09:26
|
On Mon, 16 Sep 2002, Travis Oliphant wrote: > > Frankly, I have no idea what the implimentation details would be, but > > could we get rid of rank-0 arrays altogether? I have always simply found > > them strange and confusing... What are they really neccesary for > > (besides holding scalar values of different precision that standard > > Pyton scalars)? > > With new coercion rules this becomes a possibility. Arguments against it > are that special rank-0 arrays behave as more consistent numbers with the > rest of Numeric than Python scalars. In other words they have a length > and a shape and one can right N-dimensional code that works the same even > when the result is a scalar. In addition, rank-0 arrays are mutable while Python scalars are not. Mutability is sometimes useful (e.g. when emulating C or Fortran calls in Python) but often also evil due to its unpythonic side effect. Pearu |
From: Travis O. <oli...@ee...> - 2002-09-16 22:50:52
|
> > Travis Oliphant wrote: > > > This is the gist of it. Basically you extend the Python scalars to > > include the single precision types (all the types Numeric supports). > > Would they be recognised as scalars by Python? In particular, could you > use one as an index? Personally, this is what has bit me in the past: I > could use A[3,2] as an index if A was type "Int" but not if it was > "Int16" for example. > > In any case, the type of A[3,2] should NOT depend on the precision of > the numbers stored in A. > > Frankly, I have no idea what the implimentation details would be, but > could we get rid of rank-0 arrays altogether? I have always simply found > them strange and confusing... What are they really neccesary for > (besides holding scalar values of different precision that standard > Pyton scalars)? With new coercion rules this becomes a possibility. Arguments against it are that special rank-0 arrays behave as more consistent numbers with the rest of Numeric than Python scalars. In other words they have a length and a shape and one can right N-dimensional code that works the same even when the result is a scalar. Another advantage of having a Numeric scalar is that we can control the behavior of floating point operations better. e.g. if only Python scalars were available and sum(a) returned 0, then 1 / sum(a) would behave as Python behaves (always raises error). while with our own scalars 1 / sum(a) could potentially behave however the user wanted. -Travis |
From: Perry G. <pe...@st...> - 2002-09-16 21:47:54
|
Konrad Hinsen writes: > "Chris Barker" <Chr...@no...> writes: > > > use one as an index? Personally, this is what has bit me in the past: I > > At least up to Python 1.5, no, indices have to be of integer type. I > don't know if that condition was extended in later versions. > > > could use A[3,2] as an index if A was type "Int" but not if it was > > "Int16" for example. > > Ehmmm... Are you sure that is the right example? The restriction is on > the type of the index, not on the type of the array. > I think Chris is referring to the fact that Numeric returns a rank-0 array for A[3,2] if A is of type Int16, and that value cannot be used as in index in Python sequences (at least for now; nothing technical prevents it from being implemented to accept object that have an __int__ method). This is the odd inconsistency Numeric has now. If A were a 1-d Int16 array then A[2] would not be a rank-0 array, nor is A[3,2] a rank-0 array if A is of type Int32 (apparently because a Python scalar type suffices). Why it gives rank-0 for 2-d and not 1-d I have no idea. > |
From: Tim H. <tim...@ie...> - 2002-09-16 20:42:51
|
[Me (Tim Hochberg)] > > Can someone refresh my memory as to why some properties of > > NumArrays that are conceptually attributes are accessed as > > atributes (a.shape, a.rank), while others are accessed > > through functions (a.type, a.iscontiguous,...). [Paul F Dubois] > Because that is what Numeric did, and as explained in the position > statement on the numpy website, we are not changing things unless we > have a compelling reason. Fair enough, but what about attributes new to numarray (rank, type, maybe others). Is the preference then for function syntax over attribute syntax? I'd prefer the latter, but I'm mostly trying to figure out what the pattern is. -tim |
From: Paul F D. <pa...@pf...> - 2002-09-16 20:26:57
|
Because that is what Numeric did, and as explained in the position statement on the numpy website, we are not changing things unless we have a compelling reason. Numeric did some things like a.shape as properties rather than pairs of get/set functions; there was no special reason for it except a feeling that it was easier to type or looked more mathematical. E.g., x.imaginary. I think the driving reason for shape was interactive use: x.shape=(4,5) However we got here, we aren't going to debate changing any of this. The scalar issue is on the table because it causes actual trouble for real people rather than philosophical inconsistency. > > > Can someone refresh my memory as to why some properties of > NumArrays that are conceptually attributes are accessed as > atributes (a.shape, a.rank), while others are accessed > through functions (a.type, a.iscontiguous,...). > > Thanks, > > -tim |
From: Tim H. <tim...@ie...> - 2002-09-16 20:14:04
|
Can someone refresh my memory as to why some properties of NumArrays that are conceptually attributes are accessed as atributes (a.shape, a.rank), while others are accessed through functions (a.type, a.iscontiguous,...). Thanks, -tim |
From: Tim H. <tim...@ie...> - 2002-09-16 19:38:06
|
From: "Konrad Hinsen" <hi...@cn...> > > Travis Oliphant wrote: > > Would they be recognised as scalars by Python? In particular, could you > > There is no "scalar" category in Python. New scalar datatypes would be > types with the same behaviour as the existing Python scalar types, but > different from the. That means that explicitly type-checking code would > not accept them, but everything else would. > > > use one as an index? Personally, this is what has bit me in the past: I > > At least up to Python 1.5, no, indices have to be of integer type. I > don't know if that condition was extended in later versions. In python 2.2 (I'm not sure about 2.0,2.1), you can subclass int and the resultant class can be used as an index. So I think this could be done. > > Frankly, I have no idea what the implimentation details would be, but > > could we get rid of rank-0 arrays altogether? I have always simply found > > If we introduce additional scalars, yes. Given that numarray has changed its coercion rules, is this still necessary? From http://stsdas.stsci.edu/numarray/Userguide.html: ----------start quote---------- ************* Type Coercion ************* In expressions involving only arrays, the normal coercion rules apply (i.e., the same as Numeric). However, the same rules do not apply to binary operations between arrays and Python scalars in certain cases. If the kind of number is the same for the array and scalar (e.g., both are integer types or both are float types), then the type of the output is determined by the precision of the array, not the scalar. Some examples will best illustrate: Scalar type * Array type Numeric result type numarray result type Int Int16 Int32 Int16 Int Int8 Int32 Int8 Float Int8 Float64 Float64 Float Float32 Float64 Float32 The change in the rules was made so that it would be easy to preserve the precision of arrays in expressions involving scalars. Previous solutions with Numeric were either quite awkward (using a function to create a rank-0 array from a scalar with the desired type) or surprising (the savespace attribute, that never allowed type coercion). The problem arises because Python has a limited selection of scalar types. This appears to be the best solution though it admittedy may surprise some who are used to the classical type coercion model. ----------end quote---------- > Whether or not we would want > to get rid of them is of course another question. Agreed. It's possible that rank zero arrays would be useful in certain unusual situations, although I can't think of any good examples. Regardless of that choice, if indexing never returns rank-0 arrays most people will never have to deal with them. -tim |
From: Konrad H. <hi...@cn...> - 2002-09-16 19:17:26
|
"Chris Barker" <Chr...@no...> writes: > Travis Oliphant wrote: > > > This is the gist of it. Basically you extend the Python scalars to > > include the single precision types (all the types Numeric supports). > > Would they be recognised as scalars by Python? In particular, could you There is no "scalar" category in Python. New scalar datatypes would be types with the same behaviour as the existing Python scalar types, but different from the. That means that explicitly type-checking code would not accept them, but everything else would. > use one as an index? Personally, this is what has bit me in the past: I At least up to Python 1.5, no, indices have to be of integer type. I don't know if that condition was extended in later versions. > could use A[3,2] as an index if A was type "Int" but not if it was > "Int16" for example. Ehmmm... Are you sure that is the right example? The restriction is on the type of the index, not on the type of the array. > Frankly, I have no idea what the implimentation details would be, but > could we get rid of rank-0 arrays altogether? I have always simply found If we introduce additional scalars, yes. Whether or not we would want to get rid of them is of course another question. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hi...@cn... Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- |
From: Chris B. <Chr...@no...> - 2002-09-16 18:45:43
|
Travis Oliphant wrote: > This is the gist of it. Basically you extend the Python scalars to > include the single precision types (all the types Numeric supports). Would they be recognised as scalars by Python? In particular, could you use one as an index? Personally, this is what has bit me in the past: I could use A[3,2] as an index if A was type "Int" but not if it was "Int16" for example. In any case, the type of A[3,2] should NOT depend on the precision of the numbers stored in A. Frankly, I have no idea what the implimentation details would be, but could we get rid of rank-0 arrays altogether? I have always simply found them strange and confusing... What are they really neccesary for (besides holding scalar values of different precision that standard Pyton scalars)? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Tim H. <tim...@ie...> - 2002-09-15 21:29:59
|
----- Original Message ----- From: "eric jones" <er...@en...> [Konrad] > > And it should fail, because a rank-0 array is not a sequence, so it > > doesn't have a length. [Eric] > I disagree. You should not have to write special code to check for a > specific case. It breaks one of the beauties of Numeric -- i.e. you can > write generic code that handles arrays of any size and type. Any method > that works on a 1 or more d array should also work on 0d arrays. If you > ask for its shape, it returns a tuple. No problem up to here. > If you ask for its size it > returns its length along its "first" axis. Here's where we part ways. As Konrad already pointed out, it return anArray.shape[0] except in the case of zero-D arrays where the arbitrary decision was made that: > This will always be 1. Why 1 and not 0 or -1 or 42. If you really had to return something, the thing to return would be None. > It allows for generic code. I don't see how. Even poking around in MA didn't convince me that this would help although I didn't spend enough time with it to get a great feel for it. The one function I did come close to working all the way through looked like it would be about _half_ as long if it didn't have to support the zero-rank cruft. > On this note: > I do not see the benefit of making a scalar type object that is separate > for 0d arrays. It seems to remove instead of enhance capabilities. > What does a scalar object buy that simply using 0d arrays for that > purpose does not? Unless somethings changed, you can't index into lists and what not with a rank-0 array, so returning ints or some subclass from integer arrays would be convenient. It would be easy to always return subclasses of int, float or complex (or object for those few who use object arrays) so that the results would always play nice with the rest of Python. However, given that the coercion rules have changed in numarray, I don't really see the point of returning anything other than int, float of complex. However, I have no objection to allowing the creation rank-0 arrays as long as they behave consistently with other array objects. [Konrad] > > - a.astype(N.Float) should also work for scalars > > > > Similarly, it would be nice if complex operations (real/imaginary > > part) would work on integers and floats. If these are needed, and I agree they would be nice, it seems that in order to integrate well with the rest of Python we should supply: numarray.astype(a, type) -> array or scalar of type type. numarray.imaginary(a) -> imaginary part of a numarray.imaginary(a) -> real part of a I actually had these latter two in JNumeric back when I was working on that, so I kinda thought they were Numeric, but I musta just added them in because I liked them. [Eric and Konrad agree that the inconsistency between Python and Numeric's mod should be cleaned up] Me too. -tim |
From: Konrad H. <hi...@cn...> - 2002-09-15 19:40:56
|
> > And it should fail, because a rank-0 array is not a sequence, so it > > doesn't have a length. > > I disagree. You should not have to write special code to check for a > specific case. It breaks one of the beauties of Numeric -- i.e. you can It is not a specific case, more like a specific value (for the rank). 1/a fails for a == 0, should that be changed as well? Let's examine some equivalent code pieces: - len(a) == a.shape[0] the second fails for rank 0, so the first one should fail as well - for i in range(len(a)): print a[i] works for all sequences. If len(a) doesn't fail (and I assume it would then return 1), a[1] shouldn't fail either. - len(a) == len(list(a)) for all sequences a. Should list(a) return [a] for a rank-0 array? For a scalar it fails. Actually this might be an argument for not having rank-0 arrays at all. Arrays are multidimensional sequences, but rank-0 arrays aren't. > returns its length along its "first" axis. This will always be 1. It > allows for generic code. Then please give an example where this genericity would be useful. > On this note: > I do not see the benefit of making a scalar type object that is separate > for 0d arrays. It seems to remove instead of enhance capabilities. > What does a scalar object buy that simply using 0d arrays for that > purpose does not? Compatibility, for example the ability to index a sequence with an element of an integer array. Also consistency with other Python sequence types. For example, [a][0] == a so one would expect also array([a])[0] == a but this would not be fully true if the left-hand side is a rank-0 array. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hi...@cn... Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- |