You can subscribe to this list here.
2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
From: Tom D. <tom...@al...> - 2006-08-31 18:08:24
|
Yes one can take a toy example and hack it to work but I don't necessarily have control over the input as to whether it is a list of object arrays, list of 1d heterogenous arrays, etc. Before I didn't need to worry about the input because numpy understood that a list of 1d arrays is a 2d piece of data. Now it understands this for all dtypes except object. My question was is this new set of semantics preferable to the old. I think your example kind of proves my point. Does it really make any sense for the following two ways of specifying an array give such different results? They strike me as _meaning_ the same thing. Doesn't it seem inconsistent to you? In [13]: array([array([1,'A', None], dtype=object),array([2,2,'Some string'],dtype=object)], dtype=object).shape Out[13]: (2,) and In [14]: array([array([1,'A', None], dtype=object),array([2,2,'Some string'],dtype=object)]).shape Out[14]: (2, 3) So my question is what is the _advantage_ of the new semantics? The two examples above used to give the same results. In what cases is it preferable for them to give different results? How does it make life simpler? On 8/31/06, Charles R Harris <cha...@gm...> wrote: > On 8/31/06, Tom Denniston <tom...@al...> wrote: > > > But i have hetergenious arrays that have numbers and strings and > > NoneType, etc. > > > > Take for instance: > > > > In [11]: numpy.array([numpy.array([1,'A', None]), > > numpy.array([2,2,'Some string'])], dtype=object) > > Out[11]: > > array([[1, A, None], > > [2, 2, Some string]], dtype=object) > > > > In [12]: numpy.array([numpy.array([1,'A', None]), > > numpy.array([2,2,'Some string'])], dtype=object).shape > > Out[12]: (2, 3) > > > > Works fine in Numeric and pre beta numpy but in beta numpy versions i > > get: > > > I think you want: > > In [59]: a = array([array([1,'A', None],dtype=object),array([2,2,'Some > string'],dtype=object)]) > > In [60]: a.shape > Out[60]: (2, 3) > > > Which makes good sense to me. > > Chuck > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
From: Charles R H. <cha...@gm...> - 2006-08-31 17:57:53
|
On 8/31/06, Charles R Harris <cha...@gm...> wrote: > > On 8/31/06, Tom Denniston <tom...@al...> wrote: > > > But i have hetergenious arrays that have numbers and strings and > > NoneType, etc. > > > > Take for instance: > > > > In [11]: numpy.array([numpy.array([1,'A', None]), > > numpy.array([2,2,'Some string'])], dtype=object) > > Out[11]: > > array([[1, A, None], > > [2, 2, Some string]], dtype=object) > > > > In [12]: numpy.array([numpy.array([1,'A', None]), > > numpy.array([2,2,'Some string'])], dtype=object).shape > > Out[12]: (2, 3) > > > > Works fine in Numeric and pre beta numpy but in beta numpy versions i > > get: > > > I think you want: > > In [59]: a = array([array([1,'A', None],dtype=object),array([2,2,'Some > string'],dtype=object)]) > > In [60]: a.shape > Out[60]: (2, 3) > > Which makes good sense to me. > OK, I changed my mind. I think you are right and this is a bug. Using svn revision 3098 I get In [2]: a = array([1,'A', None]) --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) /home/charris/<ipython console> TypeError: expected a readable buffer object Which is different than you get with beta 1 in any case. I think array should cast the objects in the list to the first common dtype, object in this case. So the previous should be shorthand for: In [3]: a = array([1,'A', None], dtype=object) In [4]: a.shape Out[4]: (3,) Chuck |
From: Charles R H. <cha...@gm...> - 2006-08-31 17:36:19
|
On 8/31/06, Tom Denniston <tom...@al...> wrote: > > But i have hetergenious arrays that have numbers and strings and NoneType, > etc. > > Take for instance: > > In [11]: numpy.array([numpy.array([1,'A', None]), > numpy.array([2,2,'Some string'])], dtype=object) > Out[11]: > array([[1, A, None], > [2, 2, Some string]], dtype=object) > > In [12]: numpy.array([numpy.array([1,'A', None]), > numpy.array([2,2,'Some string'])], dtype=object).shape > Out[12]: (2, 3) > > Works fine in Numeric and pre beta numpy but in beta numpy versions i get: I think you want: In [59]: a = array([array([1,'A', None],dtype=object),array([2,2,'Some string'],dtype=object)]) In [60]: a.shape Out[60]: (2, 3) Which makes good sense to me. Chuck |
From: Charles R H. <cha...@gm...> - 2006-08-31 17:26:19
|
On 8/31/06, Christopher Barker <Chr...@no...> wrote: > > Fernando Perez wrote: > > In [8]: N.array(3).shape > > Out[8]: () > > > In [11]: N.array([]).shape > > Out[11]: (0,) > > > I guess my only remaining question is: what is the difference between > > outputs #8 and #11 above? Is an empty shape tuple == array scalar, > > while a (0,) shape indicates a one-dimensional array with no elements? > > If this interpretation is correct, what is the usage of the latter > > kind of object, given how it can't even be indexed? > > It can be iterated over (with zero iterations): > > >>> a = N.array([]) > >>> for i in a: > ... print i > ... > > whereas the scalar can not: > > >>> b = N.array(3) > >>> b > array(3) > >>> for i in b: > ... print i > ... > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: iteration over a scalar (0-dim array) > > Of course the scalar isn't empty, so ti's different in that way too. Can > there be an empty scalar? It doesn't look like it. In fact, this looks > like it may be a bug: > >>> a = N.array([1,2,3]).sum(); a.shape; a.size; a > () > 1 > 6 > > That's what I'd expect, but what if you start with a (0,) array: > >>> a = N.array([]).sum(); a.shape; a.size; a > () > 1 > 0 > > where did that zero come from? I think that is correct, sums over empty sets are conventionally set to zero because they are conceived of as adding all the values in the set to zero. Typically this would be implemented as sum = 0 for i in set : sum += i; Chuck |
From: Tom D. <tom...@al...> - 2006-08-31 17:00:09
|
But i have hetergenious arrays that have numbers and strings and NoneType, etc. Take for instance: In [11]: numpy.array([numpy.array([1,'A', None]), numpy.array([2,2,'Some string'])], dtype=object) Out[11]: array([[1, A, None], [2, 2, Some string]], dtype=object) In [12]: numpy.array([numpy.array([1,'A', None]), numpy.array([2,2,'Some string'])], dtype=object).shape Out[12]: (2, 3) Works fine in Numeric and pre beta numpy but in beta numpy versions i get: In [6]: numpy.array([numpy.array([1,'A', None]), numpy.array([2,2,'Some string'])], dtype=object) Out[6]: array([[1 A None], [2 2 Some string]], dtype=object) In [7]: numpy.array([numpy.array([1,'A', None]), numpy.array([2,2,'Some string'])], dtype=object).shape Out[7]: (2,) But a lists of lists still gives: In [9]: numpy.array([[1,'A', None], [2,2,'Some string']], dtype=object).shape Out[9]: (2, 3) And if you omit the dtype and use a list of arrays then you get a string array with 2,3 dimensions: In [11]: numpy.array([numpy.array([1,'A', None]), numpy.array([2,2,'Some string'])]).shape Out[11]: (2, 3) This new behavior strikes me as inconsistent. One of the things I love about numpy is the ufuncs, array constructors, etc don't care about whether you pass a combination of lists, arrays, tuples, etc. They just know what you _mean_. And what you _mean_ by a list of lists, tuple of arrays, list of arrays, etc is very consistent across constructors and functions. I think making an exception for dtype=object introduces a lot of inconsistencies and it isn't clear to me what is gained. Does anyone commonly use the array interface in a manner that this new behavior is actuallly favorable? I may be overlooking a common use case or something like that. On 8/31/06, Charles R Harris <cha...@gm...> wrote: > > > > On 8/31/06, Tom Denniston > <tom...@al...> wrote: > > > > For this simple example yes, but if one of the nice things about the array > constructors is that they know that lists, tuple and arrays are just > sequences and any combination of them is valid numpy input. So for instance > a list of tuples yields a 2d array. A list of tuples of 1d arrays yields a > 3d array. A list of 1d arrays yields 2d array. This was the case > consistently across all dtypes. Now it is the case across all of them > except for the dtype=object which has this unusual behavior. I agree that > vstack is a better choice when you know you have a list of arrays but it is > less useful when you don't know and you can't force a type in the vstack > function so there is no longer an equivalent to the dtype=object behavior: > > > > In [7]: numpy.array([numpy.array([1,2,3]), numpy.array([4,5,6])], > dtype=object) > > Out[7]: > > array([[1, 2, 3], > > [4, 5, 6]], dtype=object) > > > What are you trying to do? If you want integers: > > In [32]: a = array([array([1,2,3]), array([4,5,6])], dtype=int) > > In [33]: a.shape > Out[33]: (2, 3) > > > If you want objects, you have them: > > In [30]: a = array([array([1,2,3]), array([4,5,6])], dtype=object) > > In [31]: a.shape > Out[31]: (2,) > > i.e, a is an array containing two array objects. > > Chuck > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
From: Tim H. <tim...@ie...> - 2006-08-31 16:57:42
|
Christopher Barker wrote: > Fernando Perez wrote: > >> In [8]: N.array(3).shape >> Out[8]: () >> > > >> In [11]: N.array([]).shape >> Out[11]: (0,) >> > > >> I guess my only remaining question is: what is the difference between >> outputs #8 and #11 above? Is an empty shape tuple == array scalar, >> while a (0,) shape indicates a one-dimensional array with no elements? >> If this interpretation is correct, what is the usage of the latter >> kind of object, given how it can't even be indexed? >> > > It can be iterated over (with zero iterations): > > >>> a = N.array([]) > >>> for i in a: > ... print i > ... > > whereas the scalar can not: > > >>> b = N.array(3) > >>> b > array(3) > >>> for i in b: > ... print i > ... > Traceback (most recent call last): > File "<stdin>", line 1, in ? > TypeError: iteration over a scalar (0-dim array) > > Of course the scalar isn't empty, so ti's different in that way too. Can > there be an empty scalar? It doesn't look like it. In fact, this looks > like it may be a bug: > >>> a = N.array([1,2,3]).sum(); a.shape; a.size; a > () > 1 > 6 > > That's what I'd expect, but what if you start with a (0,) array: > >>> a = N.array([]).sum(); a.shape; a.size; a > () > 1 > 0 > > where did that zero come from? > More or less from: >>> numpy.add.identity 0 All the ufuncs have an identity function that they use as a starting point for reduce and accumulate. Sum doesn't appear to actually ahve one, but since it's more or less the same as add.reduce it's probably good that it has the same behavior. Note that this also matches the behavior of python's built in sum, although there the identity is called 'start'. -tim > >>> N.__version__ > '1.0b4' > > -Chris > > > > |
From: Charles R H. <cha...@gm...> - 2006-08-31 16:51:06
|
On 8/31/06, Christopher Barker <Chr...@no...> wrote: > > Jonathan Taylor wrote: > > When trying to install 1.0b4 I had trouble getting it to detect my > > installed atlas. This was because the shipped site.cfg had; > > > > [atlas] > > library_dirs = /usr/lib/atlas/3dnow/ > > atlas_libs = lapack, blas > > > > but I had to change 3dnow to sse2 due to my current state of > > pentiumness. In any case it should probabally look in all the > > possible locations instead of just AMD's location. > > "All possible locations" is pretty much impossible. There really isn't > any choice but for individuals to customize site.cfg for their setup. > that's why it's called "site".cfg. > > I would like to see a pretty good collection of examples, most of them > commented out, in there, however. i.e.: I need this on fc5 x86_64 [atlas] library_dirs = /usr/lib64/atlas atlas_libs = lapack, blas, cblas, atlas I think this should be automatic. Apart from debian, the /usr/lib64 directory is pretty much standard for 64bit linux distros. Chuck |
From: Christopher B. <Chr...@no...> - 2006-08-31 16:46:16
|
Fernando Perez wrote: > In [8]: N.array(3).shape > Out[8]: () > In [11]: N.array([]).shape > Out[11]: (0,) > I guess my only remaining question is: what is the difference between > outputs #8 and #11 above? Is an empty shape tuple == array scalar, > while a (0,) shape indicates a one-dimensional array with no elements? > If this interpretation is correct, what is the usage of the latter > kind of object, given how it can't even be indexed? It can be iterated over (with zero iterations): >>> a = N.array([]) >>> for i in a: ... print i ... whereas the scalar can not: >>> b = N.array(3) >>> b array(3) >>> for i in b: ... print i ... Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: iteration over a scalar (0-dim array) Of course the scalar isn't empty, so ti's different in that way too. Can there be an empty scalar? It doesn't look like it. In fact, this looks like it may be a bug: >>> a = N.array([1,2,3]).sum(); a.shape; a.size; a () 1 6 That's what I'd expect, but what if you start with a (0,) array: >>> a = N.array([]).sum(); a.shape; a.size; a () 1 0 where did that zero come from? >>> N.__version__ '1.0b4' -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Christopher B. <Chr...@no...> - 2006-08-31 16:36:28
|
Jonathan Taylor wrote: > When trying to install 1.0b4 I had trouble getting it to detect my > installed atlas. This was because the shipped site.cfg had; > > [atlas] > library_dirs = /usr/lib/atlas/3dnow/ > atlas_libs = lapack, blas > > but I had to change 3dnow to sse2 due to my current state of > pentiumness. In any case it should probabally look in all the > possible locations instead of just AMD's location. "All possible locations" is pretty much impossible. There really isn't any choice but for individuals to customize site.cfg for their setup. that's why it's called "site".cfg. I would like to see a pretty good collection of examples, most of them commented out, in there, however. i.e.: ## for AMD atlas: #library_dirs = /usr/lib/atlas/3dnow/ #atlas_libs = lapack, blas ## for Fedora Core 4 sse2 atlas: #library_dirs = /usr/lib/sse2/ #atlas_libs = lapack, blas etc, etc. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Charles R H. <cha...@gm...> - 2006-08-31 16:24:41
|
On 8/31/06, Tom Denniston <tom...@al...> wrote: > > For this simple example yes, but if one of the nice things about the array > constructors is that they know that lists, tuple and arrays are just > sequences and any combination of them is valid numpy input. So for instance > a list of tuples yields a 2d array. A list of tuples of 1d arrays yields a > 3d array. A list of 1d arrays yields 2d array. This was the case > consistently across all dtypes. Now it is the case across all of them > except for the dtype=object which has this unusual behavior. I agree that > vstack is a better choice when you know you have a list of arrays but it is > less useful when you don't know and you can't force a type in the vstack > function so there is no longer an equivalent to the dtype=object behavior: > > In [7]: numpy.array([numpy.array([1,2,3]), numpy.array([4,5,6])], > dtype=object) > Out[7]: > array([[1, 2, 3], > [4, 5, 6]], dtype=object) > What are you trying to do? If you want integers: In [32]: a = array([array([1,2,3]), array([4,5,6])], dtype=int) In [33]: a.shape Out[33]: (2, 3) If you want objects, you have them: In [30]: a = array([array([1,2,3]), array([4,5,6])], dtype=object) In [31]: a.shape Out[31]: (2,) i.e, a is an array containing two array objects. Chuck |
From: Tom D. <tom...@al...> - 2006-08-31 15:59:41
|
For this simple example yes, but if one of the nice things about the array constructors is that they know that lists, tuple and arrays are just sequences and any combination of them is valid numpy input. So for instance a list of tuples yields a 2d array. A list of tuples of 1d arrays yields a 3d array. A list of 1d arrays yields 2d array. This was the case consistently across all dtypes. Now it is the case across all of them except for the dtype=object which has this unusual behavior. I agree that vstack is a better choice when you know you have a list of arrays but it is less useful when you don't know and you can't force a type in the vstack function so there is no longer an equivalent to the dtype=object behavior: In [7]: numpy.array([numpy.array([1,2,3]), numpy.array([4,5,6])], dtype=object) Out[7]: array([[1, 2, 3], [4, 5, 6]], dtype=object) In [8]: numpy.vstack([numpy.array([1,2,3]), numpy.array([4,5,6])], dtype=object) --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) TypeError: vstack() got an unexpected keyword argument 'dtype' On 8/31/06, Charles R Harris <cha...@gm...> wrote: > On 8/31/06, Tom Denniston > <tom...@al...> wrote: > > > In version 0.9.6 one used to be able to do this: > > > > In [4]: numpy.__version__ > > Out[4]: '0.9.6' > > > > > > In [6]: numpy.array([numpy.array([4,5,6]), numpy.array([1,2,3])], > > dtype=object).shape > > Out[6]: (2, 3) > > > > > > In beta 1 numpy.PyObject no longer exists. I was trying to get the > > same behavior with dtype=object but it doesn't work: > > > > In [7]: numpy.__version__ > > Out[7]: '1.0b1' > > > > In [8]: numpy.array([numpy.array ([4,5,6]), numpy.array([1,2,3])], > > dtype=object).shape > > Out[8]: (2,) > > > The latter looks more correct, in that is produces an array of objects. To > get the previous behaviour there is the function vstack: > > In [6]: a = array([1,2,3]) > > In [7]: b = array([4,5,6]) > > In [8]: vstack([a,b]) > Out[8]: > array([[1, 2, 3], > [4, 5, 6]]) > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
From: Charles R H. <cha...@gm...> - 2006-08-31 15:44:20
|
On 8/31/06, Tom Denniston <tom...@al...> wrote: > > In version 0.9.6 one used to be able to do this: > > In [4]: numpy.__version__ > Out[4]: '0.9.6' > > > In [6]: numpy.array([numpy.array([4,5,6]), numpy.array([1,2,3])], > dtype=object).shape > Out[6]: (2, 3) > > > In beta 1 numpy.PyObject no longer exists. I was trying to get the > same behavior with dtype=object but it doesn't work: > > In [7]: numpy.__version__ > Out[7]: '1.0b1' > > In [8]: numpy.array([numpy.array([4,5,6]), numpy.array([1,2,3])], > dtype=object).shape > Out[8]: (2,) The latter looks more correct, in that is produces an array of objects. To get the previous behaviour there is the function vstack: In [6]: a = array([1,2,3]) In [7]: b = array([4,5,6]) In [8]: vstack([a,b]) Out[8]: array([[1, 2, 3], [4, 5, 6]]) Chuck |
From: <hu...@ya...> - 2006-08-31 15:44:12
|
Hi, sorry to bother you with probably plenty of stupid question but I would like to clarify my mind with dtype. I have a problem to view a recarray, I'm not sure but I suspect a bug or at least a problem I have an array with some data, the array is very big but I have no problem with numpy. In [44]: data_end Out[44]: array([[ 2.66000000e+02, 5.16300000e+04, 1.00000000e+00, ..., -1.04130435e+00, 1.47304565e+02, 4.27402449e+00], [ 2.66000000e+02, 5.16300000e+04, 2.00000000e+00, ..., -6.52190626e-01, 1.64214981e+02, 1.58334379e+01], [ 2.66000000e+02, 5.16300000e+04, 4.00000000e+00, ..., -7.65136838e-01, 1.33340195e+02, 9.84033298e+00], ..., [ 9.78000000e+02, 5.24310000e+04, 6.32000000e+02, ..., 3.06083832e+01, 6.71210251e+01, 1.18813887e+01], [ 9.78000000e+02, 5.24310000e+04, 6.36000000e+02, ..., 3.05993423e+01, 1.10403000e+02, 5.81539488e+00], [ 9.78000000e+02, 5.24310000e+04, 6.40000000e+02, ..., 3.05382938e+01, 1.26916304e+01, 3.25683937e+01]]) In [45]: data_end.shape Out[45]: (567486, 7) In [46]: data_end.dtype Out[46]: dtype('<f8') I want to have this array with certain dtype so I transform it in recarray (I have unable ot have different type for each columns in the array but I'm just beginning to play with dtype. So I did: In [47]: fields = ['PLATEID', 'MJD', 'FIBERID', 'RA', 'DEC','V_DISP','V_DISP_ERR'] In [48]: In [48]: type_descr = numpy.dtype({'names':fields,'formats': ['>i2','>i4','>i2','>f4','>f4','>f4','>f4']}) In [49]: b = numpy.rec.fromarrays(data_end.transpose(),type_descr) In [50]: b[:1] Out[50]: recarray([ (266, 51630, 1, 146.71420288085938, -1.041304349899292, 147.3045654296875, 4.274024486541748)], dtype=[('PLATEID', '>i2'), ('MJD', '>i4'), ('FIBERID', '>i2'), ('RA', '>f4'), ('DEC', '>f4'), ('V_DISP', '>f4'), ('V_DISP_ERR', '>f4')]) In [51]: b[1] Out[51]: (266, 51630, 2, 146.74412536621094, -0.65219062566757202, 164.21498107910156, 15.833437919616699) but I obtain an error when I want to print the recarray b (it's working for smallest array): In [54]: b[:10] Out[54]: recarray([ (266, 51630, 1, 146.71420288085938, -1.041304349899292, 147.3045654296875, 4.274024486541748), (266, 51630, 2, 146.74412536621094, -0.65219062566757202, 164.21498107910156, 15.833437919616699), (266, 51630, 4, 146.62857055664062, -0.76513683795928955, 133.34019470214844, 9.8403329849243164), (266, 51630, 6, 146.63166809082031, -0.98827779293060303, 146.91035461425781, 30.08709716796875), (266, 51630, 7, 146.91944885253906, -0.99049174785614014, 152.96893310546875, 12.429832458496094), (266, 51630, 9, 146.76339721679688, -0.81043314933776855, 347.72918701171875, 41.387767791748047), (266, 51630, 10, 146.62281799316406, -0.9513852596282959, 162.53567504882812, 24.676788330078125), (266, 51630, 11, 146.93409729003906, -0.67040395736694336, 266.56011962890625, 10.875675201416016), (266, 51630, 12, 146.96389770507812, -0.54500257968902588, 92.040328979492188, 18.999214172363281), (266, 51630, 13, 146.9635009765625, -0.75935173034667969, 72.828048706054688, 13.028598785400391)], dtype=[('PLATEID', '>i2'), ('MJD', '>i4'), ('FIBERID', '>i2'), ('RA', '>f4'), ('DEC', '>f4'), ('V_DISP', '>f4'), ('V_DISP_ERR', '>f4')]) So I would like to know if it's normal. And another question is it possile to do, in theory, something like: b = numpy.array(data_end,dtype=type_descr) or all individual array element must have the same dtype? To replace the context, I have a big fits table, I want to use only some columns from the table so I did: table = pyfits.getdata('gal_info_dr4_v5_1b.fit') #pyfits can't read, at least now the gzip file #the file is a fits table file so we look in the pyfits doc to read it! fields = ['PLATEID', 'MJD', 'FIBERID', 'RA', 'DEC','V_DISP','V_DISP_ERR'] type_descr = numpy.dtype({'names':fields,'formats': ['<i2','<i4','<i2','<f8','<f8','<f8','<f8']}) data_end = numpy.zeros((table.shape[0],len(fields))) for i in range(len(fields)): data_end[:,i] = table[:].field(fields[i]) but I want to keep the type from the fits file for each field perhaps there are a better way to do it. Thank you very much. N. The error when I try to print the big recarray: In [53]: b --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) /home/gruel/Desktop/SDSS/DR4/<ipython console> /home/gruel/usr/lib/python2.4/site-packages/IPython/Prompts.py in __call__(self, arg) 514 515 # and now call a possibly user-defined print mechanism --> 516 manipulated_val = self.display(arg) 517 518 # user display hooks can change the variable to be stored in /home/gruel/usr/lib/python2.4/site-packages/IPython/Prompts.py in _display(self, arg) 538 """ 539 --> 540 return self.shell.hooks.result_display(arg) 541 542 # Assign the default display method: /home/gruel/usr/lib/python2.4/site-packages/IPython/hooks.py in __call__(self, *args, **kw) 132 #print "prio",prio,"cmd",cmd #dbg 133 try: --> 134 ret = cmd(*args, **kw) 135 return ret 136 except ipapi.TryNext, exc: /home/gruel/usr/lib/python2.4/site-packages/IPython/hooks.py in result_display(self, arg) 153 154 if self.rc.pprint: --> 155 out = pformat(arg) 156 if '\n' in out: 157 # So that multi-line strings line up with the left column of /usr/lib/python2.4/pprint.py in pformat(self, object) 108 def pformat(self, object): 109 sio = _StringIO() --> 110 self._format(object, sio, 0, 0, {}, 0) 111 return sio.getvalue() 112 /usr/lib/python2.4/pprint.py in _format(self, object, stream, indent, allowance, context, level) 126 self._readable = False 127 return --> 128 rep = self._repr(object, context, level - 1) 129 typ = _type(object) 130 sepLines = _len(rep) > (self._width - 1 - indent - allowance) /usr/lib/python2.4/pprint.py in _repr(self, object, context, level) 192 def _repr(self, object, context, level): 193 repr, readable, recursive = self.format(object, context.copy(), --> 194 self._depth, level) 195 if not readable: 196 self._readable = False /usr/lib/python2.4/pprint.py in format(self, object, context, maxlevels, level) 204 and whether the object represents a recursive construct. 205 """ --> 206 return _safe_repr(object, context, maxlevels, level) 207 208 /usr/lib/python2.4/pprint.py in _safe_repr(object, context, maxlevels, level) 289 return format % _commajoin(components), readable, recursive 290 --> 291 rep = repr(object) 292 return rep, (rep and not rep.startswith('<')), False 293 /home/gruel/usr/lib/python2.4/site-packages/numpy/core/numeric.py in array_repr(arr, max_line_width, precision, suppress_small) 389 if arr.size > 0 or arr.shape==(0,): 390 lst = array2string(arr, max_line_width, precision, suppress_small, --> 391 ', ', "array(") 392 else: # show zero-length shape unless it is (0,) 393 lst = "[], shape=%s" % (repr(arr.shape),) /home/gruel/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py in array2string(a, max_line_width, precision, suppress_small, separator, prefix, style) 202 else: 203 lst = _array2string(a, max_line_width, precision, suppress_small, --> 204 separator, prefix) 205 return lst 206 /home/gruel/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py in _array2string(a, max_line_width, precision, suppress_small, separator, prefix) 137 if a.size > _summaryThreshold: 138 summary_insert = "..., " --> 139 data = _leading_trailing(a) 140 else: 141 summary_insert = "" /home/gruel/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py in _leading_trailing(a) 108 if a.ndim == 1: 109 if len(a) > 2*_summaryEdgeItems: --> 110 b = _gen.concatenate((a[:_summaryEdgeItems], 111 a[-_summaryEdgeItems:])) 112 else: TypeError: expected a readable buffer object Out[53]: |
From: Charles R H. <cha...@gm...> - 2006-08-31 15:33:39
|
On 8/31/06, Fernando Perez <fpe...@gm...> wrote: > > On 8/31/06, Travis Oliphant <oli...@ie...> wrote: > > > What about > > > > N.array(3).size > > > > N.array([3]).size > > > > N.array([3,3]).size > > > > Essentially, the [] is being treated as an object when you explicitly > > ask for an object array in exactly the same way as 3 is being treated as > > a number in the default case. It's just that '[' ']' is "also" being > > used as the dimension delimiter and thus the confusion. > > > > It is consistent. It's a corner case, and I have no problem fixing the > > special-case code running when dtype=object so that array([], > > dtype=object) returns an empty array, if that is the consensus. > > I wasn't really complaining: these are corner cases I've never seen in > real use, so I'm not really sure how critical it is to worry about > them. Though I could see code which does automatic size/shape checks > tripping on some of them. The shape tuples shed a bit of light on > what's going on for the surprised (like myself): > > In [8]: N.array(3).shape > Out[8]: () > > In [9]: N.array([3]).shape > Out[9]: (1,) > > In [10]: N.array([3,3]).shape > Out[10]: (2,) > > In [11]: N.array([]).shape > Out[11]: (0,) > > In [12]: N.array([[]]).shape > Out[12]: (1, 0) > > In [13]: N.array([[],[]]).shape > Out[13]: (2, 0) > > > I won't really vote for any changes one way or another, as far as I'm > concerned it's one of those 'learn the library' things. I do realize > that the near-ambiguity between '[]' as an empty object and '[]' as > the syntactic delimiter for a container makes this case a bit of a > gotcha. > > I guess my only remaining question is: what is the difference between > outputs #8 and #11 above? Is an empty shape tuple == array scalar, > while a (0,) shape indicates a one-dimensional array with no elements? > If this interpretation is correct, what is the usage of the latter > kind of object, given how it can't even be indexed? > > In [15]: N.array([])[0] > > --------------------------------------------------------------------------- > exceptions.IndexError Traceback (most > recent call last) > > /home/fperez/research/code/mjmdim/pycode/<ipython console> > > IndexError: index out of bounds > > > And is this really expected? > > In [18]: N.array([]).any() > Out[18]: False This could be interpreted as : exists x, x element of array, s.t. x is true. In [19]: N.array([]).all() > Out[19]: True Seems right: for all x, x element of array, x is true. It's a bit funny to have an array for which 'no elements are true' > (any==false), yet 'all are true' (all==true), isn't it? Fun with empty sets! The question is, is a zero dimensional array an empty container or does it contain its value. The numpy choice of treating zero dimensional arrays as both empty containers and scalar values makes the determination a bit ambiguous although it is consistent with the indexing convention. Chuck |
From: Fernando P. <fpe...@gm...> - 2006-08-31 15:08:38
|
On 8/31/06, Travis Oliphant <oli...@ie...> wrote: > What about > > N.array(3).size > > N.array([3]).size > > N.array([3,3]).size > > Essentially, the [] is being treated as an object when you explicitly > ask for an object array in exactly the same way as 3 is being treated as > a number in the default case. It's just that '[' ']' is "also" being > used as the dimension delimiter and thus the confusion. > > It is consistent. It's a corner case, and I have no problem fixing the > special-case code running when dtype=object so that array([], > dtype=object) returns an empty array, if that is the consensus. I wasn't really complaining: these are corner cases I've never seen in real use, so I'm not really sure how critical it is to worry about them. Though I could see code which does automatic size/shape checks tripping on some of them. The shape tuples shed a bit of light on what's going on for the surprised (like myself): In [8]: N.array(3).shape Out[8]: () In [9]: N.array([3]).shape Out[9]: (1,) In [10]: N.array([3,3]).shape Out[10]: (2,) In [11]: N.array([]).shape Out[11]: (0,) In [12]: N.array([[]]).shape Out[12]: (1, 0) In [13]: N.array([[],[]]).shape Out[13]: (2, 0) I won't really vote for any changes one way or another, as far as I'm concerned it's one of those 'learn the library' things. I do realize that the near-ambiguity between '[]' as an empty object and '[]' as the syntactic delimiter for a container makes this case a bit of a gotcha. I guess my only remaining question is: what is the difference between outputs #8 and #11 above? Is an empty shape tuple == array scalar, while a (0,) shape indicates a one-dimensional array with no elements? If this interpretation is correct, what is the usage of the latter kind of object, given how it can't even be indexed? In [15]: N.array([])[0] --------------------------------------------------------------------------- exceptions.IndexError Traceback (most recent call last) /home/fperez/research/code/mjmdim/pycode/<ipython console> IndexError: index out of bounds And is this really expected? In [18]: N.array([]).any() Out[18]: False In [19]: N.array([]).all() Out[19]: True It's a bit funny to have an array for which 'no elements are true' (any==false), yet 'all are true' (all==true), isn't it? Regards, f |
From: Torgil S. <tor...@gm...> - 2006-08-31 14:57:35
|
> Yes, because you are adding a signed scalar to an unsigned scalar and a > float64 is the only thing that can handle it > > t+numpy.uint64(1) Thanks, this make sense. This is a good thing to have back in the head. //Torgil On 8/31/06, Travis Oliphant <oli...@ie...> wrote: > Torgil Svensson wrote: > > I'm using windows datetimes (100nano-seconds since 0001,1,1) as time > > in a numpy array and was hit by this behaviour. > > > > > >>>> numpy.__version__ > >>>> > > '1.0b4' > > > >>>> a=numpy.array([632925394330000000L],numpy.uint64) > >>>> t=a[0] > >>>> t > >>>> > > 632925394330000000L > > > >>>> type(t) > >>>> > > <type 'numpy.uint64'> > > > >>>> t+1 > >>>> > > 6.3292539433e+017 > > > >>>> type(t+1) > >>>> > > <type 'numpy.float64'> > > > >>>> t==(t+1) > >>>> > > True > > > > I was trying to set t larger than any time in an array. Is there any > > reason for the scalar to upcast in this case? > > > Yes, because you are adding a signed scalar to an unsigned scalar and a > float64 is the only thing that can handle it (well actually it should be > the long double scalar but we've made a special case here because long > doubles are not that common). Add an unsigned scalar > > t+numpy.uint64(1) > > to get what you want. > > -Travis > > > > //Torgil > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Num...@li... > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
From: Christophe-Blondeau <Chr...@on...> - 2006-08-31 14:43:51
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> <title></title> </head> <body bgcolor="#ffffff" text="#000000"> Hi everybody<br> <br> I've succesfully built Numpy-1.0b4 on HP-UX 11.11 with<br> all tests passed succesfully (gcc 4.1.1) on Python 2.4.2<br> <br> I try to import 2 basic Fortran modules compiled with numpy/f2py<br> (fcompiler = gfortran (gcc 4.1.1))<br> <br> The modules hello1 and hello2 are almost identical:<br> <br> C File hello1.f<br> subroutine foo1 (a)<br> integer a<br> print*, "Hello from Fortran(foo1)!"<br> print*, "a=",a<br> end<br> <br> C File hello2.f<br> subroutine foo2 (a)<br> integer a<br> print*, "Hello from Fortran(foo2)!"<br> print*, "a=",a<br> end<br> <br> Then run: python -c 'import hello1; import hello2' which<br> ends with a beautifull segfault !<br> I also tried the old f2py version (based on Numeric) and everything <br> goes well<br> Note that the importation of only ONE f2py module is OK.<br> <br> Investigating a little bit further, I deleted the call to import_array()<br> in "hello1module.c" and "hello2module.c" and the segfault dissapear ...<br> It turns out that something goes wrong this the importation of "numpy.core.multiarray"<br> on my system.<br> The same thing happends if I import multiarray by hand in the Python session:<br> >>> import numpy.core.multiarray<br> >>> import hello1 (with import_array() deleted)<br> >>> import hello2 (with import_array() deleted)<br> ---> segfault<br> <br> Unfortunately thats prevents me from using Scipy because modules based on Fortran<br> and compiled with numpy/f2py during the building phase also segfault.<br> <br> If anyone has some help, I will greatly appreciate<br> Thanks<br> Christophe<br> <br> <div class="moz-signature">-- <br> <meta content="text/html;" http-equiv="Content-Type"> <title></title> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> <title></title> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> <title></title> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"> <br> </div> </body> </html> |
From: Darren D. <dd...@co...> - 2006-08-31 14:37:27
|
On Tuesday 29 August 2006 19:24, Fernando Perez wrote: > On 8/29/06, Travis Oliphant <oli...@ie...> wrote: > > Hi all, > > > > Classes start for me next Tuesday, and I'm teaching a class for which I > > will be using NumPy / SciPy extensively. I need to have a release of > > these two (and hopefully matplotlib) that work with each other. > > > > Therefore, I'm going to make a 1.0b5 release of NumPy over the weekend > > (probably Monday), and also get a release of SciPy out as well. At that > > point, I'll only be available for bug-fixes to 1.0. Therefore, the next > > release after 1.0b5 I would like to be 1.0rc1 (release-candidate 1). > > What's the status of these 'overwriting' messages? > > planck[/tmp]> python -c 'import scipy;scipy.test()' > Overwriting info=<function info at 0x40ba748c> from scipy.misc (was > <function info at 0x4080409c> from numpy.lib.utils) > Overwriting fft=<function fft at 0x430ae33c> from scipy.fftpack.basic > (was <module 'numpy.fft' from > '/home/fperez/tmp/local/lib/python2.3/site-packages/numpy/fft/__init__.pyc' >> from > /home/fperez/tmp/local/lib/python2.3/site-packages/numpy/fft/__init__.pyc) > ... > > I was under the impression you'd decided to quiet them out, but they > seem to be making a comeback. Will these messages be included in NumPy-1.0? |
From: Jonathan T. <jon...@ut...> - 2006-08-31 14:19:23
|
When trying to install 1.0b4 I had trouble getting it to detect my installed atlas. This was because the shipped site.cfg had; [atlas] library_dirs = /usr/lib/atlas/3dnow/ atlas_libs = lapack, blas but I had to change 3dnow to sse2 due to my current state of pentiumness. In any case it should probabally look in all the possible locations instead of just AMD's location. Cheers. Jon. |
From: Tom D. <tom...@al...> - 2006-08-31 13:47:39
|
In version 0.9.6 one used to be able to do this: In [4]: numpy.__version__ Out[4]: '0.9.6' In [6]: numpy.array([numpy.array([4,5,6]), numpy.array([1,2,3])], dtype=object).shape Out[6]: (2, 3) In beta 1 numpy.PyObject no longer exists. I was trying to get the same behavior with dtype=object but it doesn't work: In [7]: numpy.__version__ Out[7]: '1.0b1' In [8]: numpy.array([numpy.array([4,5,6]), numpy.array([1,2,3])], dtype=object).shape Out[8]: (2,) Is this an intentional change? |
From: Travis O. <oli...@ie...> - 2006-08-31 13:46:05
|
Torgil Svensson wrote: > I'm using windows datetimes (100nano-seconds since 0001,1,1) as time > in a numpy array and was hit by this behaviour. > > >>>> numpy.__version__ >>>> > '1.0b4' > >>>> a=numpy.array([632925394330000000L],numpy.uint64) >>>> t=a[0] >>>> t >>>> > 632925394330000000L > >>>> type(t) >>>> > <type 'numpy.uint64'> > >>>> t+1 >>>> > 6.3292539433e+017 > >>>> type(t+1) >>>> > <type 'numpy.float64'> > >>>> t==(t+1) >>>> > True > > I was trying to set t larger than any time in an array. Is there any > reason for the scalar to upcast in this case? > Yes, because you are adding a signed scalar to an unsigned scalar and a float64 is the only thing that can handle it (well actually it should be the long double scalar but we've made a special case here because long doubles are not that common). Add an unsigned scalar t+numpy.uint64(1) to get what you want. -Travis > //Torgil > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
From: Travis O. <oli...@ie...> - 2006-08-31 13:40:50
|
Fernando Perez wrote: > On 8/30/06, Stefan van der Walt <st...@su...> wrote: > > >> The current behaviour makes sense, but is maybe not consistent: >> >> N.array([],dtype=object).size == 1 >> N.array([[],[]],dtype=object).size == 2 >> > > Yes, including one more term in this check: > > In [5]: N.array([],dtype=object).size > Out[5]: 1 > > In [6]: N.array([[]],dtype=object).size > Out[6]: 1 > > In [7]: N.array([[],[]],dtype=object).size > Out[7]: 2 > > Intuitively, I'd have expected the answers to be 0,1,2, instead of 1,1,2. > > What about N.array(3).size N.array([3]).size N.array([3,3]).size Essentially, the [] is being treated as an object when you explicitly ask for an object array in exactly the same way as 3 is being treated as a number in the default case. It's just that '[' ']' is "also" being used as the dimension delimiter and thus the confusion. It is consistent. It's a corner case, and I have no problem fixing the special-case code running when dtype=object so that array([], dtype=object) returns an empty array, if that is the consensus. -Travis > Cheers, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
From: Steve L. <lis...@ar...> - 2006-08-31 13:24:33
|
On Aug 31, 2006, at 6:19 AM, LANDRIU David SAp wrote: > I learned you answered me, but I did not get > your message : can you send it to me again ? Does this help? http://sourceforge.net/mailarchive/forum.php? thread_id=30384097&forum_id=4890 -steve |
From: LANDRIU D. S. <la...@di...> - 2006-08-31 10:20:04
|
Hello, I learned you answered me, but I did not get your message : can you send it to me again ? Thanks , David Landriu -------------------------------------------------------------------- David Landriu DAPNIA/SAp CEA SACLAY (France) Phone : (33|0)169088785 Fax : (33|0)169086577 --------------------------------------------------------------------- |
From: Torgil S. <tor...@gm...> - 2006-08-31 06:15:42
|
I'm using windows datetimes (100nano-seconds since 0001,1,1) as time in a numpy array and was hit by this behaviour. >>> numpy.__version__ '1.0b4' >>> a=numpy.array([632925394330000000L],numpy.uint64) >>> t=a[0] >>> t 632925394330000000L >>> type(t) <type 'numpy.uint64'> >>> t+1 6.3292539433e+017 >>> type(t+1) <type 'numpy.float64'> >>> t==(t+1) True I was trying to set t larger than any time in an array. Is there any reason for the scalar to upcast in this case? //Torgil |