From: travlr <vel...@gm...> - 2005-07-18 07:07:09
|
Hi Francesc and company :) Having recieved the 1.1 release notification, I also noticed the "support for indexes" in the 1.0 release. Does this happen to include slicing with an index array (a la numarray)? An example would be the non-sequential return of: index =3D numarray.where(...)[0], then applied to further pytable retrieval. Also, I didn't notice any mentionings of the index improvements in the documentation. Thank you guys, Peter |
From: Francesc A. <fa...@ca...> - 2005-07-18 11:06:04
|
Hi Peter, El dl 18 de 07 del 2005 a les 03:06 -0400, en/na travlr va escriure: > Having recieved the 1.1 release notification, I also noticed the > "support for indexes" in the 1.0 release. Does this happen to include > slicing with an index array (a la numarray)? An example would be the > non-sequential return of: index = numarray.where(...)[0], then applied > to further pytable retrieval. Also, I didn't notice any mentionings of > the index improvements in the documentation. Let me explain some points first. Unfortunately, the "indexing" verb receives many meanings in computer sciences. When I first announced the PyTables support for indexing, I meant that it can sort the columns of tables in order to do faster searches (i.e. values that fulfill some condition). This kind of indexation was actually implemented back in 0.9, although it was improved in 1.0 to allow the indexation of tables with more than 2**31 columns. In fact, from 1.0 on, all the objects in PyTables are supposed to support a number of entries up to 2**62. In fact, this kind of support is mentioned in the User's Manual as the first feature in the "Main Features" section: http://pytables.sourceforge.net/html-doc/usersguide1.html#section1.1 The other kind of indexation that is used on a regular basis in PyTables is what you called "slicing" indexation (ex. table[2:300:3], i.e. a la numarray, with the exception that negative values for steps are not supported). This kind of index support is in PyTables from well before 0.9. Hope I have clearified somewhat this issue, Francesc |
From: Francesc A. <fa...@ca...> - 2005-07-18 12:43:27
|
El dl 18 de 07 del 2005 a les 08:07 -0400, en/na travlr va escriure: > Actually I'm aware of the "under the hood indexing", but frankly I > don't use it because it shows nice improvement for some search result > sizes, and is quite latent on others. Yes, that's perfectly possible. This kind of indexing use is mainly aimed for large tables while with small ones (< 10**6 entries) your mileage may vary. > What I'm referring to though, with slice indexing, is being able to > use an array for the index, as numarray does. > file.table.cols.blah[array] where the array can be non-sequential, as > is produced by array=numarray.where(x>y)[0]. Ooops, I see. Well, this is in fact already implemented through the use of Table.readCoordinates() method: file.table.readCoordinates(array, field="blah") However, your suggestion is quite good, and implementing it is a matter of adding a new case in the Column.__getitem__() special method. Something like: [...] elif isinstance(key, numarray): return self.table.readCoordinates(key, self.name) would be enough. > This type of behavior if possible would be less cumbersome than > iterating, and less memory intensive when brought in and bounded to an > actual numarray Type in order to facilitate this behavior. Definitely. I'll try to add such a feature for next release. Cheers, Francesc |
From: Francesc A. <fa...@ca...> - 2005-07-18 18:05:51
Attachments:
Table.py.patch
|
A Monday 18 July 2005 16:39, travlr va escriure: > > However, your suggestion is quite good, and implementing it is a matter > > of adding a new case in the Column.__getitem__() special method. > > Something like: > > > > [...] > > elif isinstance(key, numarray): > > return self.table.readCoordinates(key, self.name <http://self.name>) > > This is terrific. I'd also like to mention two things... setting the attr > vals via file.root.group.table.cols.blah[idx] =3D array ...would also be > great. I believe this syntax congruency (a la numarray) should also be > extended to (py)tables.tables and (py)tables.array objects. Good suggestion. We will see what can we do. Nevertheless, it would be great if you can contribute the code (and docs) yourself. > I got an Error tossing in your patch : > > [...] > elif isinstance(key, numarray.numarraycore.NumArray): > return self.table.readCoordinates(key, self.name <http://self.name>) > [...] Yes. This is a bug in readCoordinates. Try applying the attached patch as well. Cheers, =2D-=20 >0,0< Francesc Altet =A0 =A0 http://www.carabos.com/ V V C=E1rabos Coop. V. =A0=A0Enjoy Data "-" |
From: travlr <vel...@gm...> - 2005-07-19 04:19:03
|
The patch worked fine Francesc, and I'm going to work on the other parts as= =20 you suggested. I'll get back to you soon. Pete On 7/18/05, Francesc Altet <fa...@ca...> wrote: >=20 > A Monday 18 July 2005 16:39, travlr va escriure: > > > However, your suggestion is quite good, and implementing it is a=20 > matter > > > of adding a new case in the Column.__getitem__() special method. > > > Something like: > > > > > > [...] > > > elif isinstance(key, numarray): > > > return self.table.readCoordinates(key, self.name <http://self.name> < > http://self.name>) > > > > This is terrific. I'd also like to mention two things... setting the=20 > attr > > vals via file.root.group.table.cols.blah[idx] =3D array ...would also b= e > > great. I believe this syntax congruency (a la numarray) should also be > > extended to (py)tables.tables and (py)tables.array objects. >=20 > Good suggestion. We will see what can we do. Nevertheless, it would be > great if you can contribute the code (and docs) yourself. >=20 > > I got an Error tossing in your patch : > > > > [...] > > elif isinstance(key, numarray.numarraycore.NumArray): > > return self.table.readCoordinates(key, self.name <http://self.name> < > http://self.name>) > > [...] >=20 > Yes. This is a bug in readCoordinates. Try applying the attached patch > as well. >=20 > Cheers, >=20 > -- > >0,0< Francesc Altet http://www.carabos.com/ > V V C=E1rabos Coop. V. Enjoy Data > "-" >=20 >=20 >=20 > |