pytables-users

[Pytables-users] Indexing Arrays: a la numarray

[travlr <vel...@gm...>]

Hi all,

In a previous thread, I discussed with Fransesc about being able to
incorporate non-sequential indexing arrays (key arrays) in the
following ways:

select and get (or set values) tables.Table.Table <rows>=20
select and get (or set values) tables.Table.Column <items>
select and get (or set values) tables.Array <items>

via: object[key] =3D value(s)

I was able to use the patch Fransesc had provided, and also (with my
extremely limited expertise) a little more including using Numeric
arrays and Python lists for the keys (all getting/setting intended
except enabling the tables.Array[key] functionality). Unfortunately, I
at some point accidently erased the damn file :-(

Today, I went back and got the "selecting and getting" Table <rows> and
Column <elements> for versions 1.0 and 1.1 working again, The attachments
provided are a tables.Table.dif file for each version.
Below is an example of it's usage.

I was not able to do any more especially for the newer versions. I am
hoping for this functionality, as well as being able to set the values
via... keys of a non-sequential index_array:

tables.Table[index_array] =3D value(s); and
tables.Column[index_array] =3D value(s); and
tables.Array[index_array] =3D value(s)

Also Fransesc, two questions:
1) Could the tables.Table.Column also have an .nrows (nelements) attribute?
2) Once bounded, tables.Table used to return as a <class
'numarray.records.RecArray'> object. Now it returns a <class
'tables.nestedrecords.NestedRecArray'> object....Is this necessary?.

Thank You
############# EXAMPLE ###########
import tables as ta
import numarray as na

# test_file is one of my actual files (renamed)
hfile =3D ta.openFile('~/data/h5/test_folder/test_file.h5','a')

# These are pointers to tables.Table=20
# objects (not bounded or returned)
mtable =3D hfile.root.d_050808.d_050808
mcolumn =3D mtable.cols.A1

# mtable is a tables.Table.Table of a=20
# particular day in my database.
print
print 'type(mtable)........ ',type(mtable)
print 'mtable.nrows....... ',mtable.nrows
print 'type(mcolumn).... ',type(mcolumn)

# NOTE to Francesc: Why no nrows (nelements) for columns?
####        print 'mcolumn.nrows... ',mcolumn.nrows

# This will return an non-sequential index=20
# smaller than the table's (or column's)=20
#     original index
index =3D na.where(mcolumn[:]=3D=3D2)[0]
print
print 'index.size....... ', index.size()

# The non-sequential index is applied
# To the tables.Table and and tables.Table.Column
# and returns a numarray.records object (or an array object=20
#    for the column) with the specified indexing
rtable =3D mtable[index]
rcolumn =3D mcolumn[index]

print
print 'type(rtable)..... ',type(rtable)
print 'rtable.size()..... ',rtable.size()
print 'rcolumn.size()..',rcolumn.size()

hfile.close()

############# RESULT ###########
/usr/bin/python -u  "~/scripts/september/testForPytables.py"

type(mtable)........  <class 'tables.Table.Table'>
mtable.nrows.......  105983
type(mcolumn)....  <class 'tables.Table.Column'>

index.size.......  1172

type(rtable).....  <class 'tables.nestedrecords.NestedRecArray'>
rtable.size().....  1172
rcolumn.size().. 1172

Re: [Pytables-users] Indexing Arrays: a la numarray

[Francesc A. <fa...@ca...>]

Hi Travlr,

A Wednesday 07 September 2005 09:32, travlr va escriure:
> Today, I went back and got the "selecting and getting" Table <rows> and
> Column <elements> for versions 1.0 and 1.1 working again, The attachments
> provided are a tables.Table.dif file for each version.
> Below is an example of it's usage.

Thanks for the patches. I think is a good idea to allow retrieving
values using an index array. I'll apply the patches and hopefully they
will appear in the forthcoming PyTables 1.2.

> Also Fransesc, two questions:
> 1) Could the tables.Table.Column also have an .nrows (nelements) attribut=
e?

Well, you have access to the table object via the .table attribute. So
you can use columnObject.table.nrows.

> 2) Once bounded, tables.Table used to return as a <class
> 'numarray.records.RecArray'> object. Now it returns a <class
> 'tables.nestedrecords.NestedRecArray'> object....Is this necessary?.

Strictly speaking no, but from PyTables 1.1 on, Table objects supports
nested fields, so we have been forced to introduce such a
NestedRecArray object. This is by not means a brand new object, but it
is strongly based on RecArray (in fact, it inherits from RecArray).
If, for whatever reason, you prefer a RecArray object, you have a
couple of possibilities:

1.- Use asRecArray() method. See:
http://pytables.sourceforge.net/html-doc/usersguide8.html#sectionB.2
for more details.

2.- If your table has not nested fields, your NestedRecArray will be
flat, and you can access the underlying RecArray object by just
getting the object pointed by the ._flatArray attribute.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Pytables-users] Indexing Arrays: a la numarray

[travlr <vel...@gm...>]

Hi Fransesc, I hope you all had a very nice vaction :-)

On 9/7/05, Francesc Altet <fa...@ca...> wrote:
> Hi Travlr,
>=20
> A Wednesday 07 September 2005 09:32, travlr va escriure:
> > Today, I went back and got the "selecting and getting" Table <rows> and
> > Column <elements> for versions 1.0 and 1.1 working again, The attachmen=
ts
> > provided are a tables.Table.dif file for each version.
> > Below is an example of it's usage.
>=20
> Thanks for the patches. I think is a good idea to allow retrieving
> values using an index array. I'll apply the patches and hopefully they
> will appear in the forthcoming PyTables 1.2.

I hope you're also able to include setting values (and other
functionality mentioned) as well. I was able to implement "setting"
initially, but was not able to reproduce it with these patches.

>=20
> > Also Fransesc, two questions:
> > 1) Could the tables.Table.Column also have an .nrows (nelements) attrib=
ute?
>=20
> Well, you have access to the table object via the .table attribute. So
> you can use columnObject.table.nrows.
>=20
> > 2) Once bounded, tables.Table used to return as a <class
> > 'numarray.records.RecArray'> object. Now it returns a <class
> > 'tables.nestedrecords.NestedRecArray'> object....Is this necessary?.
>=20
> Strictly speaking no, but from PyTables 1.1 on, Table objects supports
> nested fields, so we have been forced to introduce such a
> NestedRecArray object. This is by not means a brand new object, but it
> is strongly based on RecArray (in fact, it inherits from RecArray).
> If, for whatever reason, you prefer a RecArray object, you have a
> couple of possibilities:
>=20
> 1.- Use asRecArray() method. See:
> http://pytables.sourceforge.net/html-doc/usersguide8.html#sectionB.2
> for more details.
>=20
> 2.- If your table has not nested fields, your NestedRecArray will be
> flat, and you can access the underlying RecArray object by just
> getting the object pointed by the ._flatArray attribute.

Actually, I really don't have a direct concern about this.

I wish my understandings (and free time) were greater, because I'd
like to offer what I can. In the near future my time will be more
free, and maybe I can at least do a little something with an
improvement of the documentation... I would have loved to have had an
hyperlinked index back when I was familiarizing myself with pyTables.
When I finish with my immediate responsibilities, I will offer again.

Thanks
Pete

> Cheers,
>=20
> --
> >0,0<   Francesc Altet http://www.carabos.com/
> V   V   C=E1rabos Coop. V. Enjoy Data
>  "-"
>=20
>

Re: [Pytables-users] Indexing Arrays: a la numarray

[travlr <vel...@gm...>]

Here is the diff patch for pytables-1.1.1

The utility is the same as what I provided above for 1.0 and 1.1, but
I've also included  using either of the following object types:

Numarray array
Numeric array
List
Tuple

...(to be used as described in my prior post).

I tried once again (unsuccessfully) to enable pytables.Array[key]
functionality (as well as attempting to enable Table/Column/Array[key]
setting functionality) for non-sequential index-array [keys].


BTW... Numeric question if someone knows..

Python 2.4.1 (#1, Jul 29 2005, 03:50:01)=20
[GCC 3.4.4 (Gentoo 3.4.4, ssp-3.4.4-1.0, pie-8.7.8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import Numeric=20
>>> arr =3D Numeric.array([1,2,3,4,5])
>>> type(arr)
<type 'array'>

>>> isinstance(arr, array)
Traceback (most recent call last):
  File "<input>", line 1, in ?
NameError: name 'array' is not defined

>>> isinstance(arr, type(Numeric.array([])))
True

Why is type(arr) reported as 'array', and yet isinstance() doesn't recogniz=
e it?

Re: [Pytables-users] Indexing Arrays: a la numarray

[Francesc A. <fa...@ca...>]

El dv 16 de 09 del 2005 a les 20:22 -0400, en/na travlr va escriure:
> Here is the diff patch for pytables-1.1.1

Thanks. I'll see to integrate it shortly.

> BTW... Numeric question if someone knows..
> >>> isinstance(arr, array)
> Traceback (most recent call last):
>   File "<input>", line 1, in ?
> NameError: name 'array' is not defined
>=20
> >>> isinstance(arr, type(Numeric.array([])))
> True
>=20
> Why is type(arr) reported as 'array', and yet isinstance() doesn't recogn=
ize it?

The next seems to work:

In [7]:arr=3DNumeric.arange(2)

In [8]:isinstance(arr, Numeric.ArrayType)
Out[8]:True

I'll have to update PyTables accordingly ;-)

Cheers,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"