pytables-users — PyTables users discussion list

[Pytables-users] Reusing In-Kernel row iterators?

[Norbert N. <Nor...@gm...>]

Hi there,

is there any specific reason why in-kernel iterators cannot be used more then 
once:

Doing something like:

 selection = table.where(table.cols.somecol == someval)
 val1 = [ x.val1 for x in selection ] 
 val2 = [ x.val2 for x in selection ] 

gives me a rather intimidating errormessage from somewhere deep inside the HDF 
library. I think, it is a rather common case, so it should definitely work.

Ciao,
Nobbi

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [Pytables-users] Re: natural naming versus getitem/setitem

[Norbert N. <Nor...@gm...>]

Am Montag, 17. Januar 2005 06:12 schrieb russ:
> I would agree that the natural naming can easily get in the way.  For
> example, I tried storing stock data into pytables using the ticker
> symbol as the table name.  However, as it stands you can't use "def" as
> a table name; the "checkNameValidity" function throws a NameError, since
> "def" is a python reserved word.

This problem should already be solved in the newest CVS version of pytables 
(just a little past 0.9.1, I think). A little while ago, it was the first 
step to allow non-pythonic names in the first place. Now it would be the 
second step to make it more convenient to access such names.

> I think a preferable method would use getitem/setitem as the primary
> mechanism to access tree nodes, allowing complete freedom in node name
> specification.   Then for cases where the naming allows, the
> getattr/setattr could be used also, thus preserving the simplicity of
> natural naming for the simple cases.

Exactly. Especially the name-completion feature in certain interactive shells 
(e.g. ipython) really work together very well with natural naming. This will 
never be possible with other access methods.

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [Pytables-users] __getattr__ vs. __getitem__ (Natural naming)

[Norbert N. <Nor...@gm...>]

Am Sonntag, 16. Januar 2005 21:15 schrieb Ivan Vilata i Balaguer:
> On Sun, Jan 16, 2005 at 06:00:50PM +0100, Norbert Nemec wrote:
> > > Imagine issuing:
> > >   >>> # h5file.root.detector.readout <=>
> > >   >>> tbl = h5file.root['detector/readout']
> >
> > or
> >  tbl = h5file.root['detector']['readout']
> >
> > Anyhow: I don't see much gain here. Just the question whether you do the
> > '/' parsing or not. Don't have much of an opinion there.
>
> It would simplify specifying a path (even leading to attributes if
> necessary) and getting the object.  However, doing so would make
> dictionary key completion in some IDEs useless, so maybe it was not a
> good idea in the end.

The dictionary should, of course, contain only the names of immediate 
children. It would be __getitem__ that does the '/' parsing and looks up the 
individual steps. dictionary key completion is not an issue.

Actually: __dict__ can even contain non-pythonic names. You just cannot query 
them by natural naming, but ipython even shows these names in key completion. 
Anyhow: it certainly would be confusing to have non-pythonic names in a 
__dict__ ...


-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [Pytables-users] Problems with chunked datasets

[Francesc A. <fa...@ca...>]

Hi Antonio,

Can you send me privately both files so as to see what's going on with them?

Cheers,

A Diumenge 16 Gener 2005 16:19, Antonio Valentino va escriure:
> hi,
> I'm not an expert user and I'm having some problems trying to open an
> hdf5 file containing a chunked dataset.
> Here it is some infos
>=20
> Python 2.3.4
> [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
> numarray v. 1.1.1
> tables v. 0.9.1
> hdf5   v. 1.6.3-patch
>=20
> this is the test program
>=20
> # BEGIN file test-uchar.py
> import tables
> h5file =3D tables.openFile('data.h5')
> print h5file
> # END file test-uchar.py
>=20
> an this is the data
>=20
> # chunk (128x128x2)
>=20
> [antonio@m6n h5]$ h5dump -A data.h5
> HDF5 "data.h5" {
> GROUP "/" {
>    DATASET "ComplexUCharArray" {
>       DATATYPE  H5T_STD_U8LE
>       DATASPACE  SIMPLE { ( 200, 150, 2 ) / ( 200, 150, 2 ) }
>    }
> }
> }
>=20
> When i run the test program i get a segfault
>=20
> [antonio@m6n h5]$ python test-uchar.py
> /usr/lib/python2.3/site-packages/tables/File.py:192: UserWarning:
> 'data.h5' does exist, is an HDF5 file, but has not a PyTables format.
> Trying toguess what's there using HDF5 metadata. I can't promise you
> getting the correctobjects, but I will do my best!.
>   path, UserWarning)
> Segmentation fault
>=20
>=20
> If I try it with a *non* chunked dataset ...
>=20
> [antonio@m6n h5]$ python test-uchar.py
> /usr/lib/python2.3/site-packages/tables/File.py:192: UserWarning:
> 'data.h5' does exist, is an HDF5 file, but has not a PyTables format.
> Trying to guess what's there using HDF5 metadata. I can't promise you
> getting the correct objects, but I will do my best!.
>   path, UserWarning)
> Traceback (most recent call last):
>   File "test-uchar.py", line 6, in ?
>     print h5file
>   File "/usr/lib/python2.3/site-packages/tables/File.py", line 1000, in
> __str__
>     astring +=3D str(leaf) + '\n'
>   File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 472, in
> __str__
>     title =3D self.attrs.TITLE
>   File "/usr/lib/python2.3/site-packages/tables/AttributeSet.py", line
> 166, in __getattr__
>     raise AttributeError, \
>=20
> [SNIP]
>=20
>   File "/usr/lib/python2.3/site-packages/tables/AttributeSet.py", line
> 166, in __getattr__
>     raise AttributeError, \
>   File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 472, in
> __str__
>     title =3D self.attrs.TITLE
>   File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 189, in
> _get_attrs
>     return AttributeSet(self)
> RuntimeError: maximum recursion depth exceeded
>=20
>=20
> in this case the file seems to be correctly opened but some problem is
> met in the print statement.
>=20
> antonio
>=20
>=20

=2D-=20
>OO< =A0 Francesc Altet    ||  http://www.carabos.com/
V =A0V =A0 Carabos Coop. V.  ||  Who is your data daddy? PyTables
 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Hi Francesc,

--- Francesc Altet <fa...@ca...> wrote:

> A Dijous 13 Gener 2005 21:42, kevin lester va
> escriure:
> > Success :-)
> 
> Glad to hear that
> 
> > If PATH == PYTHONPATH, then yes, I do have
> > zlib.dll(along with the others) in the
> /WINDOWS/SYSTEM
> > folder with the PATH pointing to it.
> 
> No, PYTHONPATH should not point to the zlib.dll
> directory. Just make sure
> that the zlib.dll is in a directory listed in your
> PATH environment
> variable. That should be enough.

I have the PATH environment variable pointing to
SYSTEM32, yet I still recieve this message.
> 
> > >>> Running 'C:\H5\klester.py' ...
> > C:\Python24\lib\site-packages\tables\Leaf.py:90:
> > UserWarning: zlib compression library is not
> > available. Using zlib instead!.
> >   warnings.warn( \
> > Time for standard query--> 0.0620000362396
> > Time for inkernel query--> 0.0469999313354
> > Time for indexed query--> 0.0160000324249
> 
> Ooops, you are getting a factor 3 for speed-up
> compared with inkernel and a
> factor of 4 against standard. That's quite a lot. In
> my laptop (Pentium 4 @
> 2 GHz, python 2.3.4), I'm getting the next figures:
> 
> $ python klester.py
> Time for standard query--> 0.147189855576
> Time for inkernel query--> 0.0714249610901
> Time for indexed query--> 0.0375471115112
> 
> where the speed-up is clearly inferior. Just out of
> curiosity, which
> processor, speed, and python version are you using?
> 
I have noticed differences for each; and not one is
always better than the other. I don't have the time
(just now) to run tests to provide to you, but will
happpily do so a little later this week.

Kevin

> Cheers,
> 
> -- 
> >OO<   Francesc Altet    ||  http://www.carabos.com/
> V  V   Carabos Coop. V.  ||  Who is your data daddy?
> PyTables
>  ""
> 
> 



		
__________________________________ 
Do you Yahoo!? 
Read only the mail you want - Yahoo! Mail SpamGuard. 
http://promotions.yahoo.com/new_mail 

[Pytables-users] Re: natural naming versus getitem/setitem

[russ <ru...@ag...>]

I would agree that the natural naming can easily get in the way.  For 
example, I tried storing stock data into pytables using the ticker 
symbol as the table name.  However, as it stands you can't use "def" as 
a table name; the "checkNameValidity" function throws a NameError, since 
"def" is a python reserved word.  To me this is reminiscent of DOS 
preventing the use of filenames "prn" or "con".  It is easy to find 
workarounds, but ugly since you have to deal with a lot of special cases. 

I think a preferable method would use getitem/setitem as the primary 
mechanism to access tree nodes, allowing complete freedom in node name 
specification.   Then for cases where the naming allows, the 
getattr/setattr could be used also, thus preserving the simplicity of 
natural naming for the simple cases.

Cheers, -Russ


>I know that natural naming is seen as one of the important features of 
>pytables. For interactive use, this certainly is true. For me - mainly using 
>scripts - it gets in the way quite often. I'm using getattr/setattr more 
>often than natural naming, since the attribute names are quite often handled 
>as strings.

Re: [Pytables-users] __getattr__ vs. __getitem__ (Natural naming)

[Ivan V. i B. <iv...@ca...>]

On Sun, Jan 16, 2005 at 06:00:50PM +0100, Norbert Nemec wrote:
> Am Sonntag, 16. Januar 2005 16:35 schrieb Ivan Vilata i Balaguer:
> > [...]
> > Please note I that intentionally left out=20
> > __set{item,attr}__ because this is a greater change.  File.create*()
> > calls would be changed to direct constructor invocation for leaves and
> > groups, a thing which I definetly think Right and perfectly plausible
> > (but I might be wrong).
>=20
> I don't quite understand: it already is possible to do
>  mygroup.somename =3D Array(somevalue)
> to create an array. Where is the "greater change"? Actually, I never like=
d the=20
> File.create*() method to create elements.

Oops, you're certain!  I bypassed the fact that Group.__setattr__ takes
care of putting the object in the tree.  Much easier, then!

[...]
> > Imagine issuing:=20
> >   >>> # h5file.root.detector.readout <=3D>
> >   >>> tbl =3D h5file.root['detector/readout']
>=20
> or
>  tbl =3D h5file.root['detector']['readout']
>=20
> Anyhow: I don't see much gain here. Just the question whether you do the =
'/'=20
> parsing or not. Don't have much of an opinion there.

It would simplify specifying a path (even leading to attributes if
necessary) and getting the object.  However, doing so would make
dictionary key completion in some IDEs useless, so maybe it was not a
good idea in the end.

> >   >>> # h5file.root.column['name column'].attrs.temperature <=3D>
> >   >>> attr =3D h5file.root['column/name column/attrs/temperature']
>=20
> I have no idea what that's supposed to mean...

It means that
  >>> h5file.root.column['name column'].attrs.temperature
and
  >>> h5file.root['column/name column/attrs/temperature']
are equivalent.  They both get the 'temperature' attribute from the
'name column' Leaf, in the 'column' group, in the 'root' group.

    Anyway, I still think that allowing path notation leading down to
attributes could be a good idea, not by using a node's __???item__ or
__???attr__, but a File.???Item method that might in this way replace
File's getNode and ???AttrNode methods.  For instance:

  >>> h5file.setItem('/column/name column/attrs/temperature', 25)

--=20
Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
       C=E1rabos Coop. V.  V  V   Enjoy Data
                          ""

Re: [Pytables-users] __getattr__ vs. __getitem__ (Natural naming)

[Norbert N. <Nor...@gm...>]

Am Sonntag, 16. Januar 2005 16:35 schrieb Ivan Vilata i Balaguer:
> [...]
> Please note I that intentionally left out 
> __set{item,attr}__ because this is a greater change.  File.create*()
> calls would be changed to direct constructor invocation for leaves and
> groups, a thing which I definetly think Right and perfectly plausible
> (but I might be wrong).

I don't quite understand: it already is possible to do
 mygroup.somename = Array(somevalue)
to create an array. Where is the "greater change"? Actually, I never liked the 
File.create*() method to create elements.

> [...]
> (but they would be kept for compatibility).

I would recommend spicing as much as possible with DeprecatedWarning messages. 
Offering too many alternatives is more confusing than helpful.

> Imagine issuing: 
>   >>> # h5file.root.detector.readout <=>
>   >>> tbl = h5file.root['detector/readout']

or
 tbl = h5file.root['detector']['readout']

Anyhow: I don't see much gain here. Just the question whether you do the '/' 
parsing or not. Don't have much of an opinion there.

>   >>> # h5file.root.column['name column'].attrs.temperature <=>
>   >>> attr = h5file.root['column/name column/attrs/temperature']

I have no idea what that's supposed to mean...

>     I think this should be quite easy to implement and would make
> building pathnames and traversing them a pleasure, keeping interactive
> behaviour the same.  Maybe even File.__???{item,attr}__ could be
> tailored to defer to the root group object invocations using an item /
> attribute starting with '/', so that using h5file['/detector/readout']
> would be possible...



-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [Pytables-users] __getattr__ vs. __getitem__ (Natural naming)

[Ivan V. i B. <iv...@ca...>]

On Sun, Jan 16, 2005 at 02:30:16PM +0100, Norbert Nemec wrote:
[...]
> How about generally defining __???item__ in parallel to the corresponding=
=20
> __???attr__ ?
[...]

Hi, Norbert.  In fact, since we discussed the possibility of using
non-natural identifiers I clearly saw that __{get,del}item__ and
__{get,del}attr__ should be interchangeable, to allow accessing members
with non-natural names.  Please note I that intentionally left out
__set{item,attr}__ because this is a greater change.  File.create*()
calls would be changed to direct constructor invocation for leaves and
groups, a thing which I definetly think Right and perfectly plausible
(but I might be wrong).

> In a further step, it might even be a good idea, to make __???item__ the=
=20
> standard way that is used everywhere within the pytables library and=20
> __???attr__ just an additional option falling back to __???item__
[...]

Well, that would make code a lot clearer and uniform.

    It seems to be that it would also be quite cool to support entire
node and attribute paths in the __???item__ interface.  This would make
lots of File methods unnecessary (but they would be kept for
compatibility).  Imagine issuing:
 =20
  >>> # h5file.root.detector.readout <=3D>
  >>> tbl =3D h5file.root['detector/readout']
  >>> # h5file.root.column['name column'].attrs.temperature <=3D>
  >>> attr =3D h5file.root['column/name column/attrs/temperature']

    I think this should be quite easy to implement and would make
building pathnames and traversing them a pleasure, keeping interactive
behaviour the same.  Maybe even File.__???{item,attr}__ could be
tailored to defer to the root group object invocations using an item /
attribute starting with '/', so that using h5file['/detector/readout']
would be possible...

Regards,
    Ivan

import disclaimer
--=20
Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
       C=E1rabos Coop. V.  V  V   Enjoy Data
                          ""

[Pytables-users] Problems with chunked datasets

[Antonio V. <val...@co...>]

hi,
I'm not an expert user and I'm having some problems trying to open an
hdf5 file containing a chunked dataset.
Here it is some infos

Python 2.3.4
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
numarray v. 1.1.1
tables v. 0.9.1
hdf5   v. 1.6.3-patch

this is the test program

# BEGIN file test-uchar.py
import tables
h5file = tables.openFile('data.h5')
print h5file
# END file test-uchar.py

an this is the data

# chunk (128x128x2)

[antonio@m6n h5]$ h5dump -A data.h5
HDF5 "data.h5" {
GROUP "/" {
   DATASET "ComplexUCharArray" {
      DATATYPE  H5T_STD_U8LE
      DATASPACE  SIMPLE { ( 200, 150, 2 ) / ( 200, 150, 2 ) }
   }
}
}

When i run the test program i get a segfault

[antonio@m6n h5]$ python test-uchar.py
/usr/lib/python2.3/site-packages/tables/File.py:192: UserWarning:
'data.h5' does exist, is an HDF5 file, but has not a PyTables format.
Trying toguess what's there using HDF5 metadata. I can't promise you
getting the correctobjects, but I will do my best!.
  path, UserWarning)
Segmentation fault


If I try it with a *non* chunked dataset ...

[antonio@m6n h5]$ python test-uchar.py
/usr/lib/python2.3/site-packages/tables/File.py:192: UserWarning:
'data.h5' does exist, is an HDF5 file, but has not a PyTables format.
Trying to guess what's there using HDF5 metadata. I can't promise you
getting the correct objects, but I will do my best!.
  path, UserWarning)
Traceback (most recent call last):
  File "test-uchar.py", line 6, in ?
    print h5file
  File "/usr/lib/python2.3/site-packages/tables/File.py", line 1000, in
__str__
    astring += str(leaf) + '\n'
  File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 472, in
__str__
    title = self.attrs.TITLE
  File "/usr/lib/python2.3/site-packages/tables/AttributeSet.py", line
166, in __getattr__
    raise AttributeError, \

[SNIP]

  File "/usr/lib/python2.3/site-packages/tables/AttributeSet.py", line
166, in __getattr__
    raise AttributeError, \
  File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 472, in
__str__
    title = self.attrs.TITLE
  File "/usr/lib/python2.3/site-packages/tables/Leaf.py", line 189, in
_get_attrs
    return AttributeSet(self)
RuntimeError: maximum recursion depth exceeded


in this case the file seems to be correctly opened but some problem is
met in the print statement.

antonio


-- 
Antonio Valentino
Consorzio Innova S.r.l.
via della Scienza - Zona Paip I
75100 Matera (MT) Italy
Tel.:      +39 0835 309180
Fax:       +39 0835 264705
Home Page: www.consorzio-innova.it
Email:     val...@co...

Re: [Pytables-users] Why have column descriptors to be given as classes!?!

[Norbert N. <Nor...@gm...>]

Stupid me! It's there already...



Am Sonntag, 16. Januar 2005 14:12 schrieb Norbert Nemec:
> Hi there,
>
> just considered using Tables instead of Arrays for my purpose. Now I really
> had to fight the fact that column descriptors have to be given as classes.
>
> Problem is: defining a class the regular way means, that at the time of
> writing a python script, you have to know the names and types of all
> columns. If you get the names of the columns as strings from some other
> source, this becomes considerably harder.
>
> The hack I've come up with for the moment looks like:
>
> -----------------------------------------------------------
> #!/usr/bin/env python
>
> from tables import *
>
> def descfromdict(__d__):
>     class res(IsDescription):
>         for __k__ in __d__:
>             locals()[__k__] = __d__[__k__]
>     return res
>
> file = openFile('tryout.h5','w')
>
> mydict={}
> mydict['somecolumn'] = FloatCol()
> mydict['transmission'] = FloatCol()
> mydict['statistics'] = UInt16Col()
>
> file.root.mytable = Table(descfromdict(mydict))
>
> file.close()
> ---------------------------------------------
>
> similarly, it would be possible to us an object instead of a class, which
> would even allow natural naming:
>
> ------------------------------------------------------
> #!/usr/bin/env python
>
> from tables import *
>
> def descfromobj(__d__):
>     class res(IsDescription):
>         for __k__ in __d__.__dict__:
>             locals()[__k__] = getattr(__d__,__k__)
>     return res
>
> class MyDescription:
>     def __setitem__(self,key,value):
>         assert type(key) is str
>         assert isinstance(value,Col)
>         setattr(self,key,value)
>
>
>
> file = openFile('tryout.h5','w')
>
> myobj = MyDescription()
> myobj.somecolumn = FloatCol()
> myobj['anothercolumn'] = FloatCol()
>
> file.root.mytable = Table(descfromobj(myobj))
>
> file.close()
> --------------------------------------------
>
> Both work, but it is really ugly if you have to find such a solution
> yourself.
>
> In the least, I would suggest to offer these options as an alternative way
> to pass descriptions. But I would think, that it would simplify things a
> lot, if one of these methods (or something even better) would become the
> preferred and documented way of passing a descriptor and the old method of
> defining and passing a class be obsoleted. I think, even the internals of
> pytables might be simplified by it...
>
> Ciao,
> Norbert

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

[Pytables-users] __getattr__ vs. __getitem__ (Natural naming)

[Norbert N. <Nor...@gm...>]

Hi there,

an idea that I have carried around for some time now:

I know that natural naming is seen as one of the important features of 
pytables. For interactive use, this certainly is true. For me - mainly using 
scripts - it gets in the way quite often. I'm using getattr/setattr more 
often than natural naming, since the attribute names are quite often handled 
as strings.

How about generally defining __???item__ in parallel to the corresponding 
__???attr__ ?

 mygroup["mynode"] = Node(...)
 print mygroup["mynode"]
 del mygroup["mynode"]
would be identical to
 mygroup.mynode = Node(...)
 print mygroup.mynode
 del mygroup.mynode

in case nodename == "mynode"

 mygroup[nodename] = Node(...)
 print mygroup[nodename]
 del mygroup[nodename]
certainly is nicer than
 setattr(mygroup,nodename,Node(...))
 print getattr(mygroup,nodename)
 delattr(mygroup,nodename)

especially, since you are not dealing with an HDF5-Attribute (!!!)

In a further step, it might even be a good idea, to make __???item__ the 
standard way that is used everywhere within the pytables library and 
__???attr__ just an additional option falling back to __???item__

This would probably beautify the library code quite a bit (getattr/setattr are 
called quite often) but even more important: it would releave the name 
conflict of 'Python-attributes' vs. 'HDF5-Attributes'. Instead of thinking of 
'Python-attributes', one would think of 'items' which may be either 
'Attributes' or 'Nodes'.

Greetings,
Norbert

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

[Pytables-users] Why have column descriptors to be given as classes!?!

[Norbert N. <Nor...@gm...>]

Hi there,

just considered using Tables instead of Arrays for my purpose. Now I really 
had to fight the fact that column descriptors have to be given as classes.

Problem is: defining a class the regular way means, that at the time of 
writing a python script, you have to know the names and types of all columns. 
If you get the names of the columns as strings from some other source, this 
becomes considerably harder.

The hack I've come up with for the moment looks like:

-----------------------------------------------------------
#!/usr/bin/env python

from tables import *

def descfromdict(__d__):
    class res(IsDescription):
        for __k__ in __d__:
            locals()[__k__] = __d__[__k__]
    return res

file = openFile('tryout.h5','w')

mydict={}
mydict['somecolumn'] = FloatCol()
mydict['transmission'] = FloatCol()
mydict['statistics'] = UInt16Col()

file.root.mytable = Table(descfromdict(mydict))

file.close()
---------------------------------------------

similarly, it would be possible to us an object instead of a class, which 
would even allow natural naming:

------------------------------------------------------
#!/usr/bin/env python

from tables import *

def descfromobj(__d__):
    class res(IsDescription):
        for __k__ in __d__.__dict__:
            locals()[__k__] = getattr(__d__,__k__)
    return res

class MyDescription:
    def __setitem__(self,key,value):
        assert type(key) is str
        assert isinstance(value,Col)
        setattr(self,key,value)



file = openFile('tryout.h5','w')

myobj = MyDescription()
myobj.somecolumn = FloatCol()
myobj['anothercolumn'] = FloatCol()

file.root.mytable = Table(descfromobj(myobj))

file.close()
--------------------------------------------

Both work, but it is really ugly if you have to find such a solution yourself. 

In the least, I would suggest to offer these options as an alternative way to 
pass descriptions. But I would think, that it would simplify things a lot, if 
one of these methods (or something even better) would become the preferred 
and documented way of passing a descriptor and the old method of defining and 
passing a class be obsoleted. I think, even the internals of pytables might 
be simplified by it...

Ciao,
Norbert

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

A Dijous 13 Gener 2005 21:42, kevin lester va escriure:
> Success :-)

Glad to hear that

> If PATH =3D=3D PYTHONPATH, then yes, I do have
> zlib.dll(along with the others) in the /WINDOWS/SYSTEM
> folder with the PATH pointing to it.

No, PYTHONPATH should not point to the zlib.dll directory. Just make sure
that the zlib.dll is in a directory listed in your PATH environment
variable. That should be enough.

> >>> Running 'C:\H5\klester.py' ...
> C:\Python24\lib\site-packages\tables\Leaf.py:90:
> UserWarning: zlib compression library is not
> available. Using zlib instead!.
>   warnings.warn( \
> Time for standard query--> 0.0620000362396
> Time for inkernel query--> 0.0469999313354
> Time for indexed query--> 0.0160000324249

Ooops, you are getting a factor 3 for speed-up compared with inkernel and a
factor of 4 against standard. That's quite a lot. In my laptop (Pentium 4 @
2 GHz, python 2.3.4), I'm getting the next figures:

$ python klester.py
Time for standard query--> 0.147189855576
Time for inkernel query--> 0.0714249610901
Time for indexed query--> 0.0375471115112

where the speed-up is clearly inferior. Just out of curiosity, which
processor, speed, and python version are you using?

Cheers,

=2D-=20
>OO< =A0 Francesc Altet    ||  http://www.carabos.com/
V =A0V =A0 Carabos Coop. V.  ||  Who is your data daddy? PyTables
 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Success :-)
Also, you had asked me:
 > Mmm, are you sure that zlib DLL is in the PATH? If
 > it is, you should not get
 > this warning.

If PATH == PYTHONPATH, then yes, I do have
zlib.dll(along with the others) in the /WINDOWS/SYSTEM
folder with the PATH pointing to it.
Thanks Francesc
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> Running 'C:\H5\klester.py' ...
C:\Python24\lib\site-packages\tables\Leaf.py:90:
UserWarning: zlib compression library is not
available. Using zlib instead!.
  warnings.warn( \
Time for standard query--> 0.0620000362396
Time for inkernel query--> 0.0469999313354
Time for indexed query--> 0.0160000324249

H1 was declared indexed in isDescription...
table.cols.H1.index --> Index(100000, shape=(5,
20000), chunksize=2000)
table.indexed: --> 1
table.colindexed:
{'V1': 1, 'L1': 1, 'UD': 1, 'H2': 0, 'LO': 1, 'S1': 1,
'H1': 1, 'TV': 1, 'A1': 1, 'AP': 1, 'AS': 1, 'HI': 1,
'BP': 1, 'BS': 1, 'M1': 1, 'DT': 1}
Script '<source>' returned exit code 0
>>> 
--- Francesc Altet <fa...@ca...> wrote:

> Ooops,
> 
> Replace:
> 
>                         objgroup.append(name)
>    
> by
> 
>                        
> objgroup._v_indices.append(name)
> Hope this time works
> 
> 
> A Dijous 13 Gener 2005 16:04, kevin lester va
> escriure:
> > Francesc,
> > 
> > Regarding Patch Supplied...
> > 
> > Exception raised:
> > 
> >   File
> > "C:\Python24\lib\site-packages\tables\Group.py",
> line
> > 190, in _g_openFile
> >     objgroup.append(name)
> >   File
> > "C:\Python24\lib\site-packages\tables\Group.py",
> line
> > 528, in __getattr__
> >     if not self._v_file.isopen:
> > AttributeError: 'File' object has no attribute
> > 'isopen'
> > Exception raised while running script  <source>
> > 
> > >>> 
> > --- Francesc Altet <fa...@ca...> wrote:
> > 
> > > A Dijous 13 Gener 2005 13:00, vareu escriure:
> > > > I'm confused Francesc:
> > > 
> > > Me too. Well, it seems like you have encountered
> yet
> > > another bug. Try this
> > > patch:
> > > 
> > > --- Group.py.orig       2005-01-13
> > > 13:47:37.000000000 +0100
> > > +++ Group.py    2005-01-13 13:48:03.000000000
> +0100
> > > @@ -186,7 +186,7 @@
> > >                      # and their names will be
> > > append to a list
> > >                      if classname == "INDEX":
> > >                          # only INDEX names will
> be
> > > appended
> > > -                       
> > > self._v_indices.append(name)
> > > +                        objgroup.append(name)
> > >              for name in leaves:
> > >                 
> objleaf=objgroup._g_getLeaf(name)
> > >                  if objleaf <> None:
> > > 
> > > and tell me how it goes.
> > > 
> > > > 
> > > > >>> Running 'C:\H5\klester.py' ...
> > > >
> C:\Python24\lib\site-packages\tables\Leaf.py:90:
> > > > UserWarning: zlib compression library is not
> > > > available. Using zlib instead!.
> > > >   warnings.warn( \
> > > 
> > > Mmm, are you sure that zlib DLL is in the PATH?
> If
> > > it is, you should not get
> > > this warning.
> > 
> > If PATH == PYTHONPATH, then yes I do have
> > zlib.dll(along with the others) in the
> /WINDOWS/SYSTEM
> > folder with the PATH pointing to it.
> > 
> > Kevin :)
> > > 
> > > -- 
> > > >OO<   Francesc Altet    || 
> http://www.carabos.com/
> > > V  V   Carabos Coop. V.  ||  Who is your data
> daddy?
> > > PyTables
> > >  ""
> > > 
> > > 
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around 
> > http://mail.yahoo.com 
> > 
> > 
> >
>
-------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> > It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Pytables-users mailing list
> > Pyt...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> > 
> 
> -- 
> >OO<   Francesc Altet    ||  http://www.carabos.com/
> V  V   Carabos Coop. V.  ||  Who is your data daddy?
> PyTables
>  ""
> 
> 
> 
>
-------------------------------------------------------
> The SF.Net email is sponsored by: Beat the
> post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> 



	
		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - You care about security. So do we. 
http://promotions.yahoo.com/new_mail

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

Ooops,

Replace:

                        objgroup.append(name)
  =20
by

                        objgroup._v_indices.append(name)
Hope this time works


A Dijous 13 Gener 2005 16:04, kevin lester va escriure:
> Francesc,
>=20
> Regarding Patch Supplied...
>=20
> Exception raised:
>=20
>   File
> "C:\Python24\lib\site-packages\tables\Group.py", line
> 190, in _g_openFile
>     objgroup.append(name)
>   File
> "C:\Python24\lib\site-packages\tables\Group.py", line
> 528, in __getattr__
>     if not self._v_file.isopen:
> AttributeError: 'File' object has no attribute
> 'isopen'
> Exception raised while running script  <source>
>=20
> >>>=20
> --- Francesc Altet <fa...@ca...> wrote:
>=20
> > A Dijous 13 Gener 2005 13:00, vareu escriure:
> > > I'm confused Francesc:
> >=20
> > Me too. Well, it seems like you have encountered yet
> > another bug. Try this
> > patch:
> >=20
> > --- Group.py.orig       2005-01-13
> > 13:47:37.000000000 +0100
> > +++ Group.py    2005-01-13 13:48:03.000000000 +0100
> > @@ -186,7 +186,7 @@
> >                      # and their names will be
> > append to a list
> >                      if classname =3D=3D "INDEX":
> >                          # only INDEX names will be
> > appended
> > -                      =20
> > self._v_indices.append(name)
> > +                        objgroup.append(name)
> >              for name in leaves:
> >                  objleaf=3Dobjgroup._g_getLeaf(name)
> >                  if objleaf <> None:
> >=20
> > and tell me how it goes.
> >=20
> > >=20
> > > >>> Running 'C:\H5\klester.py' ...
> > > C:\Python24\lib\site-packages\tables\Leaf.py:90:
> > > UserWarning: zlib compression library is not
> > > available. Using zlib instead!.
> > >   warnings.warn( \
> >=20
> > Mmm, are you sure that zlib DLL is in the PATH? If
> > it is, you should not get
> > this warning.
>=20
> If PATH =3D=3D PYTHONPATH, then yes I do have
> zlib.dll(along with the others) in the /WINDOWS/SYSTEM
> folder with the PATH pointing to it.
>=20
> Kevin :)
> >=20
> > --=20
> > >OO<   Francesc Altet    ||  http://www.carabos.com/
> > V  V   Carabos Coop. V.  ||  Who is your data daddy?
> > PyTables
> >  ""
> >=20
> >=20
>=20
>=20
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around=20
> http://mail.yahoo.com=20
>=20
>=20
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>=20

=2D-=20
>OO< =A0 Francesc Altet    ||  http://www.carabos.com/
V =A0V =A0 Carabos Coop. V.  ||  Who is your data daddy? PyTables
 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Francesc,

Regarding Patch Supplied...

Exception raised:

  File
"C:\Python24\lib\site-packages\tables\Group.py", line
190, in _g_openFile
    objgroup.append(name)
  File
"C:\Python24\lib\site-packages\tables\Group.py", line
528, in __getattr__
    if not self._v_file.isopen:
AttributeError: 'File' object has no attribute
'isopen'
Exception raised while running script  <source>

>>> 
--- Francesc Altet <fa...@ca...> wrote:

> A Dijous 13 Gener 2005 13:00, vareu escriure:
> > I'm confused Francesc:
> 
> Me too. Well, it seems like you have encountered yet
> another bug. Try this
> patch:
> 
> --- Group.py.orig       2005-01-13
> 13:47:37.000000000 +0100
> +++ Group.py    2005-01-13 13:48:03.000000000 +0100
> @@ -186,7 +186,7 @@
>                      # and their names will be
> append to a list
>                      if classname == "INDEX":
>                          # only INDEX names will be
> appended
> -                       
> self._v_indices.append(name)
> +                        objgroup.append(name)
>              for name in leaves:
>                  objleaf=objgroup._g_getLeaf(name)
>                  if objleaf <> None:
> 
> and tell me how it goes.
> 
> > 
> > >>> Running 'C:\H5\klester.py' ...
> > C:\Python24\lib\site-packages\tables\Leaf.py:90:
> > UserWarning: zlib compression library is not
> > available. Using zlib instead!.
> >   warnings.warn( \
> 
> Mmm, are you sure that zlib DLL is in the PATH? If
> it is, you should not get
> this warning.

If PATH == PYTHONPATH, then yes I do have
zlib.dll(along with the others) in the /WINDOWS/SYSTEM
folder with the PATH pointing to it.

Kevin :)
> 
> -- 
> >OO<   Francesc Altet    ||  http://www.carabos.com/
> V  V   Carabos Coop. V.  ||  Who is your data daddy?
> PyTables
>  ""
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

A Dijous 13 Gener 2005 09:49, kevin lester va escriure:
> Sorry, I spoke to soon...
> The indexed=3D1 or the .createIndex() is still not being
> created. With the declaration (indexed=3D1), execution
> "tables" the data without acknowledging that indexing
> was not performed. With .createIndex() I'm getting a
> RuntimeError.
>=20
>   File "hdf5Extension.pyx", line 1411, in
> hdf5Extension.Group._g_createGroup
> RuntimeError: Can't create the group _i_raw_A1.

That's strange. Look at the code in the attachment based on your example as
a hint on how to use indexation; it works well on my laptop.

Running this code for a table with 100,000 rows, I've got the following:

$ python klester.py
Time for standard query--> 0.141028881073
Time for inkernel query--> 0.0666921138763
Time for indexed query--> 0.0650899410248

As you can see, there is no much point on indexing columns when the number
of rows is less than a million (see
http://pytables.sourceforge.net/doc/SciPy04.pdf, page 24). For one million
and up, indexation in pytables 0.9 is starting to be competitive (specially
if the index is already in the filesystem cache).

> Also Francesc, my understanding is, that this is the
> answer to "index persistence"; when an original index
> of an array is preserved after having performed "cuts"
> and other various "operations" on the data. IOW,
> unlike natural numarray characteristics which requires
> extra effort to maintain an array's original index
> references. Is this correct? If so, I have not yet
> figured out how I access it.

Sorry, I'm afraid I don't quite understand what you mean. Can you develop
this further?

Cheers,

=2D-=20
>OO< =A0 Francesc Altet    ||  http://www.carabos.com/
V =A0V =A0 Carabos Coop. V.  ||  Who is your data daddy? PyTables
 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

Hi,

SZLIB is no necessary for HDF5 nor PyTables and besides, it is not directly
supported by PyTables (i.e. PyTable can't create files compressed with SZLIB
although it can read files compressed with SZLIB as long as the underlying
HDF5 library has support for that).

On another hand, if you install SZLIB (i.e. you drop the DLL in a directory
in the PATH) that should not hurt either.

Cheers,

A Dijous 13 Gener 2005 07:26, kevin lester va escriure:
> Hi Francesc,
>=20
> First, let me say that the field variable change did
> allow indexing.(thanks).
> =20
> After My last email, I re-Installed all packages. At
> first nothing worked. IDE's and DOS command executions
> just crashed immediately. After Fooling around for
> quite some time, I got things to work by not including
> the SZLIB files(and .dll).=20
>=20
> But... PythonWin (which is the IDE I was using before
> all this crashing) is still crashing. It was
> performing before, but did need to be re-booted after
> nearly each file upload. I don't necessarily care to
> use it. Hopefully I'll find the stability in the IDE
> I'm going to try to work with now (SPE).
>=20
> Not being as computer savvy yet as I'm going to be
> ;-), I had trouble with knowing which of the
> optionable versions of szlib I needed. But, is szlib
> even necessary??? I did'nt get that impression from
> HDF5's instructions.
>=20
> At this point I was also able to utilize 'lzo' & 'ucl'
> for compression. =20
>=20
> Anyway I'm going to see how things progress and I'll
> give a report later.
>=20
> Many, many thanks,
> Kevin
>=20
>=20
> --- Francesc Altet <fa...@ca...> wrote:
>=20
> > A Dimecres 12 Gener 2005 13:11, kevin lester va
> > escriure:
> > > I'm having problems with the zlib,lzo,ucl
> > compression
> > > libraries being available to pyTables. I planted
> > the
> > > .dlls in the /WINDOWS/SYSTEM folder and have
> > > PYTHONPATH directed there as mentioned in the
> > > installation instructions.
> >=20
> > Well, that seems good. What kind of problems are you
> > having?
> >=20
> > > I included the code below for the declaritve. The
> > > .createIndex() version was just as was shown in
> > 6.22
> > > of the tutorial=20
> > >=20
> > > >>> indexrows =3D table.cols.var1.createIndex() <<<
> >=20
> > After looking at your script, I can see that you
> > have been bitten by a known
> > bug, that prevents to name a column with a name that
> > contains another column
> > name as a substring. While I release a pytables
> > version with a cure to this,
> > you may try the next column declaration that is free
> > of this problem:
> >=20
> > class ES(IsDescription):
> >     H1       =3D Int8Col(indexed=3D1)
> >     M1       =3D Int8Col(indexed=3D1)
> >     S1       =3D Int8Col(indexed=3D1)
> >     BS      =3D Int16Col(indexed=3D1)
> >     BP      =3D Float32Col(indexed=3D1)
> >     AP      =3D Float32Col(indexed=3D1)
> >     AS      =3D Int16Col(indexed=3D1)
> >     L1       =3D Float32Col(indexed=3D1)
> >     V1       =3D Int16Col(indexed=3D1)
> >     A1       =3D Int8Col(indexed=3D1)
> >     UD      =3D BoolCol(indexed=3D1)
> >     TV      =3D Int64Col(indexed=3D1)
> >     DT      =3D Float32Col(indexed=3D1)
> >     HI      =3D Float32Col(indexed=3D1)
> >     LO      =3D Float32Col(indexed=3D1)
> >=20
> > Cheers,
> >=20
> > --=20
> > Francesc Altet     >qo<   http://www.carabos.com/
> > C=E1rabos Coop. V.   V  V   Enjoy Data
> >                     ""
> >=20
> >=20
> >=20
> >
> -------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> > post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> > from ThinkGeek.
> > It's fun and FREE -- well,
> > almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Pytables-users mailing list
> > Pyt...@li...
> >
> https://lists.sourceforge.net/lists/listinfo/pytables-users
> >=20
>=20
>=20
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around=20
> http://mail.yahoo.com=20
>=20
>=20
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>=20

=2D-=20
>OO< =A0 Francesc Altet    ||  http://www.carabos.com/
V =A0V =A0 Carabos Coop. V.  ||  Who is your data daddy? PyTables
 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Sorry, I spoke to soon...
The indexed=1 or the .createIndex() is still not being
created. With the declaration (indexed=1), execution
"tables" the data without acknowledging that indexing
was not performed. With .createIndex() I'm getting a
RuntimeError.

  File "hdf5Extension.pyx", line 1411, in
hdf5Extension.Group._g_createGroup
RuntimeError: Can't create the group _i_raw_A1.

Also Francesc, my understanding is, that this is the
answer to "index persistence"; when an original index
of an array is preserved after having performed "cuts"
and other various "operations" on the data. IOW,
unlike natural numarray characteristics which requires
extra effort to maintain an array's original index
references. Is this correct? If so, I have not yet
figured out how I access it.

Thank you kindly,
Kevin

--- kevin lester <lke...@ya...> wrote:

> Hi Francesc,
> 
> First, let me say that the field variable change did
> allow indexing.(thanks).
>  
> After My last email, I re-Installed all packages. At
> first nothing worked. IDE's and DOS command
> executions
> just crashed immediately. After Fooling around for
> quite some time, I got things to work by not
> including
> the SZLIB files(and .dll). 
> 
> But... PythonWin (which is the IDE I was using
> before
> all this crashing) is still crashing. It was
> performing before, but did need to be re-booted
> after
> nearly each file upload. I don't necessarily care to
> use it. Hopefully I'll find the stability in the IDE
> I'm going to try to work with now (SPE).
> 
> Not being as computer savvy yet as I'm going to be
> ;-), I had trouble with knowing which of the
> optionable versions of szlib I needed. But, is szlib
> even necessary??? I did'nt get that impression from
> HDF5's instructions.
> 
> At this point I was also able to utilize 'lzo' &
> 'ucl'
> for compression.  
> 
> Anyway I'm going to see how things progress and I'll
> give a report later.
> 
> Many, many thanks,
> Kevin
> 
> 
> --- Francesc Altet <fa...@ca...> wrote:
> 
> > A Dimecres 12 Gener 2005 13:11, kevin lester va
> > escriure:
> > > I'm having problems with the zlib,lzo,ucl
> > compression
> > > libraries being available to pyTables. I planted
> > the
> > > .dlls in the /WINDOWS/SYSTEM folder and have
> > > PYTHONPATH directed there as mentioned in the
> > > installation instructions.
> > 
> > Well, that seems good. What kind of problems are
> you
> > having?
> > 
> > > I included the code below for the declaritve.
> The
> > > .createIndex() version was just as was shown in
> > 6.22
> > > of the tutorial 
> > > 
> > > >>> indexrows = table.cols.var1.createIndex()
> <<<
> > 
> > After looking at your script, I can see that you
> > have been bitten by a known
> > bug, that prevents to name a column with a name
> that
> > contains another column
> > name as a substring. While I release a pytables
> > version with a cure to this,
> > you may try the next column declaration that is
> free
> > of this problem:
> > 
> > class ES(IsDescription):
> >     H1       = Int8Col(indexed=1)
> >     M1       = Int8Col(indexed=1)
> >     S1       = Int8Col(indexed=1)
> >     BS      = Int16Col(indexed=1)
> >     BP      = Float32Col(indexed=1)
> >     AP      = Float32Col(indexed=1)
> >     AS      = Int16Col(indexed=1)
> >     L1       = Float32Col(indexed=1)
> >     V1       = Int16Col(indexed=1)
> >     A1       = Int8Col(indexed=1)
> >     UD      = BoolCol(indexed=1)
> >     TV      = Int64Col(indexed=1)
> >     DT      = Float32Col(indexed=1)
> >     HI      = Float32Col(indexed=1)
> >     LO      = Float32Col(indexed=1)
> > 
> > Cheers,
> > 
> > -- 
> > Francesc Altet     >qo<   http://www.carabos.com/
> > Cárabos Coop. V.   V  V   Enjoy Data
> >                     ""
> > 
> > 
> > 
> >
>
-------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> > post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> > from ThinkGeek.
> > It's fun and FREE -- well,
> > almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Pytables-users mailing list
> > Pyt...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam
> protection around 
> http://mail.yahoo.com 
> 
> 
>
-------------------------------------------------------
> The SF.Net email is sponsored by: Beat the
> post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> 


		
__________________________________ 
Do you Yahoo!? 
All your favorites on one personal page – Try My Yahoo!
http://my.yahoo.com 

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Hi Francesc,

First, let me say that the field variable change did
allow indexing.(thanks).
 
After My last email, I re-Installed all packages. At
first nothing worked. IDE's and DOS command executions
just crashed immediately. After Fooling around for
quite some time, I got things to work by not including
the SZLIB files(and .dll). 

But... PythonWin (which is the IDE I was using before
all this crashing) is still crashing. It was
performing before, but did need to be re-booted after
nearly each file upload. I don't necessarily care to
use it. Hopefully I'll find the stability in the IDE
I'm going to try to work with now (SPE).

Not being as computer savvy yet as I'm going to be
;-), I had trouble with knowing which of the
optionable versions of szlib I needed. But, is szlib
even necessary??? I did'nt get that impression from
HDF5's instructions.

At this point I was also able to utilize 'lzo' & 'ucl'
for compression.  

Anyway I'm going to see how things progress and I'll
give a report later.

Many, many thanks,
Kevin


--- Francesc Altet <fa...@ca...> wrote:

> A Dimecres 12 Gener 2005 13:11, kevin lester va
> escriure:
> > I'm having problems with the zlib,lzo,ucl
> compression
> > libraries being available to pyTables. I planted
> the
> > .dlls in the /WINDOWS/SYSTEM folder and have
> > PYTHONPATH directed there as mentioned in the
> > installation instructions.
> 
> Well, that seems good. What kind of problems are you
> having?
> 
> > I included the code below for the declaritve. The
> > .createIndex() version was just as was shown in
> 6.22
> > of the tutorial 
> > 
> > >>> indexrows = table.cols.var1.createIndex() <<<
> 
> After looking at your script, I can see that you
> have been bitten by a known
> bug, that prevents to name a column with a name that
> contains another column
> name as a substring. While I release a pytables
> version with a cure to this,
> you may try the next column declaration that is free
> of this problem:
> 
> class ES(IsDescription):
>     H1       = Int8Col(indexed=1)
>     M1       = Int8Col(indexed=1)
>     S1       = Int8Col(indexed=1)
>     BS      = Int16Col(indexed=1)
>     BP      = Float32Col(indexed=1)
>     AP      = Float32Col(indexed=1)
>     AS      = Int16Col(indexed=1)
>     L1       = Float32Col(indexed=1)
>     V1       = Int16Col(indexed=1)
>     A1       = Int8Col(indexed=1)
>     UD      = BoolCol(indexed=1)
>     TV      = Int64Col(indexed=1)
>     DT      = Float32Col(indexed=1)
>     HI      = Float32Col(indexed=1)
>     LO      = Float32Col(indexed=1)
> 
> Cheers,
> 
> -- 
> Francesc Altet     >qo<   http://www.carabos.com/
> Cárabos Coop. V.   V  V   Enjoy Data
>                     ""
> 
> 
> 
>
-------------------------------------------------------
> The SF.Net email is sponsored by: Beat the
> post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

A Dimecres 12 Gener 2005 13:11, kevin lester va escriure:
> I'm having problems with the zlib,lzo,ucl compression
> libraries being available to pyTables. I planted the
> .dlls in the /WINDOWS/SYSTEM folder and have
> PYTHONPATH directed there as mentioned in the
> installation instructions.

Well, that seems good. What kind of problems are you having?

> I included the code below for the declaritve. The
> .createIndex() version was just as was shown in 6.22
> of the tutorial=20
>=20
> >>> indexrows =3D table.cols.var1.createIndex() <<<

After looking at your script, I can see that you have been bitten by a known
bug, that prevents to name a column with a name that contains another column
name as a substring. While I release a pytables version with a cure to this,
you may try the next column declaration that is free of this problem:

class ES(IsDescription):
    H1       =3D Int8Col(indexed=3D1)
    M1       =3D Int8Col(indexed=3D1)
    S1       =3D Int8Col(indexed=3D1)
    BS      =3D Int16Col(indexed=3D1)
    BP      =3D Float32Col(indexed=3D1)
    AP      =3D Float32Col(indexed=3D1)
    AS      =3D Int16Col(indexed=3D1)
    L1       =3D Float32Col(indexed=3D1)
    V1       =3D Int16Col(indexed=3D1)
    A1       =3D Int8Col(indexed=3D1)
    UD      =3D BoolCol(indexed=3D1)
    TV      =3D Int64Col(indexed=3D1)
    DT      =3D Float32Col(indexed=3D1)
    HI      =3D Float32Col(indexed=3D1)
    LO      =3D Float32Col(indexed=3D1)

Cheers,

=2D-=20
=46rancesc Altet =A0   >qo< =A0 http://www.carabos.com/
C=E1rabos Coop. V. =A0 V =A0V =A0 Enjoy Data
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ""

Re: [Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

--- Francesc Altet <fa...@ca...> wrote:

> Hi Kevin,
> 
> Can you provide more info? An excerpt of your
> indexation code, pytables
> version, hdf5 version...
> 
> Cheers,
> 
> A Dimecres 12 Gener 2005 09:18, kevin lester va
> escriure:
> > Hi Fransesc,
> > 
> > I'm not able to index my tables.cols either
> > declaritively or after. I am incurring a runtime
> > error:  
> > 
> > File "hdf5Extension.pyx", line 1831, in
> > hdf5Extension.Table._read_field_name
> > RuntimeError: Problems reading table column.
> > 
> > My table.nrows are +-150,000 and (fyi) am
> attempting
> > this on WinXP w/1G ram. 
> > 
> > Do you have any suggestions? 
> > 
> > Thank you very much,
> > Kevin
> > 
> > 
> > 		
> > __________________________________ 
> > Do you Yahoo!? 
> > Yahoo! Mail - now with 250MB free storage. Learn
> more.
> > http://info.mail.yahoo.com/mail_250
> > 
> > 
> >
>
I'm having problems with the zlib,lzo,ucl compression
libraries being available to pyTables. I planted the
.dlls in the /WINDOWS/SYSTEM folder and have
PYTHONPATH directed there as mentioned in the
installation instructions.
I included the code below for the declaritve. The
.createIndex() version was just as was shown in 6.22
of the tutorial 

>>> indexrows = table.cols.var1.createIndex() <<<

I'm using pyTables 0.9.1 and HDF5 patched version you
provided the link for in a previous Q from me.

ftp://ftp.ncsa.uiuc.edu/HDF/HDF5/current/patches/bin/WinXP-patch.tar.gz

Thank you Francesc,
Kevin
########################################################

import es01_04_05 as es
from tables import *
from numarray import *
dat = es.dat

date = 'd01_04_05'
idxdate = 'd050104'
length = (len(dat))+20

class ES(IsDescription):
    H       = Int8Col(indexed=1)
    M       = Int8Col(indexed=1)
    S       = Int8Col(indexed=1)
    BS      = Int16Col(indexed=1)
    BP      = Float32Col(indexed=1)
    AP      = Float32Col(indexed=1)
    AS      = Int16Col(indexed=1)
    L       = Float32Col(indexed=1)
    V       = Int16Col(indexed=1)
    A       = Int8Col(indexed=1)
    UD      = BoolCol(indexed=1)
    TV      = Int64Col(indexed=1)
    DT      = Float32Col(indexed=1)
    HI      = Float32Col(indexed=1)
    LO      = Float32Col(indexed=1)
    
file  =
openFile("/MY_SCRIPTS/DataFiles/ES_DATA.h5",mode="w",title="ES_DATA_FILE")
root  = file.root

group1   = file.createGroup("/",'d050104','d01_04_05')
table1 = file.createTable(group1,'raw',ES,"RAW",
expectedrows=length)

# ATTRIBUTES---------
g1  = file.root.d050104
g1._v_attrs.date      = date
g1._v_attrs.idxdate   = idxdate

t1  = file.root.d050104.raw
t1.attrs.date      = date
t1.attrs.idxdate   = idxdate

eS = table1.row

for i in xrange(len(dat)):
    eS['H']     = dat[i][13]
    eS['M']     = dat[i][14]
    eS['S']     = int(dat[i][15])
    eS['BS']    = dat[i][0]
    eS['BP']    = dat[i][1]
    eS['AP']    = dat[i][2]
    eS['AS']    = dat[i][3]
    eS['L']     = dat[i][4]
    eS['V']     = dat[i][5]
    eS['UD']    = dat[i][12]    
    eS['A']     = dat[i][11]
    eS['TV']    = dat[i][8]
    eS['DT']    = dat[i][16]
    eS['HI']    = dat[i][6]
    eS['LO']    = dat[i][7]
    eS.append()
table1.flush()
  
file.close()

-------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> > It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Pytables-users mailing list
> > Pyt...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/pytables-users
> > 
> 
> -- 
> Francesc Altet     >qo<   http://www.carabos.com/
> Cárabos Coop. V.   V  V   Enjoy Data
>                     ""
> 
> 



		
__________________________________ 
Do you Yahoo!? 
All your favorites on one personal page – Try My Yahoo!
http://my.yahoo.com 

Re: [Pytables-users] Indexed & .createIndex() runtime

[Francesc A. <fa...@ca...>]

Hi Kevin,

Can you provide more info? An excerpt of your indexation code, pytables
version, hdf5 version...

Cheers,

A Dimecres 12 Gener 2005 09:18, kevin lester va escriure:
> Hi Fransesc,
>=20
> I'm not able to index my tables.cols either
> declaritively or after. I am incurring a runtime
> error: =20
>=20
> File "hdf5Extension.pyx", line 1831, in
> hdf5Extension.Table._read_field_name
> RuntimeError: Problems reading table column.
>=20
> My table.nrows are +-150,000 and (fyi) am attempting
> this on WinXP w/1G ram.=20
>=20
> Do you have any suggestions?=20
>=20
> Thank you very much,
> Kevin
>=20
>=20
> 	=09
> __________________________________=20
> Do you Yahoo!?=20
> Yahoo! Mail - now with 250MB free storage. Learn more.
> http://info.mail.yahoo.com/mail_250
>=20
>=20
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>=20

=2D-=20
=46rancesc Altet =A0   >qo< =A0 http://www.carabos.com/
C=E1rabos Coop. V. =A0 V =A0V =A0 Enjoy Data
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ""

[Pytables-users] Indexed & .createIndex() runtime

[kevin l. <lke...@ya...>]

Hi Fransesc,

I'm not able to index my tables.cols either
declaritively or after. I am incurring a runtime
error:  

File "hdf5Extension.pyx", line 1831, in
hdf5Extension.Table._read_field_name
RuntimeError: Problems reading table column.

My table.nrows are +-150,000 and (fyi) am attempting
this on WinXP w/1G ram. 

Do you have any suggestions? 

Thank you very much,
Kevin


		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - now with 250MB free storage. Learn more.
http://info.mail.yahoo.com/mail_250