[Pytables-users] Re: read rates...

[damon f. <dam...@ya...>]

Hi Francesc,

Thanks for your answers.

Regarding the 'unknown compression type' messages,
here is the information you requested.

~test python test_all.py --show-versions
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
PyTables version:  0.9.1
Extension version: $Id: hdf5Extension.c,v 1.170.2.2
2004/11/22 17:50:39 falted Exp $ 
HDF5 version:      1.6.3-patch
numarray version:  1.1.1
Zlib version:      1.2.2
LZO version:       1.08 (Jul 12 2002)
UCL version:       1.03 (Jul 20 2004)
Python version:    2.4 (#1, Jan 31 2005, 12:54:29) 
[GCC 3.3.5 (Debian 1:3.3.5-6)]
Platform:          linux2-x86_64
Byte-ordering:     little
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

I built PyTables from sources.  I've attached the
stdout and stderr logs from the build script.  These
error messages aren't causing me any problems, so I
can live with it.  (When I request a filter with ucl
or zlib PyTables generates the error message that the
compression type is unknown, but then goes ahead and
compresses with them anyway.)  Let me know if there is
any more data that would be helpful for you on this.


Regarding read rates, I was thinking that 4 seconds to
read 23 items was slower than expected.  But maybe
not.    There were probably 5 accesses to the index
and then 23 accesses to the tables, 140 ms per access.

What will be the function of the "sorted by" and
"group by" qualifiers.  Since they are qualifiers on
search methods it sounds as if they don't alter the
structure of the underlying table, rather they present
the results in a particular order.  So search
performance would not be impacted.

What I would like to be able to do is actually
reorganize my tables so the entries are ordered by a
particular field. I will normally be accessing data in
order of this field, so if the table were ordered by
this field I could just use a cursor to access the
data...fast fast fast :)  My tables will be small
enough to fit in memory, so in principle I could load
the entire array into memory and then sort it, but I
don't see a sort method on the table in either
PyTables or numarray.

Thanks!
  Damon


		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail