Re: [Pytables-users] PyTables performance

[Francesc A. <fa...@ca...>]

Hello Thor,

El dc 15 de 03 del 2006 a les 09:24 +0100, en/na
tho...@ib... va escriure:
> We are very interested in the possible performance improvements in the ne=
w
> Pro version. The ability to have complex indices and queries will be
> helpful.
> However, in our application a response time in the order of milliseconds =
is
> unacceptable. We were hoping that the built in caching would come to our
> rescue, but we have not been able to get the performance we need. Instead
> we have implemented our own (very ad-hoc) caching. So a couple of
> questions:
>=20
> Anyone know of a more standard way to have faster queries (in the sub
> millisecond range). Shouldn't this be possible if the result of a query i=
s
> cached and sits in RAM?

Well, generally speaking, keeping the result of a query in RAM will
always speed-up a query. Even if this is not implemented specifically,
re-doing the same (or similar) query several times, will also improve
the time because the index is probably in the filesystem cache (this
depends on how large the index is and how much memory do you have).=20

Regarding PyTables Pro, our (preliminary) timings shows that, for simple
queries involving one single index (remeber, you can always create a
complex index to speed-up a complex query) we can be in the
sub-millisecond range whenever the index is already in cache, that is,
if you have already run the query (or other similar) before, and for
tables up to 50 millions of rows. When the query is not in cache, and
for tables up to 1 million of rows, we expect sub-hundreth of second
times, while for tables with more than 1 billion of rows, our goal is
sub-second (modern relational engines usage is prohibitive in such
extremely large tables scenarios). See [1] for some slides about
PyTables Pro measured performance and our goals (only for non-cached
queries).

[1] http://www.carabos.com/downloads/resources/pytables-cstables%
40hdf__20051201.pdf

>
> Are there any pricing information for PyTables Pro. What are the licensin=
g
> options?

I'm sorry, but we don't have a definitive pricing/licensing schema yet.
But, just for your information, the prices will be around one thousand
euros (maybe a little more or a little less, but in that order of
magnitude) for one license.

Regards,

--=20
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   C=E1rabos Coop. V.   Enjoy Data
 "-"