Re: [Pytables-users] Query Tests

[Francesc A. <fa...@ca...>]

A Dilluns 24 Gener 2005 08:29, kevin lester va escriure:
> Hi Francesc,
>
> My apologies for not reciprocating to you a quick
> response.

No problem

> The only radical difference I noticed was the page
> file usage as noted. The CPU ran between 50-54%
> consistently. Every other calib. on the Win task
> manager stayed approximately equal for each test. I do

Sorry, page file usage means more access to disk?

> run into intermittent crashes (w/pytables) every once
> in a while, especially after a fresh reboot of Win.

Perhaps this is consequence of the new procedure for generating extensions
for Python 2.4 in Win. Would you be able to run these tests on Python 2.3
and check if the crashes still happen?

> Finally, notice that the UserWarning message is always
> being displayed even though all comp. package dll's
> are in order.

Uh, I've no idea of what can be happening here.

Regarding your figures, I can notice (with some surprise) that the speed-up
for bigger Tables is not as good as desired. I'm afraid that this is a
consequence of having a larger list of rows that pass the selections. The
current indexing algorithm implemented in PyTables 0.9.1 is quite slow when
the number of rows that pass the cuts is relatively large. However, it works
much better when this number is slow compared with the total number of rows
in Table. In the future, we plan to improve this by looking at the number of
rows that passes the cuts, and if that number is bigger than a certain
percentage of the total number of rows (although other facts maybe
considered as well), then dropping to a inkernel search. In fact, this will
be part of a query optimizer that we are working on, and that will be part
of the future PyTables Pro.

Thanks for the more exhaustive benchmarks anyway.

=2D-=20
>qo<   Francesc Altet =A0 =A0 http://www.carabos.com/
V =A0V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 ""