Re: [Pytables-users] Speed of in-kernel Full-Table Search

[Anthony S. <sc...@gm...>]

On Mon, Jun 24, 2013 at 4:25 AM, Wagner Sebastian <
Seb...@ai...> wrote:

>  Dear PyTables-Users,****
>
> ** **
>
> For testing purposes I use a PyTables DB with 4 columns (1x Uint8 and
> 3xFloat) with 750k rows, the total file size about 90MB. As the free
> version does no support indexing I thought that a search (full-table) on
> this database would last a least one or two seconds, because the file has
> to be loaded first (throttleneck I/O), and then the search over ~20k rows
> can begin. But PyTables took only 0.05 seconds for a full table search
> (in-kernel, so near C-speed, but nevertheless full table), while my
> bisecting algorithm with a precomputed sorted list wrapped around PyTables
> (but saved in there), took about 0.5 seconds.****
>
> ** **
>
> So the thing I don’t understand: How can PyTables be so fast without any
> Indexing?
>

Hi Sebastian,

First, there is no longer a non-free version of PyTables and v3.0 *does* have
indexing capabilities.  However, you have to enable them so you probably
weren't using them.

PyTables is fast because HDF5 is a binary format, it using pthreads under
the covers to parallelize some tasks, and it uses numexpr (which is also
parallel) to evaluate many expressions.  All of these things help make
PyTables great!

Be Well
Anthony


> ****
>
> ** **
>
> I’m using 3.0.0rc2 coming with WinPython****
>
> ** **
>
> Regards,****
>
> Sebastian****
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>