Re: [Pytables-users] Optimizing pytables for reading entire columns at a time

[Luke L. <dur...@gm...>]

Hi again,

I haven't been getting the updates via email so I'm attempting to post
again to respond.

Thanks everyone for the suggestions.  I have a few questions:

1.  What is the benefit of using the stand-alone carray project (
https://github.com/FrancescAlted/carray) vs Pytables.carray?
2.  I realized my code base never uses the query functionality of a Table.
 So, I changed all my columns to be just Pytables.carray objects instead.
 They are all sitting at the top of the hierarchy, just below root.  Is
this a good idea?

I see a big speed increase from this obviously because now everything is
stored contiguously.  However, are there any downsides to doing this?  I
suppose I could also use EArray, but we are never actually changing the
data once it is stored in HDF5.

3.  Is compression automatically happening with the Carray?  I know the
documentation says that compression is supported, but what do I need to do
to enable it?  Maybe it's already happening and this is contributing to my
big speed improvement.

4.  I would certainly love to take a look at contributing something like
this in my free time.  I don't have a whole lot at this time so the changes
could take a while.  I'm sure I need to learn a lot more about the codebase
before really giving it a try.  I'm going to take a look at this though,
thanks for the suggestion!

5.  How do I subscribe to the dev mailing list?  I only see announcements
and users.
6.  Any idea why I'm not getting the emails from the list?  I signed up 2
days ago and didn't get any of your replies via email.

Thanks!