Re: [Pytables-users] Questionnaire

[<st...@st...>]

Francesc Alted <fa...@op...> writes:

> Hi List,
>
> Now that almost a year has passed from the first public release of pytables,
> I need your feedback. So, please, if you are using pytables and like this
> effort to be continued and improved, take some time to respond to this
> questionnaire about pytables; it will help me to decide what to do in the
> next few months.
>
> Following are my plans for the next few months, before releasing the 1.0
> version. Could you respond to these questions by giving a punctuation
> ranging from +5 (I absolutely need that) to 0 (I can pass without this) to
> these planned features?
>
> - Implement enlargeable (i.e. chunked) Array objects.
+0

> - Implement support for Variable Length values (mostly in enlargeable
> Arrays, but possibly also in Tables).
+0

> - Implement relationships (apart from the existing hierarchical ones)
> between objects in the object tree.
+0

> - Make pytables less memory demanding.
+5

> - Improve speed of opening files in pytables
+2

> - Improve speed of reading and selecting values in Leaf (Table and Array)
> objects.
+2

> Now, answer the next questions. If you don't like to respond all of them,
> this is not necessary.
>
> - What feature do you like more (not necessarily listed before)?
Natural naming.

> - What feature do you miss more (not necessarily listed before)?
I wish I could write root.sfc.sfc_eqang25.lifted_index
instead of root.sfc.sfc_eqang25.read(field='lifted_index')
(You may have already implemented this...)

> - Do you prefer seeing pytables to become (even) faster or less memory
> demanding?
Less memory demanding. 

NumPy is already optimized for "fast, use much memory".

PyTables complements NumPy best if optimized for "slow, uses little memory".

If you write pytables to optimize for space and I need one section
optimized for speed, I can always read some data into an ordinary
numpy array, which will be kept in memory. But if you optimize for
speed and I need one section optimized for space, I will have to do a
lot of work breaking my dataset into separate files and opening each
only when needed.

> - Which present limitations do you find more annoying?
My code in 0.5.1 leaked memory. (I don't know whether this leak was
in my code, in pytables, or in numeric or numarray). If this leak
was in pytables, that was the most annoying limitation.

> - In which field of science or bussiness are you using pytables?
accessing hdf(4) data from NASA. (I convert to hdf5 first.)

-- 
Stan