Re: [Pytables-users] Questionnaire

[Francesc A. <fa...@op...>]

A Dissabte 20 Setembre 2003 00:22, st...@st... va escriure:
> > - What feature do you miss more (not necessarily listed before)?
>
> I wish I could write root.sfc.sfc_eqang25.lifted_index
> instead of root.sfc.sfc_eqang25.read(field='lifted_index')
> (You may have already implemented this...)

Well, I already *tried* to implement that, but I encountered the next
problem: a column name cannot be the same of a standard Table attribute
(like for example "nrows" or "rowsize"). For doing that, I would have
implemented Tables attributes as in Group (i.e. with a "_v_" prefix so as to
not collide with column names). Frankly, I do not think it would be worth
the change: I think it's better to continue with the present naming schema.
Do you agree?

>
> > - Do you prefer seeing pytables to become (even) faster or less memory
> > demanding?
>
> Less memory demanding.
>
> NumPy is already optimized for "fast, use much memory".
>
> PyTables complements NumPy best if optimized for "slow, uses little
> memory".
>
> If you write pytables to optimize for space and I need one section
> optimized for speed, I can always read some data into an ordinary
> numpy array, which will be kept in memory. But if you optimize for
> speed and I need one section optimized for space, I will have to do a
> lot of work breaking my dataset into separate files and opening each
> only when needed.

I agree in that pytables uses now too much memory. The addition of the
rootUEP parameter in openFile() function in 0.7 version it's a great relief
for dealing with that. However, I'm thinking about building a new,
lightweight access schema to the objects on files, instead of the current
full-fledged, but heavyweighted, object tree.

>
> > - Which present limitations do you find more annoying?
>
> My code in 0.5.1 leaked memory. (I don't know whether this leak was
> in my code, in pytables, or in numeric or numarray). If this leak
> was in pytables, that was the most annoying limitation.

In 0.5.1 there are memory leaks both in pytables and numarray. The memory
leak in ptables has been fixed now, and will be included in the next version
(0.7.2) that I'll be releasing very soon. The leak in numarray seems to be
unsolved yet, but anyway, the pytables 0.7.2 should be quite less memory
demanding that previous releases.

Thanks for your time filling the questionnaire!,

-- 
Francesc Alted