Re: [Pytables-users] Multithreaded decompress unexpectedly does not help

[Alvaro T. C. <al...@mi...>]

I'll answer myself on the size-checking: the right attributes are
Leaf.size_in_memory and Leaf.size_on_disk (per
http://pytables.github.com/usersguide/libref/hierarchy_classes.html)


-á.



On 6 December 2012 12:42, Alvaro Tejero Cantero <al...@mi...> wrote:

> Thank you for the comprehensive round-up. I have some ideas and reports
> below.
>
> What about ctables? The documentation says that it is specificly
> column-access optimized, which is what I need in this scenario (sometimes
> sequential, sometimes random).
>
> Unfortunately I could not get the rootdir parameter for ctables __init__
> to work in carray 0.4 and pip-installing 0.5 or 0.5.1 leads to compilation
> errors.
>
> This is the ctables-to-disk error:
>
> ct2 = ca.ctable((np.arange(30000000),), names=('range2',),
> rootdir='/tmp/ctable2.ctable')
>
> ---------------------------------------------------------------------------TypeError                                 Traceback (most recent call last)/home/tejero/Dropbox/O/nb/nonridge/<ipython-input-29-255842877a0b> in <module>()----> 1 ct2 = ca.ctable((np.arange(30000000),), names=('range2',), rootdir='/tmp/ctable2.ctable')
> /home/tejero/Local/Envs/test/lib/python2.7/site-packages/carray/ctable.pyc in __init__(self, cols, names, **kwargs)    158                 if column.dtype == np.void:    159                     raise ValueError, "`cols` elements cannot be of type void"--> 160                 column = ca.carray(column, **kwargs)    161             elif ratype:    162                 column = ca.carray(cols[name], **kwargs)
> /home/tejero/Local/Envs/test/lib/python2.7/site-packages/carray/carrayExtension.so in carray.carrayExtension.carray.__cinit__ (carray/carrayExtension.c:3917)()
> TypeError: __cinit__() got an unexpected keyword argument 'rootdir'
>
>
>
> And this is cut from the pip output when trying to upgrade carray.
>
> gcc: carray/carrayExtension.c
>
> gcc: error: carray/carrayExtension.c: No such file or directory
>
>
>
> Two more notes:
>
> * a way was added to check in-disk (compressed) vs in-memory
> (uncompressed) node sizes. I was unable to find the way to use it either
> from the 2.4.0 release notes or from the git issue
> https://github.com/PyTables/PyTables/issues/141#issuecomment-5018763
>
> * is/will it be possible to load PyTables carrays as in-memory carrays
> without decompression?
>
> Best,
>
> Álvaro
>
>
>
> On 6 December 2012 11:49, Francesc Alted <fa...@gm...> wrote:
>
>> completeness, let's see how fast can perform
>> carray (the package, n
>>
>
>