From: Alvaro T. C. <al...@mi...> - 2012-12-06 12:42:57
|
Thank you for the comprehensive round-up. I have some ideas and reports below. What about ctables? The documentation says that it is specificly column-access optimized, which is what I need in this scenario (sometimes sequential, sometimes random). Unfortunately I could not get the rootdir parameter for ctables __init__ to work in carray 0.4 and pip-installing 0.5 or 0.5.1 leads to compilation errors. This is the ctables-to-disk error: ct2 = ca.ctable((np.arange(30000000),), names=('range2',), rootdir='/tmp/ctable2.ctable') ---------------------------------------------------------------------------TypeError Traceback (most recent call last)/home/tejero/Dropbox/O/nb/nonridge/<ipython-input-29-255842877a0b> in <module>()----> 1 ct2 = ca.ctable((np.arange(30000000),), names=('range2',), rootdir='/tmp/ctable2.ctable') /home/tejero/Local/Envs/test/lib/python2.7/site-packages/carray/ctable.pyc in __init__(self, cols, names, **kwargs) 158 if column.dtype == np.void: 159 raise ValueError, "`cols` elements cannot be of type void"--> 160 column = ca.carray(column, **kwargs) 161 elif ratype: 162 column = ca.carray(cols[name], **kwargs) /home/tejero/Local/Envs/test/lib/python2.7/site-packages/carray/carrayExtension.so in carray.carrayExtension.carray.__cinit__ (carray/carrayExtension.c:3917)() TypeError: __cinit__() got an unexpected keyword argument 'rootdir' And this is cut from the pip output when trying to upgrade carray. gcc: carray/carrayExtension.c gcc: error: carray/carrayExtension.c: No such file or directory Two more notes: * a way was added to check in-disk (compressed) vs in-memory (uncompressed) node sizes. I was unable to find the way to use it either from the 2.4.0 release notes or from the git issue https://github.com/PyTables/PyTables/issues/141#issuecomment-5018763 * is/will it be possible to load PyTables carrays as in-memory carrays without decompression? Best, Álvaro On 6 December 2012 11:49, Francesc Alted <fa...@gm...> wrote: > completeness, let's see how fast can perform > carray (the package, n > |