Re: [Pytables-users] Clear chunks from CArray

[Antonio V. <ant...@ti...>]

Hi Anthony, hi Giovanni,

Il giorno 06/ago/2013, alle ore 00:45, Anthony Scopatz <sc...@gm...> ha scritto:

> 
> On Mon, Aug 5, 2013 at 3:14 PM, Giovanni Luca Ciampaglia <glc...@gm...> wrote:
> Hi Anthony,
> 
> what do you mean precisely? I tried
> 
> del ca[:,:]
> 
> but CArray does not support __delitem__. Looking in the documentation I could
> only find a method called remove_rows, but it's in Table, not CArray. Maybe I am
> missing something?
> 
> Huh, it should...  This is definitely an oversight on our part.  If you could please open an issue for this -- or better yet -- write a pull request that implements delitem, that'd be great!
> 
> So I think you are right that there is no current way to delete rows from a CArray.  Oops!  (Of course, I may still be missing something as well).
> 
> It looks like EArray also has this problem too, otherwise I would just tell you to use that.

I'm not sure to understand the problem.

The "truncate" method of arrays can be used to remove rows from an extendable array.
It seems to me that it is not documented but we should add it to the UG.

CArrays cannot be resized.

> 
> Be Well
> Anthony
>  
> 
> Thank,
> 
> Giovanni
> 
> On Mon 05 Aug 2013 03:43:42 PM EDT, pyt...@li...
> wrote:
> >
> >> Hello Giovanni, I think you may need to del that slice and then possibly
> >> repack. Hope this helps. Be Well Anthony On Mon, Aug 5, 2013 at 2:09 PM,
> >> Giovanni Luca Ciampaglia < glc...@gm...> wrote:
> >>> Hello all,
> >>>
> >>> is there a way to clear out a chunk from a CArray? I noticed that setting
> >>> the
> >>> data to zero actually takes disk space, i.e.
> >>>
> >>> ***
> >>> from tables import open_file, BoolAtom
> >>>
> >>> h5f = open_file('test.h5', 'w')
> >>> ca = h5f.create_carray(h5f.root, 'carray', BoolAtom(), shape=(1000,1000),
> >>> chunkshape=(1,1000))
> >>> ca[:,:] = False
> >>> h5f.close()
> >>> ***
> >>>
> >>> The resulting file takes 249K ...
> >>>
> >>> Best,
> >>>
> >>> --
> >>> Giovanni Luca Ciampaglia


HDF5 handles efficiently chunks that have never been written saving some disk space but I doubt that chunks can be "de-initializad".
If my understanding is correct, once one write some value in a chunk (even if it is the default value) the chunk is allocated at HDF5 level and written to disk.
At they point one can only change item values.
Also I doubt that a repack can help in this case (not tested).

The only solution IMO is compression.


cheers

--
Antonio Valentino