Re: [Pytables-users] Storing small files in the filenodes

[Valeriy S. <sok...@gm...>]

Not sure if the quoted message was delivered to the list (maybe because I
was not registered on this list), so reposting it this way...

On Fri, Jul 12, 2013 at 5:40 PM, Valeriy Sokolov
<sok...@gm...>wrote:

> Hi,
>
> I am trying to store lots of small (~2Kb) files in the filenode-s of the
> pytables. And I ran into a trouble with size overhead.
>
> 200 such files which consumes in total ~2Mb on the filesystem takes 14Mb
> in the .h5 file produced by pytables. My experiments show that if I create
> 200 file nodes and store 1 byte in each, I have .h5 of 14Mb. Approximately
> from the size like 200Kb per file node I have a linear increase of size.
> I.e. 400Kb per node leads to 89Mb, and 800Kb per node leads to 164Mb.
>
> But I would like to store ~2Kb there and current overhead (like 70Kb per
> file node) is pretty huge.
>
> Could you please help me with work-around for this issue?
>
> Thank you in advance.
>
> --
> Best regards,
> Valeriy Sokolov.
>



-- 
Best regards,
Valeriy Sokolov.