Re: [Pytables-users] Does compression applie to VLarray?

[Alain F. <ala...@fr...>]

Hello Francesc,

Yes indeed compression applies.
But I was misleaded by the fact that I also tryed to simulate virtual len=
gth=20
arrays using tables, and table compression ratio gives much more compress=
ed=20
files.
For simulating 1000 objects with an array containing 10 000 values, I had=
 a=20
file size of 1 353 Ko using tables, and 58 687 using VLarrays.

Plus I tryed to experiment row modification in VLarray, with my algorythm=
=20
define after and in attached prova.py I need to run a ptrepack on my h5=20
file, to come the optimum size.
If you have some recommendations to implement such behavior, they are=20
welcome.

Kind regards,

            Alain
-------------------------------------------------------------------
Algorythm to modify row 5 000, in a VLarray of 30 000 rows:
-------------------------------------------------------------------
    a1 =3D array([5])
    a2 =3D array([5, 6])
    a3 =3D array([5, 6, 7])
    for i in range(10000):
        vlarray.append(a1)
        vlarray.append(a2)
        vlarray.append(a3)

    # MODIFICATION OF A ROW AT 5000 IN THE VLARRAY
    tmp_vlarray =3D fileh.copyNode(where=3D'/vlarray1',
        newname=3D'tmp_vl_array', start=3D0, stop=3D5000)
    tmp_vlarray.append(array([1, 2, 3]))
    for i in range(5001,vlarray.nrows):
        tmp=3Dvlarray[i]
        tmp_vlarray.append(tmp)

    fileh.removeNode(where=3D'/', name=3D'vlarray1')
    fileh.renameNode(where=3D'/', newname=3D'vlarray1', name=3D'tmp_vl_ar=
ray')


----- Original Message -----=20
From: "Francesc Altet" <fa...@ca...>
To: <pyt...@li...>
Cc: "Alain Fagot" <ala...@fr...>
Sent: Friday, December 16, 2005 1:20 PM
Subject: Re: [Pytables-users] Does compression applie to VLarray?


Hello Alain,

A Dijous 15 Desembre 2005 14:03, Alain Fagot va escriure:
> Hello,
>
> Looking at pyTables 1.2 documentation, it seems to me that compression
> should applies to VLarray. The File.createVLarray() method having a=20
> filters
> parameters.
> When experimenting it, I have no compression for VLarray.

Using the attached script, I get:

-rw-r--r--  1 faltet faltet 1300528 Dec 16 13:10 prova.h5
-rw-r--r--  1 faltet faltet  827440 Dec 16 13:09 prova-zlib1.h5

It does not achieve a lot of compression (maybe there is an issue
related with variable length arrays management in HDF5), but it seems
to work anyway.

Regards,

--=20
>0,0<   Francesc Altet http://www.carabos.com/
V   V   C=E1rabos Coop. V. Enjoy Data
 "-"