#1 szip compression

closed
nobody
None
5
2007-10-17
2004-10-20
No

It would be nice to have the ability to create szip
compressed files. In my experience, szip produces h5
files that are about 20% smaller than zlib, and it's
faster too.

Discussion

  • Francesc Altet

    Francesc Altet - 2004-10-27

    Logged In: YES
    user_id=249033

    Yeah, I would like to have a deeper look at supporting szip into
    PyTables. However, I'm afraid that this would not take place
    until PyTables 1.0, at least.

    By the way, have you tried using zlib (or lzo or ucl) together
    with shuffle? szip+shuffle can beat zlib+shuffle?

    Cheers,

    Francesc

     
  • Jeff Whitaker

    Jeff Whitaker - 2004-10-27

    Logged In: YES
    user_id=136337

    The comparison I've seen is with HDF4, which I don't believe
    has shuffle. In that case, szip beats gzip by 20% or so.
    I'll see if I can run a test with HDF5 with shuffle and let
    you know how it comes out.

     
  • Jeff Whitaker

    Jeff Whitaker - 2004-10-27

    Logged In: YES
    user_id=136337

    I ran some tests with h5repack, and it does seem that
    although szip beats gzip without shuffle, gzip+shuffle beats
    szip+shuffle. Caveat: I tried exactly one file.

     
  • Ivan Vilata i Balaguer

    • status: open --> closed
     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks