How to choose between XZEmbedded and XZUtils for decompress

2014-04-21
2014-05-16
  • Gilles Vollant

    Gilles Vollant - 2014-04-21

    Hello,
    in SmartVersion, I need a decompression module.
    I tested both XZEmbedded and XZUtils . Both done the work in same way (I need only uncompress LZMA data compressed by XZUtils)

    For me, difference:
    - XZEmbedded can compile with old Microsoft C++. XZUtils need Visual Studio 2013 (+ minor syntax modification before release of update 2)
    - XZUtils is a little more complex to recompile (more file, config.h...)
    - XZUtils is bigger
    - XZUtils is a very little faster

    For me, the speed is more important point, so I uses now XZUtils when I can

     
    Last edit: Gilles Vollant 2014-04-21
  • Geoff Nixon

    Geoff Nixon - 2014-04-24

    I've often compared the two as well, and prefer the xz-embedded code myself. You may want to look at the code I just posted, here, particularly the bit in the README about using more aggressive optimization.
    (There's a older note on this in the xz-embedded read me as well).

    Have you by chance given a look at the Windows build of Clang/LLVM? It integrates rather nicely.

    You also might be able to gain a little more speed if you were to disable integrity checking entirely (for instance, if the data you're decompressing has another built-in check). Never tried that though.

    Best,
    -G

     
  • Lasse Collin

    Lasse Collin - 2014-04-24

    The important difference is that XZ Embedded cannot decompress all .xz files (even if today it works for most files). If you control how the files are compressed, then XZ Embedded may be the way to go. If the .xz files come from users and may have been compressed with different kinds of options and the code size doesn't matter much, then liblzma may be the safer choice.

    Also consider LZMA SDK from 7-zip.org. It has full .xz support and it's MSVC compatible.

     
  • Geoff Nixon

    Geoff Nixon - 2014-04-24

    No kidding? Are we talking > 5.0.5 stuff? Aside from SHA256, what's missing? I think maybe I saw a hardcoded memlimit (?) but shouldn't that be easy enough to adjust?

     
  • Lasse Collin

    Lasse Collin - 2014-04-24

    The Delta filter and SHA-256 are missing, and xzminidec lacks support for concatenated streams and stream padding. In XZ Utils the concatenation and padding support are in liblzma so the command line tool has to just pass a flag (LZMA_CONCATENATED) to liblzma, but currently I think that I won't add those things to the library side of XZ Embedded. They could be added to xzminidec but so far I haven't found it important since xzminidec is mostly a demo program for the library API.

    5.2.0 probably won't have new file format features but >= 5.3.x will have something (metadata at least).

     
  • Gilles Vollant

    Gilles Vollant - 2014-04-25

    I control myself my compression (in my delta code smartversion).
    I init the compression using

    lzma_easy_encoder(&xzutils_stream, preset, LZMA_CHECK_CRC32);
    with preset between 1 and 9.

    It seem there is not compatibiliy with xz embeeded

     
  • Lasse Collin

    Lasse Collin - 2014-04-25

    The files created with those options work with XZ Embedded (including preset 0). If you have problems, I need more information to help you.

     
  • Gilles Vollant

    Gilles Vollant - 2014-04-25

    Both XZ Embeeded and XZ Util work well. My only problem is select faster version :-)

     
  • Gilles Vollant

    Gilles Vollant - 2014-05-08

    I continue work about faster decompression.

    I want the faster decompression engine, with "zlib like" interface, which decompress stream create with xz util and "lzma_easy_encoder(&xzutils_stream, preset, LZMA_CHECK_CRC32);" with preset between 1 and 9.

    On my test, XZ Util is between 25 and 30% faster than XZEmbedded
    LZMA Sdk 9.20 is between 37 and 41% faster than XZUtil !

    so lzma sdk decompress between 12 and 16% faster than xzutil !

    Just for info, I asked on 7zip forum a zlib like compress interface, including partial flush (see https://sourceforge.net/p/sevenzip/discussion/45797/thread/bf68815a/ ), and Igor Pavlov suggested using XZ Util for compress.

    I made test using XCode 5.1 on MacOSX, targeting intel x86, intel x64 and ARMv7 (32 bits) on iPhone.
    I also made test with Visual Studio 2013 target x86 and x64

     
  • Lasse Collin

    Lasse Collin - 2014-05-16

    It's been a long time since I benchmarked things but those differences sound larger than I remember. However, I tested on x86 only and mostly with GCC, so my testing was very limited.

    It is known that XZ Embedded is a bit slower than others, although with a good compiler and optimizations the difference shouldn't have been that huge. I would have expected it to be only 10-15 % slower than XZ Utils. XZ Embedded tries to be both fairly small and very clean/readable code with acceptable performance.

    I should sync the encoder changes from LZMA SDK to XZ Utils and your benchmark makes me think I should look at the decoder too. The decompressor code in XZ Utils should be cleaned up anyway since it's so ugly; a better compromise between speed and readability must be possible.

    When I finally get back to working on XZ Utils, I'll focus on 5.2.0. Syncing with LZMA SDK and such things won't be in that.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks