LZMA2

2009-02-03
2014-08-20
  • Igor Pavlov
    Igor Pavlov
    2009-02-03

    Alpha version of LZMA2 codec:
    http://dl.7-zip.org/lzma2_466a.7z

    INSTALL
    -------
    Copy "Codecs" folder to 7-Zip folder.

    DESCRIPTION
    -----------
    LZMA2 provides the following advantages over LZMA:
    1) Better compression ratio for data than can't be compressed.
       It can store such blocks of data in uncompressed form.
       Also it decompresses such data faster.
    2) Better multithreading support. If you compress big file, LZMA2 can split
       that file to chunks and compress these chunks in multiple threads.

    USAGE
    -----

    -m0c=Size : sets ChunkSize.
    Examples:
      -m0c=128m

    If you don't specify ChunkSize, LZMA2 sets it to DictionarySize * 4,
    but min = 1 MB and max = 256 MB.

    LZMA2 uses:
      1 thread for each chunk in -mx1 and -mx3 modes.
      2 threads for each chunk in -mx5, -mx7 and -mx9 modes.

    If LZMA2 is set to use only such number of threads required for one chunk,
    it doesn't split stream to chunks. So you can get different compression
    ratio for different number of threads (-mmt switch).

    Examples:
      7z a a.7z files -m0=lzma2
      7z a a.7z files -m0=lzma2 -mmt2
      7z a a.7z files -m0=lzma2 -mmt4
      7z a a.7z files -m0=lzma2 -mmt4 -mx1
      7z a a.7z files -m0=lzma2 -m0c=128m -m0d=8m -mmt4

    NOTES!
    ------
    It's possible that future versions of 7-Zip will not support archives
    created with this version of LZMA2.
    So don't use this codec to backup important data.
    Use this codec only for test purposes.

    You can check the speed with timer.exe program:
    http://downloads.sourceforge.net/sevenmax/7bench801.zip

     
    • Simon Berger
      Simon Berger
      2009-02-03

      Is it only working with the console version?

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-03

        In GUI you must write in parameters:
        0=lzma2

         
    • Simon Berger
      Simon Berger
      2009-02-03

      On enwik8:

      -m0=lzma2 -mmt4 -mx8
      25,213,168 bytes
      93218ms

      -m0=lzma -mmt4 -mx8
      25,209,495 bytes
      90343ms

      On a 4,627,054,592 bytes (4,3gb) sized ps2 game iso

      -m0=lzma2 -mmt4 -mx3
      4,002,901,376 bytes
      672125ms

      -m0=lzma -mmt4 -mx3
      3,999,389,768 bytes
      1873906ms

      But be careful with the timing. I did many things while processing.
      On the first file lzma2 was both: slower and bigger on the second it was a bit worse but much faster

      Addition:
      I have a Q6600 (Quad) XP SP3 4gb Ram 32bit System

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-03

        - On enwik8:

        It's too small for big dictionary/chunk.

        You can try also -mmt5 for -mx3 or -mx1 modes.

        And for -mx5/-mx7/-mx9 you can try also -mmt6.

         
      • read this: "If you don't specify ChunkSize, LZMA2 sets it to DictionarySize * 4"

        with 32mb dictionary im -mx8, chunk is 128mb

        Igor, my congratulations!

        why lzma2 is a bit worse than lzma1 in single-chunk mode? purists will never forgive you :)

         
        • Igor Pavlov
          Igor Pavlov
          2009-02-03

          LZMA2 uses subchunks (64K of compressed / 2 MB of uncompressed). For each subchunk it strores uncompressed / compressed sizes (5 bytes) and restarts RangeCoder (~5 bytes). So there is overhead of 10 bytes per 50-64 KB.

          That schema allows also multithreading decompression - just read big amount of data and walk over 64 KB subchunks headers searching new chunk start. But I don't plan to implement multithreading decompression in near future.

           
    • Hi Igor,

      www.squeezechart.com has now benchmarks with LZMA2.

      In overall score LZMA2 did slightly worse compared to LZMA, but the way LZMA2 treats compressed inputs is far better.
      If those 'incompressible' data is just stored inside a 7z archive, this way open a way for other algorithms to process it.
      If LZMA2 support real blockwise method switching, the .7z format would be open for any additional algo's such as a wavesound
      codec.

      Very well done. Please keep up the remarkable work.

      Stephan

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-06

        LZMA2 supports only LZMA  and Uncompressed. Any other thing must be in another level.

         
    • chornobyl
      chornobyl
      2009-02-06

      Does LZMA2 contains delta filter? I tested it with few wav files, and appeared that LZMA is better.
      Also i found that for wav and other badly compressible data switches lc=0 lp=1 pb=1 doing best job.

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-07

        LZMA2 doesn't conatin delta filter. Delta filter will be part of another Subblock filter. It will be released later.

         
    • dos386
      dos386
      2009-02-07

      > Alpha version of LZMA2 codec:

      COOL :-) Does it mean 4.66 will have it included ?

      BTW: http://7-zip.org/sdk.html is not updated for 4.65 :-|

       
    • tho
      tho
      2009-02-10

      Hi,

      i have some questions:

      Why can have a chunk only 256m and not more? like 1gb when I want to compress 4gb of data with 4 threads?
      Is each chunk a new block inside the archive?

      Will the memory requirements reduced for lzma2?

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-11

        You can try bigger chunk, if you use 64-bit windows.

         
    • tho
      tho
      2009-02-10

      Hi,

      i have some questions:

      Why can have a chunk only 256m and not more? like 1gb when I want to compress 4gb of data with 4 threads?
      Is each chunk a new block inside the archive?

      Will the memory requirements reduced for lzma2 compression?

       
    • tho
      tho
      2009-02-17

      If i have 2 identical uncompressable file e.g. jpgs, stores lzma2 such files twice or once?
      What happen if such files bigger than the chunksize? than the dictionary size?

       
      • Igor Pavlov
        Igor Pavlov
        2009-02-17

        It stores one copy. The distance must be less than ChunkSize and less than DictionarySize.

         
  • Levester
    Levester
    2014-07-04

    what would be the command line statement if i want to achieve the compression like it is in the picture attached
    thanks

     
    Attachments
  • Peter
    Peter
    2014-08-02

    7z a kitchen.7z -mx9 -m0=lzma2:d=64k:fb=32 -ms=4g -mmt=8 kitchen

     
  • Levester
    Levester
    2014-08-20

    Thanks a lot Peter :)