Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

Best LZMA2 Settings For Speed

Help
Lipin
2014-01-31
2014-02-05
  • Lipin
    Lipin
    2014-01-31

    Hello,

    I am new to LZMA SDK and I got into the comprssion properties which are the ones listed here:
    http://stackoverflow.com/questions/3057171/lzma-compression-settings-details/21384797#21384797

    My question is, which is the best combination for speed? I have used the 7ZipManager and it compressed 1.2GB in about 11 seconds but using the LZMA SDK the best time I got was 40 seconds.
    I have a powerful computer with 64GB of RAM and 16-core CPU.

    The settings I used for the 40 seconds result was:

    props.lzmaProps.level = 0;
    props.lzmaProps.writeEndMark = 1;
    props.lzmaProps.numThreads = 16;
    props.numTotalThreads = 16;
    props.numBlockThreads = 4;
    props.lzmaProps.dictSize = 1024 * 1024;
    props.lzmaProps.btMode = 0;
    props.lzmaProps.algo = 0;
    props.lzmaProps.numHashBytes = 2;
    props.lzmaProps.lc = 0;
    props.lzmaProps.lp = 4;
    props.lzmaProps.pb = 4;

    I also tried to map the settings from the 7 Zip Manager GUI to code but I didn't even nearly as close as 40 seconds...

    Can anyone help me with this?

     
    Last edit: Lipin 2014-01-31
  • Igor Pavlov
    Igor Pavlov
    2014-01-31

    call Lzma2EncProps_Init at first
    then:

    props.numTotalThreads = 16; // set 32, if you have 16 cores with hyperthreading
    props.lzmaProps.level = 5; // you can change it from 1 to 9
    

    You don't need to set another options in most cases.

     
  • Lupin
    Lupin
    2014-02-05

    Hello Igor. Thank you for your answer!

    I managed to make it use all the cores 100% and compression is as fast as I was expected it to be, however decompression doesn't use the full power of the machine.

    I am using Lzma2Enc_WriteProperties(enc); to get the properties Byte which I use on the decompression side for Lzma2Dec_Allocate(&dec, propsByte, &m_szAllocForLzma);

    Is it possible to decompress on all the cores, full power?

    Thanks a lot for your help!

     
    Last edit: Lupin 2014-02-05
  • Igor Pavlov
    Igor Pavlov
    2014-02-05

    Current lzma2 decompression code uses only single thread.

     
  • Lupin
    Lupin
    2014-02-05

    I understand. Now it's clear

    Thank you, Igor!