How many threads does multithreading use?

2005-07-28
2014-03-27
  • Chris Connett

    Chris Connett - 2005-07-28

    I'd like to know how many threads multithreaded compression uses by default, and if there's an easy way to change it.

     
    • Igor Pavlov

      Igor Pavlov - 2005-07-28

      2 threads in most cases. You can't change it now.

       
    • Nobody/Anonymous

      What are they doing ?

      Is the decompression single-thread ?

       
      • Igor Pavlov

        Igor Pavlov - 2005-07-30

        > What are they doing ?

        compress data

        > Is the decompression single-thread ?

        Yes, LZMA is too fast.

         
    • Nobody/Anonymous

      "> What are they doing ?

      compress data"

      WOW. Would it be possible to reveal more ?
      For ex.:
      "
      1 thread LZ77
      1 thread MC & RANGE
      "

      ??

       
      • Igor Pavlov

        Igor Pavlov - 2005-08-01

        thread 1 finds matches
        thread 2: Range Encoding and some other things.

        speed of thread 1 depends from RAM
        speed of thread 2 depends from CPU (ALU, L1-L2 cache).

        Thats why hyper-threading in P4 is good for LZMA.

         
    • Nobody/Anonymous

      Thanks.

      Are there more threads if a filter is active, splitting 1
      stream into 4 of whitch 3 have to be compressed ?

       
      • Igor Pavlov

        Igor Pavlov - 2005-08-01

        Yes, but 2 of 3 threads are pretty small. And filter's thread is also simple.

         
    • Nobody/Anonymous

      Thanks.

      How many threads can MAXIMALLY occur ?

       
      • Igor Pavlov

        Igor Pavlov - 2005-08-01

        about 9:
        1 Main App
        1 BCJ2
        3 LZMA
        4 AES

         
        • Igor Pavlov

          Igor Pavlov - 2005-08-01

          Correction:
          10:
          1 Main App
          1 BCJ2
          3 LZMA + 1 MatchFinder thread
          4 AES

           
    • Nobody/Anonymous

      Thanks.

       
    • Nobody/Anonymous

      Hello all,

      I'm planning on using a 7zip to archive data on several machines, some of which will have hyperthreading and some that won't.  If I set mt=on, will the non-hyperthreading error out?

      Alternatively, if someone would like to help with the ideal set of commands for maximum compression and performance with the following given parameters, I'd appreciate it.

      Assume all systems are P4, some with hyperthreading, some without
      Assume all systems have at least 256MB ram, most have move but systems with just 256mb will still be in place for awhile.
      Assume that all system resources can be used for the time it takes to create the archive( it should be the only thing running early in the morning)

      Based on the documentation, my current command line is as follows,

      %saw%\7za a -t7z %ws_local%\%computername%2.7z -r c:\work\* -x@c:\backup\exclude.txt x=9 -m0=BCJ2 -m1=LZMA:d23 -m2=LZMA:d19 -m3=LZMA:d19 -mb0:1 -mb0s1:2 -mb0s2:3 -so >> %log%

      Thanks in advance!

      ~UJS

       
    • Nobody/Anonymous

      > the ideal set of commands for maximum compression and performance

      It depends on the characteristic of the data. PPMD may work better on some data. How large is the data set (sets)?

      There is only one ideal for each set, and you will have to do hundreds of test to find it. To find the ideal for all of them would probably require thousands of tests. And don't forget that the ideal command will change as the data changes. Go see www.maximumcompression.com. Have fun.

      Does time to compress matter? As soon as your ram fills up, you hit virtual and it becomes very inefficient.

      Your current options look fairly general and should work ok. (note: this is off-topic here.)

      As for your first question, why don't you test it?

       
    • Nobody/Anonymous

      > If I set mt=on, will the non-hyperthreading error out?
      I found that mt=on is slower in non-hyperthreading systems. I don`t see any error.

      > %saw%\7za a -t7z %ws_local%\%computername%2.7z -r c:\work\* -x@c:\backup\exclude.txt x=9 -m0=BCJ2 -m1=LZMA:d23 -m2=LZMA:d19 -m3=LZMA:d19 -mb0:1 -mb0s1:2 -mb0s2:3 -so >> %log%

      Try this:
      %saw%\7za a -t7z %ws_local%\%computername%2.7z -r c:\work\* -x@c:\backup\exclude.txt -mx=7 -m0=BCJ2 -m1=LZMA:d24:mf=bt4 -m2=LZMA:d19:mf=bt2 -m3=LZMA:d19:mf=bt2 -mb0:1 -mb0s1:2 -mb0s2:3 -so >> %log%

       
    • Nobody/Anonymous

      >Try this:
      >%saw%\7za a -t7z %ws_local%\%computername%2.7z -r c:\work\* -x@c:\backup\exclude.txt -mx=7 -m0=BCJ2 -m1=LZMA:d24:mf=bt4 -m2=LZMA:d19:mf=bt2 -m3=LZMA:d19:mf=bt2 -mb0:1 -mb0s1:2 -mb0s2:3 -so >> %log% 

      Doesn't the -mx=7 override the rest of the line? (If not, then it is redundant, isn't it?)

       
  • dos386

    dos386 - 2013-03-29

    Sorry for bumping a 10-years old thread, but I have a question left:

    Is the decompression single-thread ?

    Yes, LZMA is too fast.

    Is the complete 7-ZIP extraction always (especially if the
    archive is encrypted) single-thread?

     
    Last edit: dos386 2013-03-29
  • Igor Pavlov

    Igor Pavlov - 2013-03-30

    7-Zip uses additional thread for each method for 7z extraction: LZMA, 7zAES, BCJ, ...

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks