I'm trying to figure out why multithreading works the way it does in 7zip when ultra mode + LZMAv2 is selected (1GB dictionary).
Why does it seem so weird?
1 thread - 10783MB memory required
2 threads - 10789MB (why the extra 6MB?)
3 threads - 10789MB (why is this the same...?)
4-5 treads - 22573MB
6-7 threads - 33485MB
etc etc.
I always compress with 3 threads because I thought it gives the best possible performance with maximum compression, since the source data is not split into any chunks (solid archive).
I found an old comment by Igor:
for -mx5 and higher, when you specify
-mmt{N}
7-Zip creates M copies of LZMA2 encoder, where M*2=N
Each copy of LZMA2 encoder creates 3 asymmetrical threads:
1) thread 1 is 100% of CPU thread power.
1) thread 2 is 40%-70% of CPU thread power.
1) thread 3 is about 5% of CPU thread power.
So why doesn't it keep rising in sets of 3 but rather in sets of 2 threads per the next memory requirement increase? And what's with the first thread, why is it different? And lastly, it would cause reduction of compression ratio if I used more threads than 3, right? But if I use just 3 or fewer, the compression ratio is the best it can be, right?
Thanks for the answers!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So why doesn't it keep rising in sets of 3 but rather in sets of 2 threads
These 3 task threads do not give 300% of cpu load. They give 150%-190% of cpu load. So it's more close to 2 cpu load threads than to 3 threads.
2 threads and 3 threads are the same for 7z - LZMA / LZMA2 - Normal / Maximum / Ultra.
And what's with the first thread, why is it different?
Each LZMA encoder instance in multithreading mode divides the task of compression into 3 different tasks, where each task is executed in separated thread: two big threads and one small thread. The total CPU load for these 3 threads can vary from 140% to 200%.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello!
I'm trying to figure out why multithreading works the way it does in 7zip when ultra mode + LZMAv2 is selected (1GB dictionary).
Why does it seem so weird?
1 thread - 10783MB memory required
2 threads - 10789MB (why the extra 6MB?)
3 threads - 10789MB (why is this the same...?)
4-5 treads - 22573MB
6-7 threads - 33485MB
etc etc.
I always compress with 3 threads because I thought it gives the best possible performance with maximum compression, since the source data is not split into any chunks (solid archive).
I found an old comment by Igor:
So why doesn't it keep rising in sets of 3 but rather in sets of 2 threads per the next memory requirement increase? And what's with the first thread, why is it different? And lastly, it would cause reduction of compression ratio if I used more threads than 3, right? But if I use just 3 or fewer, the compression ratio is the best it can be, right?
Thanks for the answers!
These 3 task threads do not give 300% of cpu load. They give 150%-190% of cpu load. So it's more close to 2 cpu load threads than to 3 threads.
2 threads and 3 threads are the same for 7z - LZMA / LZMA2 - Normal / Maximum / Ultra.
Each LZMA encoder instance in multithreading mode divides the task of compression into 3 different tasks, where each task is executed in separated thread: two big threads and one small thread. The total CPU load for these 3 threads can vary from 140% to 200%.