Thank you for that! I understand you haven't agreed to this tweak but I'm hopeful that you will. If it helps, I'm willing to write the code changes necessary and submit them as a suggestion.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the prompt reply! I have tried the -snld switch and I'm observing the following:
I can now extract links that go up a directory level but remain inside the archive (thank you!)
Links that point outside the archive still generate an error message and cause the archive to not be extracted. This means that archives that I could extract with older versions of 7z are no longer usable with the new version :( It would be great if there was a switch that allowed these "outside" archive links to just issue a warning and not be created but the archive to be otherwise restored. Is this something you would be willing to entertain?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the prompt reply! I have tried the -snld switch and I'm observing the following: [...] archives that I could extract with older versions of 7z are no longer usable with the new version :( It would be great if there was a switch that allowed these "outside" archive links to just issue a warning and not be created but the archive to be otherwise restored. Is this something you would be willing to entertain?
It is really a shame that I am prescribed how I may use my PC with which RAM utilization! I want to be allowed to determine for myself whether I want to use a 2-3 GB dictionary or not! So I can't use 7-ZIP anymore, because I often use "these oh so dangerous" dictionary sizes regularly and that since years! They could have at least provided this with a warning instead of blocking this altogether! Especially with 3rd party extensions, which I'm sure a large amount of users use, this is just horribly horrible! I will now uninstall this version and reinstall the version before ... and if it stays like this, I'll stay with the version before! This restriction I will absolutely not accept!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I can see why the author (Igor) wants this change, as it is true that once 7-Zip starts hitting your pagefile, performance grinds to a halt.
But having a hard-limit that only a certain maximum dictionary-size can be used based on the physical RAM in the machine is also silly, as it doesn't take into account the file or folder size.
Might I suggest the issue here is that 7-Zip does not pre-allocate its RAM at the operation start -- which means if the user were to open other apps, 7-Zip could then run-out of RAM mid-operation.
As you can see, I did a test and my RAM-use went from 2.4GB at the start...
...to 5.9GB at the end:
In contrast, WinRAR pre-allocates (reserves) all the RAM it needs at the operation start:
And by the end, this has barely changed:
As a better solution here, why not just make 7-Zip also reserve all the RAM it will need at the start of the operation, so that opening other apps cannot cause it to run-out?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
7-Zip fills dictionary stuctures, while it compresses data.
When the dictionary will be filled, 7-Zip will not use more memory.
Another apps are not big problem for memory swap in many cases. The problem is 7-Zip pattern of RAM usage. 7-Zip uses random accesses to whole allocated memory. That is why swap will be slow in 7-Zip, if we exceed RAM size.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The operation was blocked by 7-Zip
The operation can require big amount of RAM (memory):
0 MB : Memory usage for Compressing:
16265 MB : RAM
15232 MB : 7-Zip limit
The system cannot allocate the required amount of memory
Это сообщение появляется при попытке сжатия файла с использованием сторонних плагинов. Попробовал в меру своих знаний плюсов почитать код CompressDialog.cpp и у меня сложилось такое понимание:
1) Внутри CCompressDialog::OnOK происходит сверка доступной памяти и памяти, требуемой для выбранного алгоритма. Само сообщение появляется только если memUsage > maxRamSize. Вычисление memUsage происходит в функции GetMemoryUsage_Dict_DecompMem.
2) Внутри GetMemoryUsage_Dict_DecompMem есть case с теми вариантами, которые знакомы архиватору, в противном случае возвращается (UInt64)(Int64)-1
3) Внутри SetErrorMessage_MemUsage (UInt64)(Int64)-1 преобразуется в строковую форму через вызов AddSize_MB
4) Не силен в плюсах, но у меня сложилось впечатление, что внутри AddSize_MB в преобразовании (size + (1 << 20) - 1) >> 20 наш (UInt64)(Int64)-1 превращается в ноль, который я и вижу в сообщении.
Все ли я правильно понял? Если да, то получается, что для сторонних плагинов единственный способ упаковать файл - это выбрать уровень сжатия Store, поскольку для этого уровня GetMemoryUsage_Dict_DecompMem вернет 1 << 20, и проверка пройдет успешно.
Last edit: Dec 2021-10-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Pavel, maybe it makes sense to add a special parameter to override memory restrictions, so advanced users would have a choice?
Some users may have swap located on very fast devices (as it was pointed here), others may be just OK if compression will take hours or days.
Let's give users a choice, leaving default memory amount check enabled if no special parameter is used. I think it could be a good compromise.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Removed - somehow ended up posting this in the wrong thread. Sorry about that.
Last edit: Halldor Fannar 2021-09-29
Thank you for that! I understand you haven't agreed to this tweak but I'm hopeful that you will. If it helps, I'm willing to write the code changes necessary and submit them as a suggestion.
Thanks for the prompt reply! I have tried the
-snldswitch and I'm observing the following:I can now extract links that go up a directory level but remain inside the archive (thank you!)
Links that point outside the archive still generate an error message and cause the archive to not be extracted. This means that archives that I could extract with older versions of 7z are no longer usable with the new version :( It would be great if there was a switch that allowed these "outside" archive links to just issue a warning and not be created but the archive to be otherwise restored. Is this something you would be willing to entertain?
@ipavlov , yes, please!
It is really a shame that I am prescribed how I may use my PC with which RAM utilization! I want to be allowed to determine for myself whether I want to use a 2-3 GB dictionary or not! So I can't use 7-ZIP anymore, because I often use "these oh so dangerous" dictionary sizes regularly and that since years! They could have at least provided this with a warning instead of blocking this altogether! Especially with 3rd party extensions, which I'm sure a large amount of users use, this is just horribly horrible! I will now uninstall this version and reinstall the version before ... and if it stays like this, I'll stay with the version before! This restriction I will absolutely not accept!
I can see why the author (Igor) wants this change, as it is true that once 7-Zip starts hitting your pagefile, performance grinds to a halt.
But having a hard-limit that only a certain maximum dictionary-size can be used based on the physical RAM in the machine is also silly, as it doesn't take into account the file or folder size.
Might I suggest the issue here is that 7-Zip does not pre-allocate its RAM at the operation start -- which means if the user were to open other apps, 7-Zip could then run-out of RAM mid-operation.
As you can see, I did a test and my RAM-use went from 2.4GB at the start...
...to 5.9GB at the end:
In contrast, WinRAR pre-allocates (reserves) all the RAM it needs at the operation start:
And by the end, this has barely changed:
As a better solution here, why not just make 7-Zip also reserve all the RAM it will need at the start of the operation, so that opening other apps cannot cause it to run-out?
7-Zip fills dictionary stuctures, while it compresses data.
When the dictionary will be filled, 7-Zip will not use more memory.
Another apps are not big problem for memory swap in many cases. The problem is 7-Zip pattern of RAM usage. 7-Zip uses random accesses to whole allocated memory. That is why swap will be slow in 7-Zip, if we exceed RAM size.
GUI -> (F3 Key) View --> Open selected item with viewer
How to view text file in archive from command line ?
or
9900K @ 5000 MHz + RAM 4000 MHz
See picture attached if anyone interested.
The same: 9900K @ 5000 MHz + RAM 4000 MHz, WSL1 on Win10Pro:
transparent_hugepage probably is disabled.
please call
Хотелось бы разобраться с сообщением
The operation was blocked by 7-Zip
The operation can require big amount of RAM (memory):
0 MB : Memory usage for Compressing:
16265 MB : RAM
15232 MB : 7-Zip limit
The system cannot allocate the required amount of memory
Это сообщение появляется при попытке сжатия файла с использованием сторонних плагинов. Попробовал в меру своих знаний плюсов почитать код CompressDialog.cpp и у меня сложилось такое понимание:
1) Внутри CCompressDialog::OnOK происходит сверка доступной памяти и памяти, требуемой для выбранного алгоритма. Само сообщение появляется только если memUsage > maxRamSize. Вычисление memUsage происходит в функции GetMemoryUsage_Dict_DecompMem.
2) Внутри GetMemoryUsage_Dict_DecompMem есть case с теми вариантами, которые знакомы архиватору, в противном случае возвращается (UInt64)(Int64)-1
3) Внутри SetErrorMessage_MemUsage (UInt64)(Int64)-1 преобразуется в строковую форму через вызов AddSize_MB
4) Не силен в плюсах, но у меня сложилось впечатление, что внутри AddSize_MB в преобразовании (size + (1 << 20) - 1) >> 20 наш (UInt64)(Int64)-1 превращается в ноль, который я и вижу в сообщении.
Все ли я правильно понял? Если да, то получается, что для сторонних плагинов единственный способ упаковать файл - это выбрать уровень сжатия Store, поскольку для этого уровня GetMemoryUsage_Dict_DecompMem вернет 1 << 20, и проверка пройдет успешно.
Last edit: Dec 2021-10-04
It will be changed in next version of 7-zip.
I suppose that that change must solve these problems and some another problems.
Pavel, maybe it makes sense to add a special parameter to override memory restrictions, so advanced users would have a choice?
Some users may have swap located on very fast devices (as it was pointed here), others may be just OK if compression will take hours or days.
Let's give users a choice, leaving default memory amount check enabled if no special parameter is used. I think it could be a good compromise.