Is there a way to set the amount of memory which will 7z use ?
I try to set -mmemuse parameter, but it does not work.
It is important for me, because at this moment compression by 7zip takes about 120GB of memory and it is about 50% of all RAM for creating 230GB zip file.
In near future the free memory will be reduced to 70GB and this way the compression will probably takes all of them.
So I need set the max memory limit for 7z zip process.
Regards
ML
Last edit: FCS 2024-09-02
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Write about 7-zip options/settings/switches that you use.
And how did you detect that it consumes 120GB of memory?
Probably you was wrong, or you used wrong options.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The problem I described in first post is on Win 2022 server, where I'm testing 7z for creating full and differential backup of users data (240 GB).
This line creates full backup.
This problem occurs on my local computer too, where I test creating of backup of 2GB files.
It is visible in the Windows Task manager. In both places (with different windows systems) the used memory grows (on server into 50% of RAM, on my computer about 2GB).
In both cases the release of taken RAM takes a while (on server about 10 min, on my computer about 10 sec) but it depends of size o the archive and amount of reserved RAM during compression.
Adding -mmemuse parameter on my local computer takes no effect:
Your conclusions are wrong.
Windows caches data of any program that reads or writes files.
So if that data will be requested again, windows will not use drive IO. Instead Windows will read data from cache.
And windows decides how much it can allocate for that file cache.
It's possible to disable cache for reading/writing. But file cache is good thing in most cases, if there is free space of RAM,
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your suggestions Sam.
I found those pages too.
Igor, I know that windows is buffering files under read and write, calculating the assigned buffer for this, usually it grows up to 50% of physical memory, but if physical memory is used in 70% it takes all the rest of memory 30% (finally 100% of it), not 50% of the free 30% , and it leads to system problems.
Reading posts on this forum I found the mmemuse parameter with context of allocating memory for 7zip, but in test it didn't give any effects, so this post from me.
In a post on this forum some one wrote about problem with "Out of memory" and system hanging during compression of big set of data.
It potentially could occurs for me too, so I would like to set the max memory available for 7z during compression.
Something like xcopy /j parameter, or Total Commnder custom sets for big files copping.
Have you more detailed description of all 7zip parameters (chm and cmd 7z help does not show mmemuse parameter but you was talking about it on this forum, so it exists)?
Regards
ML
Ps (off topic)
Is there a way to save parameters of compression set in 7z_FM into a file, to use them for batch process ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
7-zip doesn't allocate big memory for that zip operation.
Windows allocates big memory for file cache. -mmemuse switch is for 7z format, where 7-zip can allocate memory by internal puproses.
If you open Task Manager / Details and select memory usage columns, you will see how much 7-zip allocates memory for any operation.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
Is there a way to set the amount of memory which will 7z use ?
I try to set -mmemuse parameter, but it does not work.
It is important for me, because at this moment compression by 7zip takes about 120GB of memory and it is about 50% of all RAM for creating 230GB zip file.
In near future the free memory will be reduced to 70GB and this way the compression will probably takes all of them.
So I need set the max memory limit for 7z zip process.
Regards
ML
Last edit: FCS 2024-09-02
Write about 7-zip options/settings/switches that you use.
And how did you detect that it consumes 120GB of memory?
Probably you was wrong, or you used wrong options.
Hello,
I use this command:
"C:\Program Files\7-zip\7z.exe" a -tzip -mx=0 -r -bb1 H:\kopia\full E:!_Siec* 1>log.txt 2>err.txt
The problem I described in first post is on Win 2022 server, where I'm testing 7z for creating full and differential backup of users data (240 GB).
This line creates full backup.
This problem occurs on my local computer too, where I test creating of backup of 2GB files.
It is visible in the Windows Task manager. In both places (with different windows systems) the used memory grows (on server into 50% of RAM, on my computer about 2GB).
In both cases the release of taken RAM takes a while (on server about 10 min, on my computer about 10 sec) but it depends of size o the archive and amount of reserved RAM during compression.
Adding -mmemuse parameter on my local computer takes no effect:
"C:\Program Files\7-zip\7z.exe" a -tzip -mx=0 -r -bb1 -mmemuse=500M H:\kopia\full E:!_Siec* 1>log.txt 2>err.txt
Regards
ML
Your conclusions are wrong.
Windows caches data of any program that reads or writes files.
So if that data will be requested again, windows will not use drive IO. Instead Windows will read data from cache.
And windows decides how much it can allocate for that file cache.
It's possible to disable cache for reading/writing. But file cache is good thing in most cases, if there is free space of RAM,
You can find some more references here, check NT Cache Setter tool, and read this.
Hello,
Thanks for your suggestions Sam.
I found those pages too.
Igor, I know that windows is buffering files under read and write, calculating the assigned buffer for this, usually it grows up to 50% of physical memory, but if physical memory is used in 70% it takes all the rest of memory 30% (finally 100% of it), not 50% of the free 30% , and it leads to system problems.
Reading posts on this forum I found the mmemuse parameter with context of allocating memory for 7zip, but in test it didn't give any effects, so this post from me.
In a post on this forum some one wrote about problem with "Out of memory" and system hanging during compression of big set of data.
It potentially could occurs for me too, so I would like to set the max memory available for 7z during compression.
Something like xcopy /j parameter, or Total Commnder custom sets for big files copping.
Have you more detailed description of all 7zip parameters (chm and cmd 7z help does not show mmemuse parameter but you was talking about it on this forum, so it exists)?
Regards
ML
Ps (off topic)
Is there a way to save parameters of compression set in 7z_FM into a file, to use them for batch process ?
7-zip doesn't allocate big memory for that zip operation.
Windows allocates big memory for file cache.
-mmemuseswitch is for 7z format, where 7-zip can allocate memory by internal puproses.If you open Task Manager / Details and select memory usage columns, you will see how much 7-zip allocates memory for any operation.
Hello,
You right, the 7z.exe process takes small amount of RAM, I have checked it.
The potential problem is Windows caching I/O disk operations.
Did you test 7z behaviour in situation I described in the previous post ?
Is it possible on current version the "out of memory" problem ?
Regards
ML
-tzip -mx=0doesn't consume any memory.-tzipwithout-mx=0consumes small amount of memory: about 64 MB per one cpu thread.