While moving a large chunk of files (100's of GBs) to a newly created VeraCrypt 3TB container I found that even after the copy had finished I was left with less than 200MB of usable memory. I attempted to give it time to catch up but it never releases on its own. Please note that according to the Windows 7 x64 Resource monitor this memory was 'In Use' and not of the Standby(cached) variety that could be cleared and used if needed. In the end only dismounting the container 'freed' the memory. So far 'reading' the data has not produced this issue- only writing.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for this feedback.
We don't see yet where this huge memory consumption could come from. The only place to look is the device driver but so far we didn't find anything wrong with its memory management especially knowing that read and write operations share the same memory management logic.
In the Resource Monitor, did you check the "Memory" panel to see if any listed process was having a huge "Working Set" value? Maybe a process (antivirus?) is leaving open memory handles on the files copied to the volume and they are freed when the volume is ejected.
Anyway, we'll try to replicate the issue from our side and we'll keep you updated.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2014-09-29
The private, shareable and working bytes are within expected limits for all programs. There simply seems to be a large chunk (GBs) that is 'in use' but not accounted for in the list yet gets released when the drive is unmounted. I am testing the NOD AV 8 Beta, maybe I should uninstall it temporarily to see if it might be the cause. I'll check this out soon.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2014-09-29
I uninstalled NOD with no effect but I began looking at the other software used and payed closer attention to a few things. I found the culprit and it is not VeraCrypt. I verified this by using a different container on another drive and copying files to and from it. It turns out the issue is caused by the disk encryption software used on my larger drive where the veracrypt container was created. Hopefully they can fix it soon so I don't have to spend weeks copying files back and forth in order to replace it! /sigh
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just another hint for this issue. Some research on the internet show that this behavior is quiet common under Windows and there is a KB article about it although it's for Windows Server 2008/Vista : http://support2.microsoft.com/kb/976618/en-us
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2014-10-23
Sorry for the long delay. Been rather busy but it worked out as I was able to give the other company several weeks to respond to my bug report. (Never happened)
I appreciate the information but I felt it was better to find a setup that doesn't have this issue rather than 'work around' it. I've replaced the offending FDE software and recreated a new veracrypt container. It no longer has this issue with the new setup.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just to update this topic, there has been a discussion about this on VeraCrypt forum on Codeplex and while the workaround described here works, we're able to determine that a possible culprit is the use of a cascade algorithm: because of bad performance of data encryption, Windows tends to use too much cache to store intermediate data in order to make the copy operation appear quick to the user. When using hardware accelerate AES, this Windows cache issue doesn't seem to occur.
Here is the link of the complete discussion: https://veracrypt.codeplex.com/discussions/572258
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
While moving a large chunk of files (100's of GBs) to a newly created VeraCrypt 3TB container I found that even after the copy had finished I was left with less than 200MB of usable memory. I attempted to give it time to catch up but it never releases on its own. Please note that according to the Windows 7 x64 Resource monitor this memory was 'In Use' and not of the Standby(cached) variety that could be cleared and used if needed. In the end only dismounting the container 'freed' the memory. So far 'reading' the data has not produced this issue- only writing.
Thank you for this feedback.
We don't see yet where this huge memory consumption could come from. The only place to look is the device driver but so far we didn't find anything wrong with its memory management especially knowing that read and write operations share the same memory management logic.
In the Resource Monitor, did you check the "Memory" panel to see if any listed process was having a huge "Working Set" value? Maybe a process (antivirus?) is leaving open memory handles on the files copied to the volume and they are freed when the volume is ejected.
Anyway, we'll try to replicate the issue from our side and we'll keep you updated.
The private, shareable and working bytes are within expected limits for all programs. There simply seems to be a large chunk (GBs) that is 'in use' but not accounted for in the list yet gets released when the drive is unmounted. I am testing the NOD AV 8 Beta, maybe I should uninstall it temporarily to see if it might be the cause. I'll check this out soon.
I uninstalled NOD with no effect but I began looking at the other software used and payed closer attention to a few things. I found the culprit and it is not VeraCrypt. I verified this by using a different container on another drive and copying files to and from it. It turns out the issue is caused by the disk encryption software used on my larger drive where the veracrypt container was created. Hopefully they can fix it soon so I don't have to spend weeks copying files back and forth in order to replace it! /sigh
Thanks for this update.
Just another hint for this issue. Some research on the internet show that this behavior is quiet common under Windows and there is a KB article about it although it's for Windows Server 2008/Vista : http://support2.microsoft.com/kb/976618/en-us
Uwe Sieber implemented a tool called SetSystemFileCacheSize (this is the name of the function Microsoft talks about in the KB above) and everybody say that it solve their issue. You can get it here : http://www.uwe-sieber.de/ntcacheset_e.html#SetSystemFileCacheSize
A typical command line that works : SetSystemFileCacheSize off 512
Please let us know if this solves your issue.
Last edit: Mounir IDRASSI 2014-09-29
Sorry for the long delay. Been rather busy but it worked out as I was able to give the other company several weeks to respond to my bug report. (Never happened)
I appreciate the information but I felt it was better to find a setup that doesn't have this issue rather than 'work around' it. I've replaced the offending FDE software and recreated a new veracrypt container. It no longer has this issue with the new setup.
Just to update this topic, there has been a discussion about this on VeraCrypt forum on Codeplex and while the workaround described here works, we're able to determine that a possible culprit is the use of a cascade algorithm: because of bad performance of data encryption, Windows tends to use too much cache to store intermediate data in order to make the copy operation appear quick to the user. When using hardware accelerate AES, this Windows cache issue doesn't seem to occur.
Here is the link of the complete discussion: https://veracrypt.codeplex.com/discussions/572258