I'm currently experimenting with the hidden protection. It works perfectly on a FAT filesystem, but it seems the hidden protection triggers too early on an ext4 filesystem. Any help would be greatly appreciated!
Here's what I compared: Create a 600MB file container on my Windows machine, set hidden size to 300MB. After mounting the volume with hidden protection and filling it, the hidden protection will activate at around 290MB. Creat a 600MB partition container on my Linux machine, and set hidden size to 300MB. After mounting the partition with hidden protection and filling it, the hidden protection will activate at around 140MB.
I have seen and read the troubleshooting chapter which could be related to this problem, but the proposed solutions did not help me solve my problem:
When trying to create a hidden volume, its maximum possible size is unexpectedly small (there is much more free space than this on the outer volume).
Does the layout of the ext4 filesystem itself cause problems for the hidden allocation, causing the protection to trigger early? Or would a file container be the better option on the Linux machine?
In the project I'm working on, the hidden protection actually gets triggered way earlier, but I'm also using an overlayFS inside the outer volume. Does anyone have experience with that combination?
Edit: It seems another cause could be that I'm using Linux on an SD card. I suspect that wear leveling could trigger the hidden protection earlier than it should. Any thoughts on that?
Last edit: Andrea 2020-08-10
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
WARNING: The part below is pure speculation as I did not have time to test it, but maybe it helps. Anyone knowing better or having an indication that I am wrong or right is cordially invited to contribute.
I might be mistaken there, but as far as I understood, protection of the hidden volume is very basic in Veracrypt. Any access to a block that is occupied by the hidden volume is just blocked, and immediately after that, the entire partition is put to read-only mode (this is necessary to keep the FS from marking blocks as bad blocks, leaving a hint that there may be a hidden volume).
How long you may write data until the protection triggers is dependant on the strategy of the FS and its implementation. FAT usually starts after the root directory entries and fills linearly from the start to the end of the disk. Gaps in the allocated space are handled either by filling them immediately, not at all until there is no more space at the end left, or are used if the hole may accommodate the whole file (thats more of an unlikely theory I once heard).
Ext4, on the other hand, tries to minimize file fragmentation. Minimizing is done by keeping wherever files on the same block group (preferably the same block group as the directory is located) and speculatively allocate bigger chunks (I think I read the size of 8KB somewhere) and releasing the unused space upon file closure. This strategy is paired with putting new directories, preferably in an empty (or least used) block group. The last behavior triggers scattering throughout the system and may hypothetically be why protection kicks in earlier with FAT than with ext4. So my suspicion would be: If you are writing to the same directory, FAT and ext4 should beave roughly equal. If you write to different directories, scattering will take place in 100+MB chunks making the protection mechanism very soon kicking in.
I would choose an FS with a better allocation strategy (e.g., FAT) for the outer volume. A stupid idea would be to encapsulate the outer volume FS in a single file. However, this leaves clear evidence about an inner volume and its size (forget this idea). I am unaware of the implications, but I would guess that FS like udf (which is used for DVD RWs) or ext3 (I think the optimization in allocation were new in ext4, but I am unsure) would behave roughly the same as FAT.
Just my 2cts. Anyone else?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello everyone
I'm currently experimenting with the hidden protection. It works perfectly on a FAT filesystem, but it seems the hidden protection triggers too early on an ext4 filesystem. Any help would be greatly appreciated!
Here's what I compared:
Create a 600MB file container on my Windows machine, set hidden size to 300MB. After mounting the volume with hidden protection and filling it, the hidden protection will activate at around 290MB.
Creat a 600MB partition container on my Linux machine, and set hidden size to 300MB. After mounting the partition with hidden protection and filling it, the hidden protection will activate at around 140MB.
I have seen and read the troubleshooting chapter which could be related to this problem, but the proposed solutions did not help me solve my problem:
Does the layout of the ext4 filesystem itself cause problems for the hidden allocation, causing the protection to trigger early? Or would a file container be the better option on the Linux machine?
In the project I'm working on, the hidden protection actually gets triggered way earlier, but I'm also using an overlayFS inside the outer volume. Does anyone have experience with that combination?
Edit: It seems another cause could be that I'm using Linux on an SD card. I suspect that wear leveling could trigger the hidden protection earlier than it should. Any thoughts on that?
Last edit: Andrea 2020-08-10
WARNING: The part below is pure speculation as I did not have time to test it, but maybe it helps. Anyone knowing better or having an indication that I am wrong or right is cordially invited to contribute.
I might be mistaken there, but as far as I understood, protection of the hidden volume is very basic in Veracrypt. Any access to a block that is occupied by the hidden volume is just blocked, and immediately after that, the entire partition is put to read-only mode (this is necessary to keep the FS from marking blocks as bad blocks, leaving a hint that there may be a hidden volume).
How long you may write data until the protection triggers is dependant on the strategy of the FS and its implementation. FAT usually starts after the root directory entries and fills linearly from the start to the end of the disk. Gaps in the allocated space are handled either by filling them immediately, not at all until there is no more space at the end left, or are used if the hole may accommodate the whole file (thats more of an unlikely theory I once heard).
Ext4, on the other hand, tries to minimize file fragmentation. Minimizing is done by keeping wherever files on the same block group (preferably the same block group as the directory is located) and speculatively allocate bigger chunks (I think I read the size of 8KB somewhere) and releasing the unused space upon file closure. This strategy is paired with putting new directories, preferably in an empty (or least used) block group. The last behavior triggers scattering throughout the system and may hypothetically be why protection kicks in earlier with FAT than with ext4. So my suspicion would be: If you are writing to the same directory, FAT and ext4 should beave roughly equal. If you write to different directories, scattering will take place in 100+MB chunks making the protection mechanism very soon kicking in.
I would choose an FS with a better allocation strategy (e.g., FAT) for the outer volume. A stupid idea would be to encapsulate the outer volume FS in a single file. However, this leaves clear evidence about an inner volume and its size (forget this idea). I am unaware of the implications, but I would guess that FS like udf (which is used for DVD RWs) or ext3 (I think the optimization in allocation were new in ext4, but I am unsure) would behave roughly the same as FAT.
Just my 2cts. Anyone else?