I was wondering whether it'd be possible to allow the creation of an encrypted image without specifying the volume size beforehand. The file would grow dynamically and empty blocks would be "reclaimed" by shrinking the file size.
Alternatively a very big volume size could by defined, but its image files size would still grow and shrink according to the allocated data inside it.
This is the main issue I have with e.g. VeraCrypt: while you can create a sparse volume, empty space cannot be reclaimed by the volume's amount of data being reduced. Also, when moving the VC volume file elsewhere, you'll lose the advantage of sparse files altogether.
I saw that ImDisk Toolkit has a way to do something like this in-memory: by regalr use of TRIM commands, dynamic ramdisks shrink to use no more memory than is allocated in the volume. This might be translated to the image file somehow.
I realize this could "leak" some metadata about the contents of the volume, but it's hardly an issue in my book.
I'm trying to get rid of a few Pismo Private Folder files, which do have these advantages.
Last edit: Guillaume 2023-06-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There would be a big difference with dynamic ramdisks: we don't have to handle the location of a memory block. We only have to allocate it.
With an image file, we would end up with the good old issue of fragmentation.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That may be true, but fragmentation is but a small issue by now with everyone using SSDs. The only "problems" left with fragmentation I can think of is that it may leave a bit more parts of sectors unused and that the file table of whatever file system you use will somewhat increase in size.
Last edit: Guillaume 2023-06-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ah wait, do you perhaps mean here not that fragmentation of the file system per se is a problem, but that it makes the allocation, or rather mapping of image file to volume an issue? Like, for instance, some old implementations of GRUB handling booting from ISO images required the .iso file to be written 100% contiguously on disk?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, this has nothing to do with the file system.
But at the end, the same issues than a file system would appear: empty blocks in the middle, and how to remove them.
It's not impossible, but this is far too much work for this kind of tool.
An alternative would be to do something similar to VDI/VHD/VMDK image files: something that grows regularly but not shrinks. But this is doing half the work. And it's still a lot of work.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
An alternative would be to do something similar to VDI/VHD/VMDK image files: something that grows regularly but not shrinks. But this is doing half the work. And it's still a lot of work.
Yeah, and that would be pretty much like a sparse file anyway, so supporting the creation of an image files as sparse file would do exactly that, apart from the fact that moving it would require creating a new image file at the destination and then moving the data.
Shrinking files would cause it more to be like an archive, though, and definitely a USP.
With the way ImDisk works, I thought that perhaps proxycrypt resaves the entire image file anew when unmounting?
If so, perhaps the code from ImDisk Toolkit's RamDyn.exe is already enough to get this working?
I realize I'm making a lot of assumptions here, so sorry for that in advance, but I'm just brainstorming and trying to understand in the process how it actually all works.
Last edit: Guillaume 2023-06-24
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I was wondering whether it'd be possible to allow the creation of an encrypted image without specifying the volume size beforehand. The file would grow dynamically and empty blocks would be "reclaimed" by shrinking the file size.
Alternatively a very big volume size could by defined, but its image files size would still grow and shrink according to the allocated data inside it.
This is the main issue I have with e.g. VeraCrypt: while you can create a sparse volume, empty space cannot be reclaimed by the volume's amount of data being reduced. Also, when moving the VC volume file elsewhere, you'll lose the advantage of sparse files altogether.
I saw that ImDisk Toolkit has a way to do something like this in-memory: by regalr use of TRIM commands, dynamic ramdisks shrink to use no more memory than is allocated in the volume. This might be translated to the image file somehow.
I realize this could "leak" some metadata about the contents of the volume, but it's hardly an issue in my book.
I'm trying to get rid of a few Pismo Private Folder files, which do have these advantages.
Last edit: Guillaume 2023-06-21
There would be a big difference with dynamic ramdisks: we don't have to handle the location of a memory block. We only have to allocate it.
With an image file, we would end up with the good old issue of fragmentation.
That may be true, but fragmentation is but a small issue by now with everyone using SSDs. The only "problems" left with fragmentation I can think of is that it may leave a bit more parts of sectors unused and that the file table of whatever file system you use will somewhat increase in size.
Last edit: Guillaume 2023-06-21
Ah wait, do you perhaps mean here not that fragmentation of the file system per se is a problem, but that it makes the allocation, or rather mapping of image file to volume an issue? Like, for instance, some old implementations of GRUB handling booting from ISO images required the .iso file to be written 100% contiguously on disk?
Yes, this has nothing to do with the file system.
But at the end, the same issues than a file system would appear: empty blocks in the middle, and how to remove them.
It's not impossible, but this is far too much work for this kind of tool.
An alternative would be to do something similar to VDI/VHD/VMDK image files: something that grows regularly but not shrinks. But this is doing half the work. And it's still a lot of work.
Yeah, and that would be pretty much like a sparse file anyway, so supporting the creation of an image files as sparse file would do exactly that, apart from the fact that moving it would require creating a new image file at the destination and then moving the data.
Shrinking files would cause it more to be like an archive, though, and definitely a USP.
With the way ImDisk works, I thought that perhaps proxycrypt resaves the entire image file anew when unmounting?
If so, perhaps the code from ImDisk Toolkit's RamDyn.exe is already enough to get this working?
I realize I'm making a lot of assumptions here, so sorry for that in advance, but I'm just brainstorming and trying to understand in the process how it actually all works.
Last edit: Guillaume 2023-06-24