Is it possible for future versions of VeraCrypt to verify the encryption has been correctly written to or read from the storage device e.g. by using checksums? Could this method prevent tampering with the encryption itself?
The Windows NTFS file system does not detect silent data corruption and neither does Linux ext4. Yes, this may change if future versions of Windows use ReFS as the default file system, and if Linux and MacOS were to use similar error-checking file systems in future e.g. Btrfs, ZFS etc. Until then, encrypted files or containers could be damaged even if one bit of data gets corrupted, resulting in loss of files.
Last edit: Dave 2019-07-02
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is difficult to implement for partition or system encryption since VeraCrypt is acting as a simple translator. Adding in an extra checksum would require a place to put all the checksums data. There's no "extra" space in each sector to store checksums. It would have to go into the sector's data.
Besdies, tampering with encrypted data is usually pretty easy to detect. Someone who doesn't have the password will only see random hash. They can't tell what files are where, so there is no real way to target particular data. And an adversary can't change the data so that it decrypts to something intelligible. So there is no way for an adversary, even if they knew what exact portion of a hard drive stored a particular file, to make that file say something different. All they could do is make that file have random garbage in it.
Thus the threat to your data's integrity is pretty low.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That caveat is correct, but not something that VeraCrypt can or necessarily should try and protect against.
First of all, the difficulty in implementing checksums - where do you put them? If you are speaking of an encrypted container, then it's doable. But if it's an encrypted partition or drive, where you are encrypting each sector transparently, you can't just reduce each sector size by 16 bytes. It's not an insurmountable challenge, but one that adds a large level of complexity.
Before adding htat level of complexity you have to consider the threat model you are defending against. The threat model that something like VeraCrypt is intended to defend against is a known semi-permanent total loss of control of your computer. Situations where your computer is lost, stolen, or seized by officials and where you know it is. If there is an ongoing covert effort going on where you are unaware that dedicated adversaries are gaining control of your computer in periods where you are unaware, or If you are in an environment where an actual adversary can intercept data going to and from storage on an ongoing basis, then you have far worse problems than data authenticity and integrity. If you are under that sort of threat, then simple surveillance will gain an adverary your password. Physical key loggers, replacing the camera in your laptop with one that is always on and transmits, replacing the VeraCrypt boot loader with one that logs your password, or just sticking an internal USB device where it's not obvious.
This is why something like VeraCrypt doesn't even try and maintain data integrity. It would be "mommy medicine" - something that would give a false sense of security but wouldn't actually protect against being in that kind of threat environment.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes that's true. If an attacker gains physical access to the computer, they can modify it in any way to intercept the password, keyfile etc. VeraCrypt (or any other software) cannot physically secure a computer.
With regard to checksums, could the loss of file data be (much) worse with a layer of encryption? Minor corruption may still allow a file to be opened and edited e.g. a text document saved on an unencrypted disk may suffer minor corruption, but still allow editing for repair, likewise JPEG files could have minor damage but remain mostly intact. If the file is encrypted, I wonder if the same (minor) data corruption could damage the whole file? In the case of VeraCrypt containers, lots of files could be lost forever (if the container was not backed up)!
Unfortunately, NTFS does not guard against silent data corruption in the same way as ZFS, Btrfs, ReFS etc. Also, checksums require the computer’s RAM memory to be working properly. Home computers do not have (or support) ECC RAM, so any checksumming would need to verify data integrity in RAM.
As for where the checksums could be stored, I don’t know if VeraCrypt could write this data to an extra disk sector for each file or create its own file with checksums?
Last edit: Dave 2019-07-05
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As for where the checksums could be stored, I don’t know if VeraCrypt could write this data to an extra disk sector for each file or create its own file with checksums?
The difficulty lies in the way VeraCrypt works. It's not file-based encryption. VeraCrypt isn't even aware of files or filesystems. It is a block encryptor and it provides a block device. It acts as an intermediary between the operating system and the disk drive. The operating system has the filesystem code that understands how sectors go together to make files. VeraCrypt is just responding to when the operating system asks for the disk to provide a certain disk sector. VeraCrypt dutifully reads the sector, decrypts it, then gives it to the OS which then says, oh, ok, this belongs in this file. The same way, when you write to a file the OS figures out what sectors needs to be physically written to, sends that to VeraCrypt which then encrypts each one and sends them to the disk not knowing (or caring) what those sectors actually belong to. The OS sends it 512 bytes to write in one physical sector, VeraCrypt encrypts it and then writes that 512 encrypted bytes to that sector. No room in the sector that's being written for extra checksum data. You'd have to have VeraCrypt become aware of the underlying filesystem, or else you'd have to provide it with raw space to use in an otherwise unused partition, or a file to use somewhere (which would then have to be outside the encryption scope or else you have race conditions). It's complex considerations for little real-world gain.
In Linux you have file-based encryption with products like EncFS. It is able to use the underlying filesystem to add on intialization vector data to the encrypted written-to-disk file that is filtered out when it provides the decrypted data. Every actual file is 16 bytes larger than then decrypted version. This system is a good candidate for adding checksum-style integrity checking.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I guess the only long term solution is to wait for ReFS to become the mainstream file system in future versions of Windows 10 (apparently the "last" version of Windows)?
Work is underway to port (Open)ZFS to Windows, but not by Microsoft and I don't think it’s possible to convert an existing NTFS partition to ZFS?
If ZFS on Windows becomes a reality in future, hopefully VeraCrypt could support it? ZFS = Verified encryption and guard agaist almost all forms data corruption.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Does anyone know a third party software that checks CRC or better SHA256 checksum ?? https://www.veracrypt.fr/en/Authenticity%20and%20Integrity.html
that must be done in RAM when copying 2 hards drives to each other and must be quite long...I guess one must have an original snapshot (to refer to) before starting
Last edit: infodan36 2020-05-14
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Is it possible for future versions of VeraCrypt to verify the encryption has been correctly written to or read from the storage device e.g. by using checksums? Could this method prevent tampering with the encryption itself?
The Windows NTFS file system does not detect silent data corruption and neither does Linux ext4. Yes, this may change if future versions of Windows use ReFS as the default file system, and if Linux and MacOS were to use similar error-checking file systems in future e.g. Btrfs, ZFS etc. Until then, encrypted files or containers could be damaged even if one bit of data gets corrupted, resulting in loss of files.
Last edit: Dave 2019-07-02
This is difficult to implement for partition or system encryption since VeraCrypt is acting as a simple translator. Adding in an extra checksum would require a place to put all the checksums data. There's no "extra" space in each sector to store checksums. It would have to go into the sector's data.
Besdies, tampering with encrypted data is usually pretty easy to detect. Someone who doesn't have the password will only see random hash. They can't tell what files are where, so there is no real way to target particular data. And an adversary can't change the data so that it decrypts to something intelligible. So there is no way for an adversary, even if they knew what exact portion of a hard drive stored a particular file, to make that file say something different. All they could do is make that file have random garbage in it.
Thus the threat to your data's integrity is pretty low.
I was wondering if checksums would offer a solution to this problem:
https://www.veracrypt.fr/en/Authenticity%20and%20Integrity.html
That caveat is correct, but not something that VeraCrypt can or necessarily should try and protect against.
First of all, the difficulty in implementing checksums - where do you put them? If you are speaking of an encrypted container, then it's doable. But if it's an encrypted partition or drive, where you are encrypting each sector transparently, you can't just reduce each sector size by 16 bytes. It's not an insurmountable challenge, but one that adds a large level of complexity.
Before adding htat level of complexity you have to consider the threat model you are defending against. The threat model that something like VeraCrypt is intended to defend against is a known semi-permanent total loss of control of your computer. Situations where your computer is lost, stolen, or seized by officials and where you know it is. If there is an ongoing covert effort going on where you are unaware that dedicated adversaries are gaining control of your computer in periods where you are unaware, or If you are in an environment where an actual adversary can intercept data going to and from storage on an ongoing basis, then you have far worse problems than data authenticity and integrity. If you are under that sort of threat, then simple surveillance will gain an adverary your password. Physical key loggers, replacing the camera in your laptop with one that is always on and transmits, replacing the VeraCrypt boot loader with one that logs your password, or just sticking an internal USB device where it's not obvious.
This is why something like VeraCrypt doesn't even try and maintain data integrity. It would be "mommy medicine" - something that would give a false sense of security but wouldn't actually protect against being in that kind of threat environment.
Yes that's true. If an attacker gains physical access to the computer, they can modify it in any way to intercept the password, keyfile etc. VeraCrypt (or any other software) cannot physically secure a computer.
With regard to checksums, could the loss of file data be (much) worse with a layer of encryption? Minor corruption may still allow a file to be opened and edited e.g. a text document saved on an unencrypted disk may suffer minor corruption, but still allow editing for repair, likewise JPEG files could have minor damage but remain mostly intact. If the file is encrypted, I wonder if the same (minor) data corruption could damage the whole file? In the case of VeraCrypt containers, lots of files could be lost forever (if the container was not backed up)!
Unfortunately, NTFS does not guard against silent data corruption in the same way as ZFS, Btrfs, ReFS etc. Also, checksums require the computer’s RAM memory to be working properly. Home computers do not have (or support) ECC RAM, so any checksumming would need to verify data integrity in RAM.
As for where the checksums could be stored, I don’t know if VeraCrypt could write this data to an extra disk sector for each file or create its own file with checksums?
Last edit: Dave 2019-07-05
The difficulty lies in the way VeraCrypt works. It's not file-based encryption. VeraCrypt isn't even aware of files or filesystems. It is a block encryptor and it provides a block device. It acts as an intermediary between the operating system and the disk drive. The operating system has the filesystem code that understands how sectors go together to make files. VeraCrypt is just responding to when the operating system asks for the disk to provide a certain disk sector. VeraCrypt dutifully reads the sector, decrypts it, then gives it to the OS which then says, oh, ok, this belongs in this file. The same way, when you write to a file the OS figures out what sectors needs to be physically written to, sends that to VeraCrypt which then encrypts each one and sends them to the disk not knowing (or caring) what those sectors actually belong to. The OS sends it 512 bytes to write in one physical sector, VeraCrypt encrypts it and then writes that 512 encrypted bytes to that sector. No room in the sector that's being written for extra checksum data. You'd have to have VeraCrypt become aware of the underlying filesystem, or else you'd have to provide it with raw space to use in an otherwise unused partition, or a file to use somewhere (which would then have to be outside the encryption scope or else you have race conditions). It's complex considerations for little real-world gain.
In Linux you have file-based encryption with products like EncFS. It is able to use the underlying filesystem to add on intialization vector data to the encrypted written-to-disk file that is filtered out when it provides the decrypted data. Every actual file is 16 bytes larger than then decrypted version. This system is a good candidate for adding checksum-style integrity checking.
Thanks for explaining this so well. :)
I guess the only long term solution is to wait for ReFS to become the mainstream file system in future versions of Windows 10 (apparently the "last" version of Windows)?
Work is underway to port (Open)ZFS to Windows, but not by Microsoft and I don't think it’s possible to convert an existing NTFS partition to ZFS?
See: https://github.com/openzfsonwindows/ZFSin
If ZFS on Windows becomes a reality in future, hopefully VeraCrypt could support it? ZFS = Verified encryption and guard agaist almost all forms data corruption.
Hi
Does anyone know a third party software that checks CRC or better SHA256 checksum ??
https://www.veracrypt.fr/en/Authenticity%20and%20Integrity.html
that must be done in RAM when copying 2 hards drives to each other and must be quite long...I guess one must have an original snapshot (to refer to) before starting
Last edit: infodan36 2020-05-14