I tried to find the answer but couldn't really find a technical one. I originally asked myself what the difference between snapraid scrub and snapraid diff technically means, and later figured the question in the title might provide a mutual answer but is actually more to the point.
So I guess I understand the parity bit of snapraid, where something like a hamming code (the from-scratch RAIDlibrary) provides both error detection and recovery to a certain degree. If something in the data section changes, the equation must be run again (sync) to be on the safe side again. This process also hashes the files for integrity checks. To see which files changed intentionally I can use diff, to find files with data corruption I can use scrub. To revert intentional changes and data corruption I can use fix. To revert specific intentional changes there is snapraid fix -f $. To fix data corruption only fix --filter-error.
Is that correct so far?
If yes, I still can't get my head around the initial question. (and I'm completely making this following question up because I have additionally no clue how filesystems work)
I imagine, that if I have a filesystem like ZFS or btrfs, the fs can take care of writing updated checksums and timestamps on write after an intentional changes to a file. With snapraid in place, working on top of eg. ext4 I do get updated timestamps on change, but no updated checksums as this must be run manually (sync). Now how can the system differentiate between a intentional change to a file and plain data corruption? Is it just comparison of the recorded timestamp (in conjunction with the hashvalue during sync) with the actual timestamp provided by the filesystem? If yes, what if the data corruption happens additionally and alongside an intentional change on the same file between sync instances?
Where really is the difference between data corruption and intentional changes?
I tried to mimic a data loss with dd conv=notrunc if=/dev/zero of=/path/to/snapdata/file bs=1024 count=1024 and now diff is showing me an updated file, checkan error, and scrub says there is nothing to do. And this result confuses me.
Maybe one of you could help me understand snapraid a bit better.
Sidenote,
is snapraid check basically just a snapraid -p full scrub?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There's no way to detect changes to a file + corruption if they happen between snapraid runs. Once a file was updated "legitimately" any other changes/corruption won't be seen, it will be just a changed file.
From the manual:
"Files are identified by path and/or inode and checked by size and time-stamp. If the file size or time-stamp are different, the parity data is recomputed for the whole file. If the file is moved or renamed in the same disk, keeping the same inode, the parity is not recomputed. If the file is moved to another disk, the parity is recomputed, but the previously computed hash information is kept. "
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there,
I tried to find the answer but couldn't really find a technical one. I originally asked myself what the difference between
snapraid scrubandsnapraid difftechnically means, and later figured the question in the title might provide a mutual answer but is actually more to the point.So I guess I understand the parity bit of snapraid, where something like a hamming code (the from-scratch RAIDlibrary) provides both error detection and recovery to a certain degree. If something in the data section changes, the equation must be run again (
sync) to be on the safe side again. This process also hashes the files for integrity checks. To see which files changed intentionally I can usediff, to find files with data corruption I can usescrub. To revert intentional changes and data corruption I can usefix. To revert specific intentional changes there issnapraid fix -f $. To fix data corruption onlyfix --filter-error.Is that correct so far?
If yes, I still can't get my head around the initial question. (and I'm completely making this following question up because I have additionally no clue how filesystems work)
I imagine, that if I have a filesystem like ZFS or btrfs, the fs can take care of writing updated checksums and timestamps on write after an intentional changes to a file. With snapraid in place, working on top of eg. ext4 I do get updated timestamps on change, but no updated checksums as this must be run manually (
sync). Now how can the system differentiate between a intentional change to a file and plain data corruption? Is it just comparison of the recorded timestamp (in conjunction with the hashvalue duringsync) with the actual timestamp provided by the filesystem? If yes, what if the data corruption happens additionally and alongside an intentional change on the same file betweensyncinstances?Where really is the difference between data corruption and intentional changes?
I tried to mimic a data loss with
dd conv=notrunc if=/dev/zero of=/path/to/snapdata/file bs=1024 count=1024and nowdiffis showing me an updated file,checkan error, andscrubsays there is nothing to do. And this result confuses me.Maybe one of you could help me understand snapraid a bit better.
Sidenote,
is
snapraid checkbasically just asnapraid -p full scrub?There's no way to detect changes to a file + corruption if they happen between snapraid runs. Once a file was updated "legitimately" any other changes/corruption won't be seen, it will be just a changed file.
From the manual:
"Files are identified by path and/or inode and checked by size and time-stamp. If the file size or time-stamp are different, the parity data is recomputed for the whole file. If the file is moved or renamed in the same disk, keeping the same inode, the parity is not recomputed. If the file is moved to another disk, the parity is recomputed, but the previously computed hash information is kept. "
I just came across my own post to find out my response was never posted. It was more or less like this:
Thank you for the quick answer!