The order of maintenance operations in SnapRAID
A backup program for disk arrays
Brought to you by:
amadvance
Menu
▾
▴
I've noticed that many scripts run a "scrub" after a "sync". That doesn't make sense to me.
As is obvious from the initial sync operation, all files are checksummed, parity is stored and content file(s) are populated. I also understand that if there's a scrub immediately before a sync, that errors may occur due to new un-checkedsummed files that are not in the content file(s). However, just as it is with a diff operation (added, deleted, modified files, etc.), the user must make a determination for the reasons why these scrub file errors were generated.
Assuming 100% scrubs; I'd appreciate someone explaining to me why the following is the wrong approach:
-diff:
The diff tool tallies files differences and displays files added, deleted, modified, etc. Corrective actions can be taken with snapraid fix -f FILE, snapraid fix -f DIR/, etc, if needed.
-scrub: (Assume snapraid -p 100 -o 7 scrub)
Identifies silent errors, bad blocks, etc, that can be identified with snapraid status and fixed with snapraid -e fix.
Where scrub is concerned, the following is from the user manual:
Take care that running "scrub" on a not synced array may result in errors caused by removed or modified files. These errors are reported in the "scrub" result, but related blocks are not marked as bad.
My comment to the above would be; if the number of file errors matches the number of diff files changed, why would that part of scrub's output matter? It would be expected.
The main point to be made is, in the user manual, under the fix topic:
Fix all the files and the parity data. All the files and the parity data are compared with the snapshot state saved in the last "sync". If a difference is found, it's reverted to the stored snapshot. The "fix" command doesn't differentiate between errors and intentional modifications. It unconditionally reverts the file state at the last "sync".
-sync:
(Going on the behavior of the very first sync) snapraid sync resets ALL. If I understand the behavior of sync correctly, parity information and the entries in the content file(s) are fully updated.
When all is considered, I came to a couple of conclusions:
A scrub to detect and correct silent errors is necessary before a sync. (With the understanding that errors for added, modified and deleted files, equaling those noted in the diff command, will be in scrub output.) In any case, the array needs to be absolutely clean, with all added and/or deleted files accounted for, before a sync.
A scrub immediately after a sync is nearly pointless, because the sync command has reset parity for "all files". Without parity information, there is no way to "fix" anything. After a sync, the status of the SnapRAID array's parity information is at day zero. If files were corrupted before the last sync, they can't be retrieved after the next sync because their parity information is gone.
Can someone explain to me why the above is wrong?
Last edit: Floyd Maxey 2023-02-02
sync will just update/touch parity with new, modified or deleted files. it doesn't touch existing non-modified files so it is relatively quick and those files don't get "reset" . Scrub -p 100 -o 7 checks every file that hasn't been synced/touched in the last 7 days. So recommendation might be:
diff - so you can see what has changed and make sure you want to sync
sync - to sync those changes. no going back to old file versions or getting missing files that were flagged with diff.
scrub -p 10 -o 7 - make sure your files really haven't changed. If run daily every 10 days your all your files will be verified. Any errors here can potentially still be recovered from parity.
Thanks for your time and for answering.
That makes sense - if files are untouched, sync bypasses them. Thanks for the clarification.
So, may I assume that blocks marked "bad" (as in silent errors) are not affected by the sync operation? Maybe bad blocks are logged into the content file(s) or something like that? That would make sense in that a subsequent scrub, after a sync, could detect them..