Does `sync` verify existing data parity of affected files before updating parity information with changes?

[Supercom32]

Hi Everyone,

When you perform a sync operation, does snapraid verify that the data your building your parity information against is error free, before it actually performs the operation? Or will sync blindly recompute parity, without first checking that the existing files are error free?

Essentially, I was wondering if scrubbing or checking for errors was something that has to be done first before performing a sync operation. I'd hate to be in a situation where sync backs up silent errors introduced at some point?

[Leifi Plomeros]

When you perform a sync operation, does snapraid verify that the data your building your parity information against is error free, before it actually performs the operation?

Yes, this is true for all previously synced data.
It is not true for any data which contains corruption before sync or data which is corrupted during sync.

It is logically impossible for Snapraid to discover any corruption that existed before sync.

Corruption that occurs during sync can be detected:
Before corrupted file is added to array via snapraid sync --pre-hash option.
immediately after corrupted file was added via snapraid scrub -p new option.

Both these options results in all new data being read twice and therefore takes twice as much time as not doing it.

-h, --pre-hash
In "sync" runs a preliminary hashing phase of all the new data to have an additional verification before the parity computation. Usually in "sync" no preliminary hashing is done, and the new data is hashed just before the parity computation when it's read for the first time. Unfortunately, this process happens when the system is under heavy load, with all disks spinning and with a busy CPU. This is an extreme condition for the machine, and if it has a latent hardware problem, it's possible to encounter silent errors what cannot be detected because the data is not yet hashed. To avoid this risk, you can enable the "pre-hash" mode and have all the data read two times to ensure its integrity. This option also verifies the files moved inside the array, to ensure that the move operation went successfully, and in case to block the sync and to allow to run a fix operation. This option can be used only with "sync".

5.7 scrub
Scrubs the array, checking for silent or input/output errors in data and parity disks.
For each command invocation, about the 8% of the array is checked, but nothing that was already scrubbed in the last 10 days. This means that scrubbing once a week, every bit of data is checked at least one time every three months.

You can define a different scrub plan or amount using the -p, --plan option that takes as argument: bad - Scrub blocks marked bad. new - Scrub just synced blocks not yet scrubbed. full - Scrub everything. 0-100 - Scrub the exact percentage of blocks.

If you specify a percentage amount, you can also use the -o, --older-than option to define how old the block should be. The oldest blocks are scrubbed first ensuring an optimal check. If instead you want to scrub the just synced blocks, not yet scrubbed, you should use the "-p new" option.

https://www.snapraid.it/manual