I'm new to snapraid and intend to create a new array with 15 4TB drives plus 3 4TB parity drives. They are all external USB 3.0 drives (I know, I know). The data is already on the drives and they are nearly full.
This will be for infrequently used and even more rarely changed archive/cold storage.
Now, I realize that with that many external USB drives it will take a long time to sync the array. Since, as I understand it, during a sync, all drives will be accessed simultaneously in parallel, I'm trying to figure out whether I'm better off starting with, say, 3 data drives, syncing, and then adding 3 data drives at a time and re-syncing,etc. vs. doing all the drives at once. This is to maximize sync speed, given that simultaneously reading so many drives on one USB 3.0 bus will slow them all down dramatically.
This assumes that when you add drives to an existing already sync'd array, the old already processed drives will not have to be re-read during the sync for the added drives. Is that a correct assumtion? Am I estimating the tradeoffs correctly? Any gotchas for doing it this way? Thanks in advance.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
15+3 array is more secure than three sets 5+1.
But it will take a really really long time to build and hopefully you do not have some USB problems (like 256 byte mismatch of same sector data).
Do you really have 18* USB3 on your computer? Or do you use USB3 Hubs (one more point for errors) and big slowdown as you mentioned dramtic slowdown if all disks are read in parallel.
I think, currently if you add a data disk, all data and parity disks will be read (for checking and calculating parity).
Do not forget to make a scrub after first sync (to check if all data was protected right). If you do not and there were some transfer problems, you will not have full protection.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Create a folder on each disk and put a 4 GB file in each of them.
Setup a snapraid array protecting only the contents of that folder.
Run snapraid sync and observe how long it takes to sync (ignore the first part when files are being indexed and content files created). Then multiply by 1.000 to get an estimated total sync time.
If you have spread the USB drives fairly even on two separate USB controllers (and don't have any other significant bottle necks) you could quite possibly squeeze out more than 50 MB/s per drive which should then translate to less than 24 hours for a complete sync.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree about the points of failure, but the system has been rock solid and I don't have a practical choice. That does bring to mind a question though:
How sensitive is snapraid to a small number of errors not detected during build? The data is all video, which is very tolerant of losses, but will snapraid be able to recover all non-errored blocks? In that regard, will using a smaller block size help? (I have 32GB of RAM)
Re. the scrub immediately after, doesn't snapraid verify during build?
Leifi - that's a great suggestion. I will do that. What do you think of the one-at-a-time option? By my estimate that should allow me to get well over 100 MB/s. If worse comes to worse even well over 24 hours just for the initial sync is not a problem. I wonder though about subsequent scrubs. Although they will be infrequent, the slow speed could be more annoying there. Is it possible to limit the scrub to data on one disk only (even if by manually specifying a particular drive/folder?)
Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Never mind the question about limiting the scrub to certain disks. I realize that's impossible because the parity is built from all disks.
That does make the question about snapraid's ability to recover all blocks with non-errored parity (and related to that whether smaller blocks would help) more important to me, since the slow speed will limit the frequency of scrubs more than normal which increases the likelihood of "silent" errors.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm new to snapraid and intend to create a new array with 15 4TB drives plus 3 4TB parity drives. They are all external USB 3.0 drives (I know, I know). The data is already on the drives and they are nearly full.
This will be for infrequently used and even more rarely changed archive/cold storage.
Now, I realize that with that many external USB drives it will take a long time to sync the array. Since, as I understand it, during a sync, all drives will be accessed simultaneously in parallel, I'm trying to figure out whether I'm better off starting with, say, 3 data drives, syncing, and then adding 3 data drives at a time and re-syncing,etc. vs. doing all the drives at once. This is to maximize sync speed, given that simultaneously reading so many drives on one USB 3.0 bus will slow them all down dramatically.
This assumes that when you add drives to an existing already sync'd array, the old already processed drives will not have to be re-read during the sync for the added drives. Is that a correct assumtion? Am I estimating the tradeoffs correctly? Any gotchas for doing it this way? Thanks in advance.
You could connect 5 data + 1 parity at a time to create 3 independant arrays.
Last edit: Stefan Thüring 2019-03-06
From all the reading I've done, my understanding is that one 15 + 3 array is more secure than 3 separate 5 + 1 arrays. Is that not the case?
15+3 array is more secure than three sets 5+1.
But it will take a really really long time to build and hopefully you do not have some USB problems (like 256 byte mismatch of same sector data).
Do you really have 18* USB3 on your computer? Or do you use USB3 Hubs (one more point for errors) and big slowdown as you mentioned dramtic slowdown if all disks are read in parallel.
I think, currently if you add a data disk, all data and parity disks will be read (for checking and calculating parity).
Do not forget to make a scrub after first sync (to check if all data was protected right). If you do not and there were some transfer problems, you will not have full protection.
I would suggest a practical test:
If you have spread the USB drives fairly even on two separate USB controllers (and don't have any other significant bottle necks) you could quite possibly squeeze out more than 50 MB/s per drive which should then translate to less than 24 hours for a complete sync.
I agree about the points of failure, but the system has been rock solid and I don't have a practical choice. That does bring to mind a question though:
How sensitive is snapraid to a small number of errors not detected during build? The data is all video, which is very tolerant of losses, but will snapraid be able to recover all non-errored blocks? In that regard, will using a smaller block size help? (I have 32GB of RAM)
Re. the scrub immediately after, doesn't snapraid verify during build?
Leifi - that's a great suggestion. I will do that. What do you think of the one-at-a-time option? By my estimate that should allow me to get well over 100 MB/s. If worse comes to worse even well over 24 hours just for the initial sync is not a problem. I wonder though about subsequent scrubs. Although they will be infrequent, the slow speed could be more annoying there. Is it possible to limit the scrub to data on one disk only (even if by manually specifying a particular drive/folder?)
Thanks!
Never mind the question about limiting the scrub to certain disks. I realize that's impossible because the parity is built from all disks.
That does make the question about snapraid's ability to recover all blocks with non-errored parity (and related to that whether smaller blocks would help) more important to me, since the slow speed will limit the frequency of scrubs more than normal which increases the likelihood of "silent" errors.