Hi, I have seen joined data disks had been brought up before in 2013 and dismissed for increased complexity and not much gain but hear me out here.
Allow me to argue my case. Let's consider the following 2 scenarios.
Scenario A (current)
D: 10TB data
E: 2TB data
F: 3TB data
G: 5TB data
P: 10TB parity
vs
Scenario B (proposal)
D: 10TB data
E: 2TB + F: 3TB + G: 5TB data (aka split data drive)
P: 10TB parity
By treating E, F & G as a single disk under Scenario B, even if 2 of the 3 (EFG) or even if all 3 (EFG) drives fail, you can still don't lose any data. This would not be true under Scenario A which allows a single drive to fail before you start losing data.
This is especially applicable since the smaller drives tend to be the older models and more prone to failure. There's appears to be a huge benefit to allow data drive splits. All other pros/cons appear to be the same under both scenarios unless I'm missing out on something.
Thoughts?
Last edit: alabama 2019-08-02
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just to be clear, I'm not suggesting joining 3 physical drives into a single virtual drive letter.
I'm proposing something similar to how the split parity currently works where the data drives are separate and distinct with their own drive assignments. There's no change to how the data is currently stored. The only difference is in snapraid config and how snapraid treats the data drives when it computes the parity.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, this is really a good idea :)
My 2 parity disks are 5TB and have quite a few 2 and 3TB data disks.
But I could imagine that this would complicate the source code quite a bit.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the vote! I am upgrading to larger drives and in a pickle whether I should include the older 2/3/5TB drives as I feel it would reduce the effectiveness of raid but hate to lose the space. ie the higher number of (older) drives will increase the probability of data loss as result of multiple drive failures. Being able to 'stack' the older & smaller drives will mitigate that increased risk.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Walter, thanks for suggestion! Yes, already using 2 sets of split parity (2x5TB, 2x5TB), still have bunch of drives leftover I rather use the extra drives for data than 3rd set of parity (a bit excessive for my current setup). Sorry my scenarios were very simplified just to highlight my proposal.
Maybe something like spanned volumes is the solution after all although I really prefer the simplicity & safety of JBOD and don't mind a tiny bit more complexity in snapraid setup.
Last edit: alabama 2019-08-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You could configure those excess/older drives as a 2nd SnapRAID array.
Then snap2.bat would just be snapraid -c retirees.conf %*.
And, for examples: snap2 sync snap2 -p new scrub
Maybe shuffle some data around so that your really static stuff is on the 2nd array, and will be subject to less updating "stress"; just a relaxed scrub schedule.
Also, segregating the older/slower drives will allow your primary array's operations to run faster. (Bigger isn't always better.)
Last edit: UhClem 2019-08-16
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, that would seem to be right option at this point in time for the retirees! Great suggestion.
And good point regarding the speed although that's not an issue at the moment.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'd like to echo this request! I'm switching over to SnapRAID on account of FlexRAID going dark (website is gone and everything!) and split data drives was one of Brahim's best features! It vastly improves the safety of data as alabama mentioned and has the potential to reduce disk thrashing during parity checks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I have seen joined data disks had been brought up before in 2013 and dismissed for increased complexity and not much gain but hear me out here.
Allow me to argue my case. Let's consider the following 2 scenarios.
Scenario A (current)
D: 10TB data
E: 2TB data
F: 3TB data
G: 5TB data
P: 10TB parity
vs
Scenario B (proposal)
D: 10TB data
E: 2TB + F: 3TB + G: 5TB data (aka split data drive)
P: 10TB parity
By treating E, F & G as a single disk under Scenario B, even if 2 of the 3 (EFG) or even if all 3 (EFG) drives fail, you can still don't lose any data. This would not be true under Scenario A which allows a single drive to fail before you start losing data.
This is especially applicable since the smaller drives tend to be the older models and more prone to failure. There's appears to be a huge benefit to allow data drive splits. All other pros/cons appear to be the same under both scenarios unless I'm missing out on something.
Thoughts?
Last edit: alabama 2019-08-02
Just to be clear, I'm not suggesting joining 3 physical drives into a single virtual drive letter.
I'm proposing something similar to how the split parity currently works where the data drives are separate and distinct with their own drive assignments. There's no change to how the data is currently stored. The only difference is in snapraid config and how snapraid treats the data drives when it computes the parity.
Hi, this is really a good idea :)
My 2 parity disks are 5TB and have quite a few 2 and 3TB data disks.
But I could imagine that this would complicate the source code quite a bit.
Thanks for the vote! I am upgrading to larger drives and in a pickle whether I should include the older 2/3/5TB drives as I feel it would reduce the effectiveness of raid but hate to lose the space. ie the higher number of (older) drives will increase the probability of data loss as result of multiple drive failures. Being able to 'stack' the older & smaller drives will mitigate that increased risk.
you could do it still with split parity:
Scenario C (proposal)
D: 10TB data
P: 10TB data
E: 2TB + F: 3TB + G: 5TB parity
and that works already today (with much less complexity). The parity disk is less accessed than data disks.
Hi Walter, thanks for suggestion! Yes, already using 2 sets of split parity (2x5TB, 2x5TB), still have bunch of drives leftover I rather use the extra drives for data than 3rd set of parity (a bit excessive for my current setup). Sorry my scenarios were very simplified just to highlight my proposal.
Maybe something like spanned volumes is the solution after all although I really prefer the simplicity & safety of JBOD and don't mind a tiny bit more complexity in snapraid setup.
Last edit: alabama 2019-08-04
You could configure those excess/older drives as a 2nd SnapRAID array.
Then
snap2.batwould just besnapraid -c retirees.conf %*.And, for examples:
snap2 syncsnap2 -p new scrubMaybe shuffle some data around so that your really static stuff is on the 2nd array, and will be subject to less updating "stress"; just a relaxed scrub schedule.
Also, segregating the older/slower drives will allow your primary array's operations to run faster. (Bigger isn't always better.)
Last edit: UhClem 2019-08-16
Yes, that would seem to be right option at this point in time for the retirees! Great suggestion.
And good point regarding the speed although that's not an issue at the moment.
I'd like to echo this request! I'm switching over to SnapRAID on account of FlexRAID going dark (website is gone and everything!) and split data drives was one of Brahim's best features! It vastly improves the safety of data as alabama mentioned and has the potential to reduce disk thrashing during parity checks.