I have been using SnapRAID for a couple years now, I'd like to thank you for saving me a couple times from hard drive failures.
Now I have another drive that needs to be replaced, bad sectors are coming up in SMART.
The array consists of five 2TB drives, four data and one parity.
I am also in need of more drive space. My original plan was to start buying 4TB drives to build a second array but now with one of the drives failing in the first array and needing to buy a replacement (thanks Seagate for a bad warranty drive) I am trying to think of a way to upgrade the original array to larger disks over time. The safest option is to just replace the 2TB with another 2TB and hold off building a second array but I personally feel buying another 2TB drive is a waste of money.
I can't really think of a way to do this without leaving data out of the array. I can't use the full 4TBs as data since the parity is 2TBs. I can't partition the 4TBs into two 2TBs and parity both as if that drive fails the entire array fails.
A couple options I have come up with:
Partitioning the 4TB into two 2TBs, replacing the failed drive with one partition and using the second partition as data but leaving it out of the array. The issue with this is merging the two partitions once the parity can become 4TBs.
Partitioning the 4TB into two 2TBs, replacing the parity with one partition and using the second partition as data, leaving it out of the array. The original parity drive would become the replacement for the failed drive. Again, I would lose the second partition if anything were to happen to the drive. But with this method when I can afford it or another 2TB fails I could replace that with a 4TB, transfer everything from the second partition of the parity to the new drive, delete that partition then expand the parity to 4TB. The issue with going this route is I leave my array in a vulnerable state using a failing drive to rebuild the parity before I can use the original parity as a replacement for the failing drive.
I am personally leaning towards the second option. As long as the drive can rebuild the parity it gives me a better path to upgrade but a slightly higher risk at first. While the data on these drives is just media, I can always replace the data, it is time consuming and a huge hassle to do it and I'd like to avoid it if I can. ANY thoughts anyone has on a way to handle this would be appreciated.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ideally, buy a 4TB drive for parity and a 4TB drive to replace the failing data drive.
If that's not an option, buy a 4TB drive for parity. Move the parity file to the new drive. Just leave the second half of the drive empty, don't partition it into two volumes. Now you have 4TB parity and room to expand all data drives up to 4TB. Use the old 2TB parity drive to replace the failing data drive.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ideally, I would love to just buy a new 2TB drive to replace the failing one and two 4TBs to start a second array. Sadly, money in the real world while dealing with what is basically a hobby prevents this.
While I would normally upgrade the old array by doing just that, replacing the parity to 4TB without using the extra space then later replace the drives one by one with 4TBs, I am in some serious need of more data space. Even if that space, for a short time, does not have a parity.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Alternatives: 1) Copy 2TB parity to 4TB drive, verify the copy. Delete
the original 2TB parity, and copy the failing 2TB data to it, verify.
or, better: 2) Replace the failing 2TB with 4TB, verify. Replace the
2TB parity with 4TB, verify. Sell the old 2TB parity on ebay, or throw
it in a drawer so you'll have a spare the next time this happens.
On 3/25/2015 8:10 PM, CharredChar wrote:
I have been using SnapRAID for a couple years now, I'd like to thank you for saving me a couple times from hard drive failures.
Now I have another drive that needs to be replaced, bad sectors are coming up in SMART.
The array consists of five 2TB drives, four data and one parity.
I am also in need of more drive space. My original plan was to start buying 4TB drives to build a second array but now with one of the drives failing in the first array and needing to buy a replacement (thanks Seagate for a bad warranty drive) I am trying to think of a way to upgrade the original array to larger disks over time. The safest option is to just replace the 2TB with another 2TB and hold off building a second array but I personally feel buying another 2TB drive is a waste of money.
I can't really think of a way to do this without leaving data out of the array. I can't use the full 4TBs as data since the parity is 2TBs. I can't partition the 4TBs into two 2TBs and parity both as if that drive fails the entire array fails.
A couple options I have come up with:
Partitioning the 4TB into two 2TBs, replacing the failed drive with one partition and using the second partition as data but leaving it out of the array. The issue with this is merging the two partitions once the parity can become 4TBs.
Partitioning the 4TB into two 2TBs, replacing the parity with one partition and using the second partition as data, leaving it out of the array. The original parity drive would become the replacement for the failed drive. Again, I would lose the second partition if anything were to happen to the drive. But with this method when I can afford it or another 2TB fails I could replace that with a 4TB, transfer everything from the second partition of the parity to the new drive, delete that partition then expand the parity to 4TB. The issue with going this route is I leave my array in a vulnerable state using a failing drive to rebuild the parity before I can use the original parity as a replacement for the failing drive.
I am personally leaning towards the second option. As long as the drive can rebuild the parity it gives me a better path to upgrade but a slightly higher risk at first. While the data on these drives is just media, I can always replace the data, it is time consuming and a huge hassle to do it and I'd like to avoid it if I can. ANY thoughts anyone has on a way to handle this would be appreciated.
While I do like the idea of a hot swap drive, it may be something I do in the future, but again money in the real world for something that isn't crucial data kind of negates the usefulness of it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First of all I'm not sure about the penalty of having some (non-snapraid) data on the same filesystem as the parity data. Especially if you are a bit smart about it and you do a sync first (and you have some of the data disks close to full). Yea, maybe you'll get some fragmentation in time but shouldn't be that bad.
Either that or just do two partitions (make sure the parity one will be a bit larger, like 100-200GB to be able to use all your 2TB data disks completely, remember the overhead in parity where you need to leave some free space on each data disks if parity is equal to the data disks). Also use some partitioning scheme+filesystem that allows for quick extension of the partition so at a later time you can just extend the partition if you wish. But anyway this is secondary, when you'll get a new 4TB drive you can just make one big partition there and basically start over in whatever configuration fits you.
Also if you want to be nasty you can just keep the old failing disk and backup (rsync/whatever) there the data you put in the remaining ~2TB! Spin down the disk for most of the time and make sure you don't use huge files for backup (like huge tar's or snapraid parity, disk images, etc) - just small 1:1 files as on the original disk.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One of the reason I am in serious need for more data drives is the fact that my current four drives are beyond what I am comfortable with in free space, averaging ~75GBs free, each. While the parity still has 45Gbs free, it is not something I want to push my luck with and would feel better if I have even just an extra 2TBs to put some data on until I can get a second array up and running.
I forgot to mention before that these drives are NTFS, Windows 7 Pro. It is fairly easy to use my second method to expand the 2TB partition of a 4TB drive for the parity once I pull the data off the second partition onto a new drive.
Windows is set to spin the drives down after 10 minutes, since it is all media the data rarely gets read and even more rare to be modified.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you can afford it I guess the most straightforward solution to both problems would be:
1 Buy two 4 TB (or larger) drives.
2 Replace the failing data disk and run fix.
3 Replace the parity disk and run fix again.
Bonus level:
4 "Move" all files from one of the 2 TB data disks to the 4 TB data disk (copy and put them in a hidden folder and don't delete until after sync)
5 Sync
6 Merge, JBOD, Raid0, span or stripe the unused disks into a 4 TB data disk.
Result:
1x2 TB data disk (nothing changed)
1x2 TB data disk (nothing changed)
1x(2+2) TB data disk (empty)
1x4 TB data disk (full)
1x4 TB parity disk
7 "Move" all files from the 2 TB data disks to the empty 4 TB disk
8 Merge, JBOD, Raid0, span or stripe the remainging empty data disks into a 4 TB data disk.
9 Sync
Result:
1x(2+2) TB data disk (empty)
1x(2+2) TB data disk (full)
1x4 TB data disk (full)
1x4 TB parity disk
The added risk of having one more physical disk in the array is negated by the chance that second drive is part of the same logical 4 TB disk as the first one.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As with a previous reply, while I would love to do the straightforward solution I can not afford it. I can only buy one drive currently, be it 2TB or 4TB and I will not be able to buy a second for at least 30 days. If this issue never came up I would have just started building the second array with a data drive first then a parity in 30 days.
I do find the "bonus level" Very interesting, I might very well be the way I upgrade the first array once the second is complete. My original plan with this entire thing was to add a second array as I needed space then once that is complete go about replacing the first array with larger drives as they fail/need more space. I would just go back and forth between the arrays as needed, one basically being the last generation of drives and one being current.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have a problem with combining RAID-0 (or similar) with RAID for no good reason. There is one extra layer of complications (that can create both technical and human problems) for no good reason. And if you lose more than one disk you (can) start losing more than what is on the disks you lost.
Space-wise it's about the same.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The main reason would be that 12 TB data protected is better than 10 TB protected + 2 TB unprotected, given that lack of space was one the described problems.
Increased read and write speed is usually also nice even though it may not be necessary.
It is correct that if you loose more volumes than you have parity to recover then data loss is proportional to the lost volume size, just the same as if a physical 4 TB disk was lost.
This could however be negated by using disk spaning instead of stripe/RAID0 in case increased speed is irrelevant. In that case only the data located on the individual broken disks would be lost.
Last edit: Leifi Plomeros 2015-03-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1x(2+2) TB data disk (empty)
1x(2+2) TB data disk (full)
1x4 TB data disk (full)
1x4 TB parity disk
if you just make it flat: 4x2TB + 1x4TB data disks you still have 12TB, all protected!
And I put "about" the same because you'll have in fact a bit more, depending on the number of files - you can fill each of the 2TB drives tot the brim, which is not the case if you keep them in 2+2=4=parity size.
Last edit: John 2015-03-28
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, but then he would be outside the recommended number of parity disks.
With flat setup and one disk lost he has 0% chance to survive an additional drive failure without data loss.
By following my recommendation with striped disks he has 4/6 * 1/5 = 13.3% probability that he can fully recover from two failed disks without any data loss.
With spaning disks instead of striped disks he would also have an added chance of partial recovery from two disk failures.
This is without considering that the striped/spaned disks are older and more likely to fail.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
13.3% is worse than russian roulette with 5 bullets and one empty. Better than zero but it comes at a cost in space, complexity and even pure chances of data loss if the disks are striped.
As for being outside the recommended number of parity disk(s) that is no clear threshold and we know perfectly well (as in well tested) what happens if you run an array with less than the recommended number (you have slightly higher risks and that's all).
Do we have any data about what happens when you use something else than normal devices for snapraid? There are a lot of device id's checked, will it work at all? The new SMART stats for sure aren't expecting multiple devices behind the folders presented to snapraid. Apart from that, what is the spanning solution proposed, I could just try to make a quick test?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A spanned volume is the native Windows implementation of JBOD. The disks are linearly concatenated. When the first disk is full it starts to fill the next one. I have personally never used that alternative but it is supposedly easy to recover files from the survivng disk in event of disk crash.
I would say that both spanning and striping is very simple and safe to use. If you disconnect one disk then the entire volume becomes unavailable until the disk is reconnected (to any port, on any controller). You can unplug both and put them in another computer and they will still be identified as a single volume. So if you are unsure which physical disk belongs to which volume. You can just disconnect them one by one if needed.
But yes sure, I didn't specify to use any specific RAID implementation. So the advice could be interpreted as "Hey, why not use any $15 hardware raid controller with 2 star review on amazon, setup it up as RAID0 and forget all about it? It is safe I promise!"
In that scenario I share your concern. But I think that is part of a much wider topic which usually ends up on forums with at least 1 person religiously claiming that HARDWARE RAID10 is the only alternative. :)
The new SMART features will most likely not work. But since we now know that CharredChar is using Windows 7 I would recommend him to use a dedicated SMART monitoring tool such as HDD Guardian instead (which is a really good one).
As for other incompatibility issues. I can't think of any. SnapRaid sees it as a single volume with a single identifier. I haven't encountered any issues when fixing, syncing, scrubing, moving, copying, adding or removing data in my own setup.
As for the space issue, there is a flip side to it. Fewer and larger volumes could also help keep down the reserved space for new files.
Regarding Russian roulette... Think of it as a life vest on a small boat in a storm (first disk lost). If the boat sinks (second disk lost) you are probably f****d but most people would still prefer to have the life vest to cover the small chance that someone will find and rescue them.
Edit: I just realized. My recommendation is pretty much just SnapRaid adapted variant of RAID10... In RAID10 you have 2 sets of RAID0 protected in a RAID1 set :)
Last edit: Leifi Plomeros 2015-03-29
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have been using SnapRAID for a couple years now, I'd like to thank you for saving me a couple times from hard drive failures.
Now I have another drive that needs to be replaced, bad sectors are coming up in SMART.
The array consists of five 2TB drives, four data and one parity.
I am also in need of more drive space. My original plan was to start buying 4TB drives to build a second array but now with one of the drives failing in the first array and needing to buy a replacement (thanks Seagate for a bad warranty drive) I am trying to think of a way to upgrade the original array to larger disks over time. The safest option is to just replace the 2TB with another 2TB and hold off building a second array but I personally feel buying another 2TB drive is a waste of money.
I can't really think of a way to do this without leaving data out of the array. I can't use the full 4TBs as data since the parity is 2TBs. I can't partition the 4TBs into two 2TBs and parity both as if that drive fails the entire array fails.
A couple options I have come up with:
Partitioning the 4TB into two 2TBs, replacing the failed drive with one partition and using the second partition as data but leaving it out of the array. The issue with this is merging the two partitions once the parity can become 4TBs.
Partitioning the 4TB into two 2TBs, replacing the parity with one partition and using the second partition as data, leaving it out of the array. The original parity drive would become the replacement for the failed drive. Again, I would lose the second partition if anything were to happen to the drive. But with this method when I can afford it or another 2TB fails I could replace that with a 4TB, transfer everything from the second partition of the parity to the new drive, delete that partition then expand the parity to 4TB. The issue with going this route is I leave my array in a vulnerable state using a failing drive to rebuild the parity before I can use the original parity as a replacement for the failing drive.
I am personally leaning towards the second option. As long as the drive can rebuild the parity it gives me a better path to upgrade but a slightly higher risk at first. While the data on these drives is just media, I can always replace the data, it is time consuming and a huge hassle to do it and I'd like to avoid it if I can. ANY thoughts anyone has on a way to handle this would be appreciated.
Ideally, buy a 4TB drive for parity and a 4TB drive to replace the failing data drive.
If that's not an option, buy a 4TB drive for parity. Move the parity file to the new drive. Just leave the second half of the drive empty, don't partition it into two volumes. Now you have 4TB parity and room to expand all data drives up to 4TB. Use the old 2TB parity drive to replace the failing data drive.
Ideally, I would love to just buy a new 2TB drive to replace the failing one and two 4TBs to start a second array. Sadly, money in the real world while dealing with what is basically a hobby prevents this.
While I would normally upgrade the old array by doing just that, replacing the parity to 4TB without using the extra space then later replace the drives one by one with 4TBs, I am in some serious need of more data space. Even if that space, for a short time, does not have a parity.
Alternatives: 1) Copy 2TB parity to 4TB drive, verify the copy. Delete
the original 2TB parity, and copy the failing 2TB data to it, verify.
or, better: 2) Replace the failing 2TB with 4TB, verify. Replace the
2TB parity with 4TB, verify. Sell the old 2TB parity on ebay, or throw
it in a drawer so you'll have a spare the next time this happens.
On 3/25/2015 8:10 PM, CharredChar wrote:
See my comment above.
While I do like the idea of a hot swap drive, it may be something I do in the future, but again money in the real world for something that isn't crucial data kind of negates the usefulness of it.
First of all I'm not sure about the penalty of having some (non-snapraid) data on the same filesystem as the parity data. Especially if you are a bit smart about it and you do a sync first (and you have some of the data disks close to full). Yea, maybe you'll get some fragmentation in time but shouldn't be that bad.
Either that or just do two partitions (make sure the parity one will be a bit larger, like 100-200GB to be able to use all your 2TB data disks completely, remember the overhead in parity where you need to leave some free space on each data disks if parity is equal to the data disks). Also use some partitioning scheme+filesystem that allows for quick extension of the partition so at a later time you can just extend the partition if you wish. But anyway this is secondary, when you'll get a new 4TB drive you can just make one big partition there and basically start over in whatever configuration fits you.
Also if you want to be nasty you can just keep the old failing disk and backup (rsync/whatever) there the data you put in the remaining ~2TB! Spin down the disk for most of the time and make sure you don't use huge files for backup (like huge tar's or snapraid parity, disk images, etc) - just small 1:1 files as on the original disk.
One of the reason I am in serious need for more data drives is the fact that my current four drives are beyond what I am comfortable with in free space, averaging ~75GBs free, each. While the parity still has 45Gbs free, it is not something I want to push my luck with and would feel better if I have even just an extra 2TBs to put some data on until I can get a second array up and running.
I forgot to mention before that these drives are NTFS, Windows 7 Pro. It is fairly easy to use my second method to expand the 2TB partition of a 4TB drive for the parity once I pull the data off the second partition onto a new drive.
Windows is set to spin the drives down after 10 minutes, since it is all media the data rarely gets read and even more rare to be modified.
If you can afford it I guess the most straightforward solution to both problems would be:
1 Buy two 4 TB (or larger) drives.
2 Replace the failing data disk and run fix.
3 Replace the parity disk and run fix again.
Bonus level:
4 "Move" all files from one of the 2 TB data disks to the 4 TB data disk (copy and put them in a hidden folder and don't delete until after sync)
5 Sync
6 Merge, JBOD, Raid0, span or stripe the unused disks into a 4 TB data disk.
Result:
1x2 TB data disk (nothing changed)
1x2 TB data disk (nothing changed)
1x(2+2) TB data disk (empty)
1x4 TB data disk (full)
1x4 TB parity disk
7 "Move" all files from the 2 TB data disks to the empty 4 TB disk
8 Merge, JBOD, Raid0, span or stripe the remainging empty data disks into a 4 TB data disk.
9 Sync
Result:
1x(2+2) TB data disk (empty)
1x(2+2) TB data disk (full)
1x4 TB data disk (full)
1x4 TB parity disk
The added risk of having one more physical disk in the array is negated by the chance that second drive is part of the same logical 4 TB disk as the first one.
As with a previous reply, while I would love to do the straightforward solution I can not afford it. I can only buy one drive currently, be it 2TB or 4TB and I will not be able to buy a second for at least 30 days. If this issue never came up I would have just started building the second array with a data drive first then a parity in 30 days.
I do find the "bonus level" Very interesting, I might very well be the way I upgrade the first array once the second is complete. My original plan with this entire thing was to add a second array as I needed space then once that is complete go about replacing the first array with larger drives as they fail/need more space. I would just go back and forth between the arrays as needed, one basically being the last generation of drives and one being current.
I have a problem with combining RAID-0 (or similar) with RAID for no good reason. There is one extra layer of complications (that can create both technical and human problems) for no good reason. And if you lose more than one disk you (can) start losing more than what is on the disks you lost.
Space-wise it's about the same.
The main reason would be that 12 TB data protected is better than 10 TB protected + 2 TB unprotected, given that lack of space was one the described problems.
Increased read and write speed is usually also nice even though it may not be necessary.
It is correct that if you loose more volumes than you have parity to recover then data loss is proportional to the lost volume size, just the same as if a physical 4 TB disk was lost.
This could however be negated by using disk spaning instead of stripe/RAID0 in case increased speed is irrelevant. In that case only the data located on the individual broken disks would be lost.
Last edit: Leifi Plomeros 2015-03-27
As I mentioned in the configuration proposed:
1x(2+2) TB data disk (empty)
1x(2+2) TB data disk (full)
1x4 TB data disk (full)
1x4 TB parity disk
if you just make it flat: 4x2TB + 1x4TB data disks you still have 12TB, all protected!
And I put "about" the same because you'll have in fact a bit more, depending on the number of files - you can fill each of the 2TB drives tot the brim, which is not the case if you keep them in 2+2=4=parity size.
Last edit: John 2015-03-28
Yes, but then he would be outside the recommended number of parity disks.
With flat setup and one disk lost he has 0% chance to survive an additional drive failure without data loss.
By following my recommendation with striped disks he has 4/6 * 1/5 = 13.3% probability that he can fully recover from two failed disks without any data loss.
With spaning disks instead of striped disks he would also have an added chance of partial recovery from two disk failures.
This is without considering that the striped/spaned disks are older and more likely to fail.
13.3% is worse than russian roulette with 5 bullets and one empty. Better than zero but it comes at a cost in space, complexity and even pure chances of data loss if the disks are striped.
As for being outside the recommended number of parity disk(s) that is no clear threshold and we know perfectly well (as in well tested) what happens if you run an array with less than the recommended number (you have slightly higher risks and that's all).
Do we have any data about what happens when you use something else than normal devices for snapraid? There are a lot of device id's checked, will it work at all? The new SMART stats for sure aren't expecting multiple devices behind the folders presented to snapraid. Apart from that, what is the spanning solution proposed, I could just try to make a quick test?
A spanned volume is the native Windows implementation of JBOD. The disks are linearly concatenated. When the first disk is full it starts to fill the next one. I have personally never used that alternative but it is supposedly easy to recover files from the survivng disk in event of disk crash.
I would say that both spanning and striping is very simple and safe to use. If you disconnect one disk then the entire volume becomes unavailable until the disk is reconnected (to any port, on any controller). You can unplug both and put them in another computer and they will still be identified as a single volume. So if you are unsure which physical disk belongs to which volume. You can just disconnect them one by one if needed.
But yes sure, I didn't specify to use any specific RAID implementation. So the advice could be interpreted as "Hey, why not use any $15 hardware raid controller with 2 star review on amazon, setup it up as RAID0 and forget all about it? It is safe I promise!"
In that scenario I share your concern. But I think that is part of a much wider topic which usually ends up on forums with at least 1 person religiously claiming that HARDWARE RAID10 is the only alternative. :)
The new SMART features will most likely not work. But since we now know that CharredChar is using Windows 7 I would recommend him to use a dedicated SMART monitoring tool such as HDD Guardian instead (which is a really good one).
As for other incompatibility issues. I can't think of any. SnapRaid sees it as a single volume with a single identifier. I haven't encountered any issues when fixing, syncing, scrubing, moving, copying, adding or removing data in my own setup.
As for the space issue, there is a flip side to it. Fewer and larger volumes could also help keep down the reserved space for new files.
Regarding Russian roulette... Think of it as a life vest on a small boat in a storm (first disk lost). If the boat sinks (second disk lost) you are probably f****d but most people would still prefer to have the life vest to cover the small chance that someone will find and rescue them.
Edit: I just realized. My recommendation is pretty much just SnapRaid adapted variant of RAID10... In RAID10 you have 2 sets of RAID0 protected in a RAID1 set :)
Last edit: Leifi Plomeros 2015-03-29