So, I'm new to snapRAID.
Still don't have it installed but I'm planning on using it on a new JBOD system.
The thing is, I have one 6TB HD, several 4TB drives but most are 3TB.
Will I be able to take full advantage of my drives and what is the best way of doing this?
What I'm thinking is, if I put the 6TB drive and only use 3TB drives as data drives, would the 6TB be able to keep double parity and protect against the failure of two 3TB drives? (Obvious if one of the drives to fail was that 6TB one, any other that fails at the same time is lost and I'm aware of that. Just trying to understand the fully functionality to decide how to best use my current resources).
I've read the manual but there is no mention of these types of scenarios.
Would this be decided by defining two parity files on the same drive on the configuration file?
What about mixing the 4TB drives in it?
A 4TB drive will not be able to hold 2 full parity files for 3TB drives but three 4TB drives would be able to hold parity for four 3TB drives. Can snapRAID break parity files into multiple pieces into multiple drives or does it require that a parity file be in a single continuous file on a single drive?
Thanks and sorry if this is answered somewhere but I wasn't able to find it
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So I can have the parity of a 6TB drive into two 3TB drives?
If I have an array with 1x 6TB + nx 3TB I can set 2x 3TB drives for parity and the 6TB drive will be protected. If I add a 3rd 3TB parity will that provide 2x parity protection for the other 3TB drives or will I always need 2x 3TB drives per parity from that point on?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
FR is very flexible with splitting parrity among all drives or between multiple drives.
However, I believe in simplicity and when we all have RAIDs to recover data. When I need to recover a drive I want things to be as simple as possbile. I don't want to have have to worry about anything other than physically replacing a drive and recovering it.
I'd recommend setting the 6TB & a 4TB as the parity. Sure, you would have 2TB on the 6TB as "unused" by the RAID, but you can still put data there. You will just need to make sure 4TB on that 6TB is available for parity. When you add another 6TB drive, simply make it a parity and move the 4TB to a data. then when you add a 6TB after that, you'll have the full space available.
Yeah, some space will be free on the parity drive(s) for a little while, but it's what I did going from 4TB drives to 6TB to 8TB drives.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Eya, thanks for the answers (forgot to say that on my previous reply).
Thing is, right now the best price per GB by far is for 3TB drives so that's what I'm buying right now. (I live in EU. Can find drives cheaper online but shipping charges ruin it.)
I even took the power consumption of one larger drive vs multiple smaller ones and couldn't justify getting a larger one. The price difference is just too big.
I'm not going to buy larger drives just to take advantage of the full space of those I already have and fall into a sunken cost falacy. But I would like to take advantage of them as most as possible and not let them go to waste.
It's nice to know that free space on a parity drive can be used for other things. Maybe I'll do that.
Can parity drives be partitioned? So I would partition the space needed for parity and the other partition would be used for something else.
SnapRAID works at the level of partitions or physical drives?
Cheers
B0rg
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thing is, right now the best price per GB by far is for 3TB drives so that's what I'm buying right now. (I live in EU. Can find drives cheaper online but shipping charges ruin it.)
I even took the power consumption of one larger drive vs multiple smaller ones and couldn't justify getting a larger one. The price difference is just too big.
Unless you have already invested in something like a Backblaze storage pod you will eventually run out of drive slots or SATA ports. When factoring in that you will then either need to get rid of smaller drives or buy a new chassis/controller card, the small drives rarely turn out to be the cheapest option per TB.
Can parity drives be partitioned? So I would partition the space needed for parity and the other partition would be used for something else.
Yes. but make sure that you create the parity partition first and add the second parition last so that you can easily get rid of the second parition and expand the parity partition without any hazzle if you need more parity space later.
Also consider making the parity parition a little bit larger than the largest data disk to compensate for the larger data blocks used by snapraid parity compared to your file system.
It works on file level limited to a single parition per drive (excluding the SMART feature and other drive related stuff).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm actually mostly talking about a sbc nas with usb disks so that is not a concern. It is always possible to add more disks easily. Or to just add another nas. They are cheap.
Ya, performance is not great but its mostly not a consideration for this.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ya. Well, I have a tower with 14 drives where I'm also installing it. But what I'm building right now and the plans for growing are those cheap nas.
I'm also wandering about how feasible it would be to run it inter-nas.
Like, having it run on each nas so that each nas has some internal safety with its own parity drives.
But for backup, I would like to try the following: Instead of having two identical nas mirroring each other for backup, I'm thinking about starting with 3 smaller ones and having one being an entire parity nas for the other two. That would allow me to grow them in different ways, either grow each individual nas in the number of drives in them or add another nas and so on. That would protect me against the loss of a full nas without having to keep an entire mirror all the time.
This is probably feasible by mounting all the nas drives on a machine where snapraid is installed. Just bring them all online to run scrubs and syncing when new data is added. Should work. Probably slow as hell but should work I hope. Anyone sees any pitfalls or has a better idea?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Assuming that you eventually have 10 drives connected via the network (hosted at the NAS devices) and that you have a 1 Gigabit switch or router somewhere in between this is what I expect that you will find:
Horrible sync performance at around 10 MB/s per disk ( 4 days initial sync if only using 3 TB disks).
Horrible network performance in general during sync since the switch/router is being completely saturated.
DAS is a much more attractive alternative if you want all drives in a single snapraid array.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You are talking about the snapraid idea for using multiple NAS right?
Those times are perfectly acceptable.
I intend to only add new data in accumulated batches. I have no problem if the sync afterwards takes a long time. I'll just regulate the interval at which new data is added based on how long it takes to sync the new data.
I already have and will keep a fast but more space limited data storage for first stage backup storage with its own higher cost redundancy which is used for imediate storage and working and update data in batches from this to the less performing / increased space / lower cost storage.
The initial sync time is not even a consideration. It could take a month for all I care. It just needs to take less time than the time it takes to fill up the first stage storage.
My only considerations: how easy and cheap it is to expand the arrays and the amount of protection they have.
DAS is limited in both expandability and how costly it can become after some point.
A minimum amount of performance of the arrays for normal usage is a consideration, but performance for syncing is not.
If I need the performance during a syncing job, it can be interrupted and continued later per the manual (haven't tried this yet).
Cheers
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
General disk performance during sync at really slow speed will be good. The obvious risk is network congestion. But if you have that under controll and/or is not bothered by poor network performance during sync, then I don't see any problem.
Just keep in mind that if you add X amount of data to any individual data disk, then sync will typically need to read an equal amount of data from each data disk to be able to calculate new parity for all files which share the same parity blocks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That means that to reduce the amount of data that needs to be read in total one should always try to divide new data by all drives as equally as possible right?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So, I'm new to snapRAID.
Still don't have it installed but I'm planning on using it on a new JBOD system.
The thing is, I have one 6TB HD, several 4TB drives but most are 3TB.
Will I be able to take full advantage of my drives and what is the best way of doing this?
What I'm thinking is, if I put the 6TB drive and only use 3TB drives as data drives, would the 6TB be able to keep double parity and protect against the failure of two 3TB drives? (Obvious if one of the drives to fail was that 6TB one, any other that fails at the same time is lost and I'm aware of that. Just trying to understand the fully functionality to decide how to best use my current resources).
I've read the manual but there is no mention of these types of scenarios.
Would this be decided by defining two parity files on the same drive on the configuration file?
What about mixing the 4TB drives in it?
A 4TB drive will not be able to hold 2 full parity files for 3TB drives but three 4TB drives would be able to hold parity for four 3TB drives. Can snapRAID break parity files into multiple pieces into multiple drives or does it require that a parity file be in a single continuous file on a single drive?
Thanks and sorry if this is answered somewhere but I wasn't able to find it
Hi Borg,
SnapRAID doesn't allow to put more than one parity on a single disk. It refuses to start. Any extra space will be unused.
This is done for the recovery reason you mentioned, but also for performance.
You can instead split one parity in multiple disks.
Ciao,
Andrea
So I can have the parity of a 6TB drive into two 3TB drives?
If I have an array with 1x 6TB + nx 3TB I can set 2x 3TB drives for parity and the 6TB drive will be protected. If I add a 3rd 3TB parity will that provide 2x parity protection for the other 3TB drives or will I always need 2x 3TB drives per parity from that point on?
FR is very flexible with splitting parrity among all drives or between multiple drives.
However, I believe in simplicity and when we all have RAIDs to recover data. When I need to recover a drive I want things to be as simple as possbile. I don't want to have have to worry about anything other than physically replacing a drive and recovering it.
I'd recommend setting the 6TB & a 4TB as the parity. Sure, you would have 2TB on the 6TB as "unused" by the RAID, but you can still put data there. You will just need to make sure 4TB on that 6TB is available for parity. When you add another 6TB drive, simply make it a parity and move the 4TB to a data. then when you add a 6TB after that, you'll have the full space available.
Yeah, some space will be free on the parity drive(s) for a little while, but it's what I did going from 4TB drives to 6TB to 8TB drives.
Eya, thanks for the answers (forgot to say that on my previous reply).
Thing is, right now the best price per GB by far is for 3TB drives so that's what I'm buying right now. (I live in EU. Can find drives cheaper online but shipping charges ruin it.)
I even took the power consumption of one larger drive vs multiple smaller ones and couldn't justify getting a larger one. The price difference is just too big.
I'm not going to buy larger drives just to take advantage of the full space of those I already have and fall into a sunken cost falacy. But I would like to take advantage of them as most as possible and not let them go to waste.
It's nice to know that free space on a parity drive can be used for other things. Maybe I'll do that.
Can parity drives be partitioned? So I would partition the space needed for parity and the other partition would be used for something else.
SnapRAID works at the level of partitions or physical drives?
Cheers
B0rg
Unless you have already invested in something like a Backblaze storage pod you will eventually run out of drive slots or SATA ports. When factoring in that you will then either need to get rid of smaller drives or buy a new chassis/controller card, the small drives rarely turn out to be the cheapest option per TB.
Yes. but make sure that you create the parity partition first and add the second parition last so that you can easily get rid of the second parition and expand the parity partition without any hazzle if you need more parity space later.
Also consider making the parity parition a little bit larger than the largest data disk to compensate for the larger data blocks used by snapraid parity compared to your file system.
It works on file level limited to a single parition per drive (excluding the SMART feature and other drive related stuff).
I'm actually mostly talking about a sbc nas with usb disks so that is not a concern. It is always possible to add more disks easily. Or to just add another nas. They are cheap.
Ya, performance is not great but its mostly not a consideration for this.
So basically the plan is to run snapraid locally on each individual nas?
Ya. Well, I have a tower with 14 drives where I'm also installing it. But what I'm building right now and the plans for growing are those cheap nas.
I'm also wandering about how feasible it would be to run it inter-nas.
Like, having it run on each nas so that each nas has some internal safety with its own parity drives.
But for backup, I would like to try the following: Instead of having two identical nas mirroring each other for backup, I'm thinking about starting with 3 smaller ones and having one being an entire parity nas for the other two. That would allow me to grow them in different ways, either grow each individual nas in the number of drives in them or add another nas and so on. That would protect me against the loss of a full nas without having to keep an entire mirror all the time.
This is probably feasible by mounting all the nas drives on a machine where snapraid is installed. Just bring them all online to run scrubs and syncing when new data is added. Should work. Probably slow as hell but should work I hope. Anyone sees any pitfalls or has a better idea?
Assuming that you eventually have 10 drives connected via the network (hosted at the NAS devices) and that you have a 1 Gigabit switch or router somewhere in between this is what I expect that you will find:
DAS is a much more attractive alternative if you want all drives in a single snapraid array.
You are talking about the snapraid idea for using multiple NAS right?
Those times are perfectly acceptable.
I intend to only add new data in accumulated batches. I have no problem if the sync afterwards takes a long time. I'll just regulate the interval at which new data is added based on how long it takes to sync the new data.
I already have and will keep a fast but more space limited data storage for first stage backup storage with its own higher cost redundancy which is used for imediate storage and working and update data in batches from this to the less performing / increased space / lower cost storage.
The initial sync time is not even a consideration. It could take a month for all I care. It just needs to take less time than the time it takes to fill up the first stage storage.
My only considerations: how easy and cheap it is to expand the arrays and the amount of protection they have.
DAS is limited in both expandability and how costly it can become after some point.
A minimum amount of performance of the arrays for normal usage is a consideration, but performance for syncing is not.
If I need the performance during a syncing job, it can be interrupted and continued later per the manual (haven't tried this yet).
Cheers
General disk performance during sync at really slow speed will be good. The obvious risk is network congestion. But if you have that under controll and/or is not bothered by poor network performance during sync, then I don't see any problem.
Just keep in mind that if you add X amount of data to any individual data disk, then sync will typically need to read an equal amount of data from each data disk to be able to calculate new parity for all files which share the same parity blocks.
Ya, that makes sense. Thanks :)
That means that to reduce the amount of data that needs to be read in total one should always try to divide new data by all drives as equally as possible right?
Correct, assuming that they are evenly filled to begin with.
Last edit: Leifi Plomeros 2018-07-12