I'm following the instruction to setup mergerfs+snapraid to make an expandable storage, as the FAQ suggested, 8 data disks(24TB total)+2 parity disks(2 * 14TB, 2-parity). The plan is to add other 5 * 14T and one each time. Here is the current setup, and I found each parity file almost the same size as the data disk, where the data disk was about 1.2TB, and parity is 943GB after the first sync. If snapraid need the same capacity as data disk why not use raid instead. the FAQ says, every 7 disks need one parity and parity diks capacity need to be larger than any single data disk, is this true?
Filesystem Size Used Avail Use% Mounted on
tmpfs 38G 28M 38G 1% /run
efivarfs 256K 182K 70K 73% /sys/firmware/efi/efivars
/dev/mapper/ubuntu--vg-ubuntu--lv 437G 37G 378G 9% /
tmpfs 189G 16K 189G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock mergerfs 24T 1.7T 23T 8% /mnt/storage
/dev/sda2 2.0G 183M 1.7G 10% /boot
/dev/sde1 1.8T 64M 1.8T 1% /mnt/disk2
/dev/sdb1 1.4T 64M 1.4T 1% /mnt/disk1
/dev/sdj1 1.8T 64M 1.8T 1% /mnt/disk7
/dev/sdk1 1.8T 64M 1.8T 1% /mnt/disk8
/dev/sdi1 5.5T 64M 5.5T 1% /mnt/disk6
/dev/sda1 1.1G 6.2M 1.1G 1% /boot/efi
/dev/sdh1 1.8T 64M 1.8T 1% /mnt/disk5 /dev/sdd1 13T 943G 12T 8% /mnt/parity2
/dev/sdc1 13T 943G 12T 8% /mnt/parity1
/dev/sdg1 2.7T 64M 2.7T 1% /mnt/disk4
/dev/sdf1 7.3T 1.7T 5.6T 24% /mnt/disk3
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm now understanding how parity works and why parity disk must be no less than the largest disk in the array. i do recommend snapraid+mergerfs for home data storage. lvm requires all the disks move all together, and mergerfs can move one each time.
Last edit: Gilbert 2024-06-22
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Actually the parity disk should be bigger than the data disks or you run into problems.
I have 5x12TB data and 2x12TB parity and I have to hold back about 300GB on each data disk with minfreespace=300G on my mergerfs fstab line otherwise my parity drives fill up and snapraid won't sync.
See section 7.8 blocksize SIZE_IN_KIBIBYTES in the snapraid manual for a better explanation.
And you could use RAID, nothing stopping you, there are reasons we use Snapraid instead. It is explained quite well on the main website.
Last edit: Night Rider 2024-07-07
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm following the instruction to setup mergerfs+snapraid to make an expandable storage, as the FAQ suggested, 8 data disks(24TB total)+2 parity disks(2 * 14TB, 2-parity). The plan is to add other 5 * 14T and one each time. Here is the current setup, and I found each parity file almost the same size as the data disk, where the data disk was about 1.2TB, and parity is 943GB after the first sync. If snapraid need the same capacity as data disk why not use raid instead. the FAQ says, every 7 disks need one parity and parity diks capacity need to be larger than any single data disk, is this true?
Filesystem Size Used Avail Use% Mounted on
tmpfs 38G 28M 38G 1% /run
efivarfs 256K 182K 70K 73% /sys/firmware/efi/efivars
/dev/mapper/ubuntu--vg-ubuntu--lv 437G 37G 378G 9% /
tmpfs 189G 16K 189G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
mergerfs 24T 1.7T 23T 8% /mnt/storage
/dev/sda2 2.0G 183M 1.7G 10% /boot
/dev/sde1 1.8T 64M 1.8T 1% /mnt/disk2
/dev/sdb1 1.4T 64M 1.4T 1% /mnt/disk1
/dev/sdj1 1.8T 64M 1.8T 1% /mnt/disk7
/dev/sdk1 1.8T 64M 1.8T 1% /mnt/disk8
/dev/sdi1 5.5T 64M 5.5T 1% /mnt/disk6
/dev/sda1 1.1G 6.2M 1.1G 1% /boot/efi
/dev/sdh1 1.8T 64M 1.8T 1% /mnt/disk5
/dev/sdd1 13T 943G 12T 8% /mnt/parity2
/dev/sdc1 13T 943G 12T 8% /mnt/parity1
/dev/sdg1 2.7T 64M 2.7T 1% /mnt/disk4
/dev/sdf1 7.3T 1.7T 5.6T 24% /mnt/disk3
I'm now understanding how parity works and why parity disk must be no less than the largest disk in the array. i do recommend snapraid+mergerfs for home data storage. lvm requires all the disks move all together, and mergerfs can move one each time.
Last edit: Gilbert 2024-06-22
Actually the parity disk should be bigger than the data disks or you run into problems.
I have 5x12TB data and 2x12TB parity and I have to hold back about 300GB on each data disk with minfreespace=300G on my mergerfs fstab line otherwise my parity drives fill up and snapraid won't sync.
See section 7.8 blocksize SIZE_IN_KIBIBYTES in the snapraid manual for a better explanation.
And you could use RAID, nothing stopping you, there are reasons we use Snapraid instead. It is explained quite well on the main website.
Last edit: Night Rider 2024-07-07
You are right, nice to have parity disks larger than data disks, it's much safer for SYNC. while 16TB parity for 14TB data disks is not cost effective, i will choose to use mkfs.ext4 -m 5 to format the largest data disk and leave minfreespace=50G unchanged.
for parity disk formatting i run "mkfs.ext4 -m 0 -J size=4 -i 67108864 DEVICE", it's working good and the parity size is only about 100GB larger than any single data disk.
Filesystem Size Used Avail Use% Mounted on
tmpfs 38G 21M 38G 1% /run
efivarfs 256K 254K 0 100% /sys/firmware/efi/efivars
/dev/mapper/ubuntu--vg-ubuntu--lv 437G 56G 359G 14% /
tmpfs 189G 32K 189G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 98G 0 98G 0% /mnt/memdisk
mergerfs 68T 63T 4.8T 93% /mnt/storage
mergerfs 5.5T 4.2T 1.3T 77% /mnt/dbSSD
/dev/sdb2 2.0G 183M 1.7G 10% /boot
/dev/nvme0n1p1 1.9T 1.3T 646G 66% /mnt/dbSSD2T
/dev/nvme1n1p1 3.6T 3.0T 652G 83% /mnt/dbSSD4T
/dev/sdb1 1.1G 6.2M 1.1G 1% /boot/efi
/dev/sdh1 1.8T 1.6T 256G 87% /mnt/disk2
/dev/sdn1 1.4T 1.1T 256G 82% /mnt/disk1
/dev/sde1 1.8T 1.6T 256G 87% /mnt/disk7
/dev/sdc1 1.8T 1.6T 256G 87% /mnt/disk8
/dev/sdd1 5.5T 5.4T 44G 100% /mnt/tmp
/dev/sdi1 7.3T 7.0T 256G 97% /mnt/disk3
/dev/sdl1 13T 12T 846G 94% /mnt/disk10
/dev/sda1 13T 12T 845G 94% /mnt/disk5
/dev/sdm1 2.7T 2.5T 256G 91% /mnt/disk4
/dev/sdj1 13T 12T 846G 94% /mnt/disk9
/dev/sdk1 13T 12T 846G 94% /mnt/disk6
/dev/sdg1 13T 12T 768G 95% /mnt/parity2
/dev/sdf1 13T 12T 768G 95% /mnt/parity1
tmpfs 38G 12K 38G 1% /run/user/1000