Your data requires more parity than the available space

[Hendrik Friedel]

Hello,

first of all, let me thank you for your work in SnapRaid!

I just set up an Array:

/dev/sda1       3,1T    3,1T     0  100%  CONTENTFILE+PARITY
/dev/sdc        2,7T    2,3T  352G   87%  CONTENTFILE+DATA
/dev/sdd        2,7T    2,0T  637G   76%  CONTENTFILE+DATA
/dev/sde        2,7T    2,0T  597G   78%  CONTENTFILE+DATA
/dev/sda2       1,5T    1,3T  100G   93%  Non-Protected data on the 'rest' of sda

As you can see, the drive holding the Parity information has 4,6TB, while the other have 2,7TB.
I decided to partition the drive, so that SnapRaid can be sure to have the 3.1TB reserved (maybe I could also use quota?)

But despite reserving >10% excess for Parity, I run out of available Space.
The sync ends with:
Your data requires more parity than the available space
Can someone explain, what the reason for this could be?
The Drives are to a large extend filled with Videos and Backups, but also with Photos and personal Data.

The only special thing is, that I use rsnapshot for backing up, which uses many hard links. Could this be the cause?

What would be the recommended way here? Should I make one large partition (the whole 4.6T) so that the Parity File can grow? But how can I then ensure that space is reserved for futre growth and that not another tool moves data there?

Regards,
Hendrik

[Leifi Plomeros]

snapraid list combined with grep and a suspected filename should be able to confirm if snapraid interprets the hardlinked files as multiple copies.

[Hendrik Friedel]

Thanks.
The files are interpreted as hardlink. So that'll not be the reason.
What else could it be?

Is this normal?

Regards,
Hendrik

[Leifi Plomeros]

You typically need 1 GiB extra space per ~8000 files for overhead (aka waste) on the parity disk.
So unless you have more than 3 million tiny files on any of the 2.7 TiB data disks this is not normal.

My best guess is something filesystem related.

[MW]

Hello,

I have a similar question, my setup is as follow:

df -h

Filesystem Size Used Avail Use% Mounted on
/dev/sdd2 2.6T 1.6T 982G 63% /mnt/snapraid-parity-disk-1
/dev/sda2 2.6T 1.6T 815G 67% /mnt/snapraid-data-disk-1
/dev/sdc2 2.6T 134M 2.4T 1% /mnt/snapraid-data-disk-3
/dev/sdb2 2.6T 134M 2.4T 1% /mnt/snapraid-data-disk-2
none 7.6T 1.6T 5.6T 23% /mnt/snapraid-pool

find /mnt/snapraid-data-disk-1/ -type f | wc -l

73815

That's 3 x 2.6TB for data (EXT4 with 5% reserved blocks) and 1 x 2.6TB for parity (EXT4 with 0% reserved blocks).

First data disk contains now 73815 files (1.6TB) and parity file already has 1.6TB. Is that OK? I'm afraid that I will run of space for parity file before I use rest of available data space (5.6TB).

Best Regards,

[Leifi Plomeros]

You only need as much parity as the single fullest disk (+a little extra for overhead/waste on that disk).

If you put 1 TB each on sdb2 and sdc2 it will require no additional parity at all.

1 MiB on D1 + 1 MiB on D2 + 1 MiB on D3 = 1 MiB on parity disk

[Hendrik Friedel]

Hello,

for me, my fullest disk looks like this:
find /media/678c601d-8d2e-4be0-9226-d20a6f772f8b -type f | wc -l
5074629
--> 5Million Files

So that will be the reason?!

Greetings,
Hendrik

[Leifi Plomeros]

Yes defintely the reason.

Your parity disk needs to have room for about 0.65 TB extra parity space with 5 million files.

Possible workarounds:
1. Put small files on a smaller disk (a full 2 TB disk with 5 million files = 2 TB + 0.65 parity size required)
2. Leave more than 0.65 TB free space on the disk with 5 million files
3. Use a larger parity disk
4. Put the tiny files into archive files in order to decrease the number of files

Note: The free space in option 2 doesn't really need to be unused. An unprotected folder outside of the array or a folder ignored by snapraid would also be OK.

[Master CATZ]

/dev/sda1 3,1T 3,1T 0 100% CONTENTFILE+PARITY

remove the CONTENTFILE its not needed on the PARITY drive

also I would still have the partition with some blocks still in reserve even if its 0.1 %
better to play it safe than sorry just incase the inode can not be updated

personaly I have my EXT4 data drives with 1% reserve than have 0.25% reserved on the parity drive