Ben

Well, I moved my directory with many files into a .tar.gz, removed the originals, and excluded the archive from SnapRAID. snapraid sync is getting killed again while loading state from my content file. I'm open to more suggestions, but if there aren't any I'll probably just build new parity and content files. I've got a hot spare in this machine, so I'll just unmount the old parity drive and mount the new one in place for that so I'm not in a state with no parity (though I'm not sure how I would...

Assuming I have enough space for that file, which I probably don't unless the compression is crazy efficient. This is a 240 GB SSD that is usually around 90% full. Another possible option is to store my backups in a virtual file system on disk[1]. ie. create a sparse file on disk and mount it as another ext4 system. I could run my rsyncs to that file and then let SnapRAID deal with the gigantic file. If I find that there is just too much changing and each sync takes forever, I could probably just...

Ah, interesting. There they are zipping on the source side and the rsyncing to the destination. I guess I didn't realize that zip archives were that consistent in their internal layout. I'm not sure this will work for my setup. My script is running on the backup server with read only access to the source and I don't think I have enough space to make a zip on the the source machine before doing the transfer anyway. This method also eliminates my ability to keep a history of snapshots in a small amount...

Could you provide a link to what you are reading about rsync archiving? I know that rsync supports compression during transmission and has an archive flag that aims to reproduce all permissions and what not for a file, but I hadn't seen anything about lumping files together in an archive file.

Okay, I'll give it a shot. I guess worst case if I can't get it running again I'll just rebuild my parity and content files. Thanks!

They do change occasionally, though I could stop that temporarily. Currently I'm using rsync every night to copy the Linux OS partition from my main workstation and creating hard links to unchanged files to give me snapshots from multiple points in time. As a short term thing I could disable that backup and then move/archive the files. Deleting the files will keep the memory usage down and let status/sync/scrub finish without being killed?

My SnapRAID pool contains a bunch of backup files. One particular directory contains a very large number of small files which I know is a bad fit for snapraid. I'm planning to either move these files to another filesystem or just get SnapRAID to ignore them since they are just for emergency backup. The problem is that SnapRAID has been tracking them up until this point and now snapraid status gets killed each time it runs, presumably by Linux seeing it is using too much RAM + swap space. I have added...

I figured it would be slower, but not quite that slow. I don't mind it being a bit slow as it is just for offsite backup but wanted to make sure this was a reasonable speed. As soon as I inherit a machine with USB 3 this laptop will get swapped out. The bay supports UASP so the situation should be much better then. To be clear, I'm not trying to complain about SnapRAID performance, just trying to understand where the bottlenecks are so I can be strategic in my upgrades. The output of snapraid sync...