Re: [Moosefs-users] Backup strategies

[Steve W. <st...@pu...>]

On 04/03/2012 03:56 PM, Steve Thompson wrote:
> OK, so now you have a nice and shiny and absolutely massive MooseFS file
> system. How do you back it up?
>
> I am using Bacula and divide the MFS file system into separate areas (eg
> directories beginning with a, those beginning with b, and so on) and use
> several different chunkservers to run the backup jobs, on the theory that
> at least some of the data is local to the backup process. But this still
> leaves the vast majority of data to travel the network twice (a planned
> dedicated storage network has not yet been implemented). This results in
> pretty bad backup performance and high network load. Any clever ideas?
>
> Steve

We have four 22TB and one 14TB MooseFS volumes that we backup onto 
disk-based backup servers.  We used to use rsnapshot but now we use 
rsync in combination with ZFS snapshots.  Each evening before our backup 
run, we take a snapshot on the backup filesystem and label it with the 
date.  Then we run rsync on the volumes being backed up and only what 
has been modified since the previous backup is transfered over the 
network.  The result is the equivalent of taking a full backup each 
night and it's very easy to recover data.  I also use ZFS compression 
and dedup to help conserve space on our backup servers.  The dedup 
option is especially helpful when a user decides to rename a large 
directory; rsync may have to bring it across the network and write it to 
the filesystem but ZFS will recognize the data as duplicates of already 
stored data.

Steve