Change archive directory
From backuppc
The topic of this wiki page mixes up two independent issues which may or may not arise at the same time. Do you want to:
- change the name of the archive directory from /data/BackupPC to /var/lib/backuppc (or any other values), because that better fits your taste?
- change the storage location of the BackupPC pool, for one of several good reasons?
You may think changing the storage location implies changing the name of the archive directory. That is not true.
In general, changing the name of the archive directory does not involve copying any files (if your new name points to a different partition than the old name did, you are in fact also changing the storage location!). Changing the storage location does. Copying the BackupPC pool can be non-trivial once you have done one or more backups due to the extensive use of hard links. Try to avoid this by choosing wisely when you install BackupPC. Plan for extensibility by using LVM, a RAID mode that supports resizing or something equivalent (EVMS?), and a resizable file system, or make sure your policy allows you to discard your online backup history, should you ever need to expand your storage (you can of course keep the history as such, but restoring files from it will be more complicated).
We'll start with the second point because it is probably the more common, and because it may lead you to want to do the first - or at least think you do.
Contents |
Changing the storage location of the BackupPC pool
If you're halfway serious about using BackupPC, you will want to keep your backup data on a separate partition, or better yet a separate (set of) disk(s). This decision is beyond the scope of the BackupPC default installation, which will simply create a directory wherever you (or the packager of your BackupPC package) tell it to - on whichever file system that happens to be on. It's a good idea to change the storage location of the BackupPC pool, and an even better idea to do this right at the start, before doing any backups. Once you've decided where on which physical disk you want your BackupPC pool to live, your options are:
- Mount your partition (disk, block device) on the directory BackupPC uses as $Conf{TopDir} (/data/BackupPC, /var/lib/backuppc, just to name two commonly used values).
- Mount your partition anywhere else. Point a softlink from $Conf{TopDir} (/data/BackupPC, /var/lib/backuppc, ...) to where your partition is mounted.
- Mount your partition anywhere else and use a bind mount to make your partition also visible where BackupPC expects the pool.
- Mount your partition anywhere else and change the name of the archive directory in BackupPC.
The first option will only work if you dedicate the whole partition to BackupPC (which you should), the others are really independent of where you put your pool. Choose your option. They are listed in my order of preference, though others are known to disagree.
For options 1 and 3 (and maybe 2), you can set things up before installing BackupPC to avoid any copying whatsoever. Just create the TopDir directory, do the mount described below, and install BackupPC. (If there are permission problems, please report them on the mailing list with a reference to this page, so I can update the instructions.)
If you cannot or do not want to do that, there is no reason to worry: copying the directory structure from a fresh BackupPC install (without any contained backups) is trivial and will work with any of "cp", "rsync", or "tar".
Chances are, you've already installed BackupPC and perhaps done some backups, maybe even many. You might have come here either to change your decision (i.e. put your BackupPC pool on a seperate storage medium) or because you've already done that, but you need to change the physical medium (defective or too small disk). You will need to copy the pool. Unlike with a fresh install, this is non-trivial once a significant amount of backup data has accumulated (which might be after only a few large backups). There are no definite figures when this will become difficult. What won't work for others might work perfectly well for you. Try it out, but know what to expect and when to abort it. See the section Copying for some alternatives you can try.
For simplicity, I'll use a "cp" command in the following examples. Note that you need to stop BackupPC before copying the pool.
Let's assume your BackupPC pool is located under /data/BackupPC.
Mounting on TopDir
If your new partition is /dev/md0, execute the following commands as root
mount /dev/md0 /mnt cp -dpR /data/BackupPC/. /mnt umount /dev/md0 mv /data/BackupPC /data/BackupPC.orig mkdir /data/BackupPC mount /dev/md0 /data/BackupPC
You can remove /data/BackupPC.orig once you're satisified you don't need it any more. If your /data/BackupPC is already a mounted file system, umount it instead of the "mv" and "mkdir" sequence (that's important, because "mv" would effectively try to copy the data).
Edit your /etc/fstab to reflect the change.
Softlinking TopDir
If you don't want to mount on TopDir, you've already got your destination mounted somewhere. Let's say you want to use /export/disk/backuppc for storing your data (whether that's a mount point or not). We're still root for the following commands:
cp -dpR /data/BackupPC/. /export/disk/backuppc mv /data/BackupPC /data/BackupPC.orig ln -s /export/disk/backuppc /data/BackupPC
Again, you can remove /data/BackupPC as soon as you're satisfied you don't need it any more. If /data/BackupPC is a mounted file system, instead of the "mv", you'd need to umount it, remove the directory, and update /etc/fstab.
Bind-mounting TopDir
A bind mount is, in some ways, similar to a hardlink. You get to see the same contents in a different place in your file tree, and the reference is not dynamically looked up on access. A softlink can point to something that does not exist (has been renamed or not mounted), a bind mount cannot. Access may be marginally faster, because you don't have the dynamic lookup, though I doubt that makes much of a difference (whenever it matters, the relevant inodes should be cached).
In contrast to a hardlink (which can only reference a file on the same file system), a bind mount needs to be re-established after every reboot (as it can cross file system boundaries, it cannot be stored in the file system). The initial command sequence is
cp -dpR /data/BackupPC/. /export/disk/backuppc mv /data/BackupPC /data/BackupPC.orig mkdir /data/BackupPC mount --bind /export/disk/backuppc /data/BackupPC
The note from "Mounting on TopDir" applies. An fstab line for a bind mount (to automatically reestablish it after reboot) looks like this:
/export/disk/backuppc /data/BackupPC none bind
Changing storage medium
This is an advanced case that suggests you have already done a lot of backups. You basically need to copy the file system the pool is on and then mount the copy instead of the original device on the directory where you mount it. See the section Copying below.
Copying pool for change of archive directory
If you're simultaneously changing the archive directory in BackupPC and moving the storage location of the pool, you will need to copy the pool:
cp -dpR /data/BackupPC/. /export/disk/backuppc
Since you are going to tell BackupPC to look in a different location (see next section), you don't need to rename the source directory, though you can, of course, delete it at your convenience ;-).
Changing the name of the archive directory
The name of the archive directory (TopDir) is partly configurable through the BackupPC main configuration file (/etc/BackupPC/config.pl or /etc/backuppc/config.pl, possibly /data/BackupPC/conf/config.pl on older installations). Because the main configuration file used to be below TopDir, and BackupPC needs to be able to deal with that situation for seamless upgrades, it is also hardcoded into the Perl module BackupPC::Lib. Starting with BackupPC version 3.2.0, it is sufficient to set
$Conf{TopDir} = '/export/disk/backuppc';
in your toplevel configuration file (FIXME: is this possible via the CGI configuration editor?). For earlier versions of BackupPC, you need to additionally change BackupPC::Lib (/usr/local/BackupPC/lib/BackupPC/Lib.pm, /usr/share/backuppc/lib/BackupPC/Lib.pm or similar) accordingly. Look for "sub new". Near the beginning, there's an assignment to $topDir or a usage of "$topDir || '/some/path'" or something. Change the path.
If you are not also changing the storage location (sure?), you can simply rename (or remount) your existing archive directory to the new value:
mv /data/BackupPC /var/lib/backuppc
or
mount --move /data/BackupPC /var/lib/backuppc
If you are using the 'mv' example, that, of course, means that /data/BackupPC and /var/lib/backuppc are on the same file system - in this case the root file system - otherwise you would be changing storage location. You shouldn't keep the BackupPC pool on your root file system.
You should also note that TopDir is usually used as HOME directory of the "backuppc" user account. If you are changing TopDir, you will also need to update /etc/passwd accordingly.
You may also need to reload/restart your web server (commonly Apache) once the change has be made to make the BackupPC web interface access the new location.
Reasons for changing the storage location
There are several reasons for changing the physical location where your backups are stored.
- Logical structure. Your backups are not part of your OS installation. In particular, if you choose to reinstall your OS, you will not want to wipe out your backups. You may even be making backups expressly for this case.
- Fault isolation. You are keeping backups in case something with your computer goes wrong. If something with your computer goes wrong, you don't want it to take your backups with it. Similarly, if something with your backups goes wrong, you don't want it to take your system and data with it.
- Space allocation. You don't want your BackupPC pool filling up to interfere with system operation. BackupPC tries to avoid the pool from running completely full, but it cannot guarantee that.
- Independence. If your BackupPC pool is on a seperate file system, you can mount it whereever you want or need to. Changing the path does not involve copying data, just re-mounting the file system on a different mount point.
- Performance. Backups put heavy load on the disk subsystem. Disks work in parallel. Spreading the load makes things faster. Keeping backup I/O off your system disks should make the system much more responsive during backups. If you are backing up data of the computer BackupPC is running on, this is even more so, because you avoid disk seek operations between reading data and writing backup.
- Redundancy. People often put their BackupPC pool on RAID disks. Contrary to popular belief, RAID does not always save you from data corruption problems, but it does mean that if one of your disks dies, your data survives [correct RAID level assumed].
- Even more redundancy. People like to keep off-site copies of their BackupPC pool.
- Space exhaustion. If your BackupPC pool fills up, you may have to change the storage location (then again, you may not have to, if you were wise enough to prepare for this case).
- Faulty hardware. If you find out that your BackupPC pool disk is dying (e.g. from the SMART values or I/O failures), you will desperately want to move your pool to a new disk.
- Change of file system. If you want to use a different file system, you will need to copy your pool to a different location. At least I don't know of any in-place file system converters for UNIX file systems.
Copying
What alternatives are there to copying the pool with "cp" and why would I use them?
- rsync. You need the options "-aH", that is important. rsync is originally meant for copying across the network, but it also copies locally quite nicely. You should be able to interrupt an rsync copy and restart it later without needing to re-copy the files it has already done (FIXME: does that case handle hardlinks correctly?). When dealing with the BackupPC pool, rsync has different limitations than cp - don't ask me whether it's better or worse. It's simply something different to try.
- tar. tar creates or unpacks a tar archive. Piping a "tar -c" to a "tar -x" effectively gives you a copy operation over an arbitrary data stream (a pipe in the simple case). Combined with ssh or netcat or anything else that transparently passes on data, you can use tar for copying across the network. tar should be available on any UNIX system, rsync may not be. Again, if copying doesn't work, tar is something different you could try.
While these are generic tools that copy files through the file system layer, there are other tools that bypass it. This is generally faster (possibly by orders of magnitude) and may also circumvent the problems copying a BackupPC pool poses. However, it means that you need to keep using the same file system type for your pool file system.
- dump. Some people advocate using your file system's variant of the "dump" command, though it's not really clear whether this actually does bypass the file system layer. FIXME: someone please describe this in more detail.
- dd. Disk level copy of the file system. Requires a destination partition at least as big as the source partition (if your file system supports resizing, you can make it take advantage of a larger destination partition after copying). Will copy the entire partition, even if only 1% of it is used.
- dd_rescue. dd-like tool that handles disks with read errors better than dd (which simply aborts at the first error). Can copy backwards, restart at an offset, retry to read the source disk, gives feedback on progress etc.
- pvmove. If your BackupPC pool is on an LVM LV, you can use the normal LVM tools to move it around. This only partly qualifies as copying or changing the storage location, because, in terms of block device name (and probably major and minor device numbers), the storage location remains unchanged. It can move data from one physical storage device to another one, though. You can pvmove an LV containing a mounted, active file system, though doing it during backup is unlikely to be practical :).
What to expect when copying doesn't work
When your pool is too large to be correctly handled by the generic tools, any of the following may happen, depending on which of the tools you are using.
- At some point, the command aborts with an "out of memory" type message (or, if you are less lucky, without a meaningful message).
- Your destination partition quickly fills up, almost to the point you would expect, then the command continues for days without apparent progress. The problem here is that copying the data is reasonably fast, but re-establishing all the hardlinks can take virtually forever. There is no real indication how long the process will take to complete - presuming it does at all without crashing at some point. So, you might be 5 minutes from completion or 5 years, when you finally kill the copy process.
