From: Oliver F. <Oli...@io...> - 2004-05-28 09:54:56
|
Hello, I'm currently trying to switch the Backup-system of my company to a disk-based system running on linux (along with switching the windows servers to samba, but that's another story). So I'm investigating BackupPC, so far with great results. So far I backup a linux server with rsync, and (for testing) a windows client via smbclient. We're gonna have a pool of about 320GB, on 2 disks probably connected by LVM, so there's room to add more disks if needed. For now the company uses a tape-based backup called Veritas Backup Exec, and the guy how uses it doesn't like it... Now I have to find out how to archive the data on the Backup-PC onto tape, for long-term-archival (they wanna be able to get back everything somehow) and for disaster-recovery. So I could just run amanda in the usual, client-server way, but it would duplicate the effort of BackupPC, pulling all the data over the net, and write it to tapes. Or I would tar the whole BackupPC pool in one archive, but it might be too big for one tape, and useful only for desabster recovery (Write back the whole thing, but not easily get out a single file, caused by the name mangling etc.) Or BackupPC_tarCreate: Let it create a tar archive for each back-upped machine and dump that to tape. Later we can directly pipe that to the machine in need for restore. Well, if anybody else is doing something like this, I really like to share their point of view. Thanks in advance, Oliver Freyd |
From: Joshua M. <jo...@bu...> - 2004-05-28 10:43:47
|
Hi Oliver, The beta version of BackupPC has a feature called Archive which allows you to do just that. Just specify the tape device you want to archive to and it'll stream the data in tar format (uncompressed, gzip'd or bzip2'd if you wish) Your other option if you want to automate it is to just write some cron scripts to run the BackupPC_tarCreate and pipe to the tape. What I did before the BackupPC Archive feature was written. I wrote a small script to grab the latest backups from all the hosts, write a small header file to the tape and then write each backup as a separate session. The header file was plaintext so anyone could put the tape in and just run cat /dev/st0 and it would print something like: Tape created on 28 May 2004. Contents: Server1 Server2 Server3 Then the user could fast forward to the session he wanted and restore from that. For disaster recovery, I'd suggest you just pull the data straight from backuppc. You can stream the tar directly from it, and pipe it through tar at the other end. I've got an emergency disk that I boot from, partition / format / mount the drives, then run: nc backupservername port | tar x on the backup server I have a script running via inetd on an unused tcp port that simply calls BackupPC_tarCreate. Once the tar restore is finished, simply chroot to the restored system, lilo and reboot. Depending on the amount of data, a bare-metal recovery of linux is about 5 minutes + data transfer time. In fact, I've been cheating and using this method for server duplication and installation (I think I installed from scratch twice now for about 20 servers here) Let me know if you want more details Regards, Josh. On Fri, 28 May 2004 19:54, Oliver Freyd wrote: > Now I have to find out how to archive the data on the Backup-PC onto > tape, for long-term-archival (they wanna be able to get back everything > somehow) and for disaster-recovery. |
From: Oliver F. <Oli...@io...> - 2004-05-28 11:43:19
|
Hi Joshua, thanks for this really helpful answer! Did you upgrade to the 2.1 beta version? Does it run reliably? Maybe I should just check it out, I've browsed the documentation and the "archive" feature sounds great... Joshua Marshall wrote: > Hi Oliver, > > The beta version of BackupPC has a feature called Archive which allows you to > do just that. Just specify the tape device you want to archive to and it'll > stream the data in tar format (uncompressed, gzip'd or bzip2'd if you wish) > > Your other option if you want to automate it is to just write some cron > scripts to run the BackupPC_tarCreate and pipe to the tape. What I did before > the BackupPC Archive feature was written. I wrote a small script to grab the > latest backups from all the hosts, write a small header file to the tape and > then write each backup as a separate session. The header file was plaintext > so anyone could put the tape in and just run cat /dev/st0 and it would print > something like: That's what I was thinking before, did you also keep some logs so that one could search for something and find out which tape to load? > For disaster recovery, I'd suggest you just pull the data straight from > backuppc. You can stream the tar directly from it, and pipe it through tar at > the other end. I've got an emergency disk that I boot from, partition / > format / mount the drives, then run: > > nc backupservername port | tar x > I've tried to use backupPC_TarCreate over ssh, that works nicely, too. ssh -l backuppc backupserver "path-to-backupPC_tarCreate bla bla" | tar -xvf - Well, quite a lot of typing, but works. i used it also with tar -d at the end of the pipe, to check the backup against the original data. Quite reassuring... > on the backup server I have a script running via inetd on an unused tcp port > that simply calls BackupPC_tarCreate. Once the tar restore is finished, > simply chroot to the restored system, lilo and reboot. Depending on the > amount of data, a bare-metal recovery of linux is about 5 minutes + data > transfer time. In fact, I've been cheating and using this method for server > duplication and installation (I think I installed from scratch twice now for > about 20 servers here) That's cool, too. I once used rsync to duplicate a server. Booted with knoppix, partitioned the harddrive, rsynced the root partition, installed grub, rebooted... Anyway, linux is great, there are so many ways to do things. Bye, regards, Oliver |
From: Daniel P. <da...@ri...> - 2004-05-28 11:01:17
|
On 28 May 2004, Oliver Freyd wrote: [...] > So I'm investigating BackupPC, so far with great results. > So far I backup a linux server with rsync, and (for testing) > a windows client via smbclient. We're gonna have a pool > of about 320GB, on 2 disks probably connected by LVM, so there's > room to add more disks if needed. That is a lot of storage for two disks without redundancy; given the cost of recovery, try to get them to spring for (at least) a RAID5 or RAID1 mirror to prevent you losing the lot when one drive goes bad. > For now the company uses a tape-based backup called Veritas Backup Exec, > and the guy how uses it doesn't like it... Neither did I, when I had to deal with the ... less than ideal way it worked. :) > Now I have to find out how to archive the data on the Backup-PC onto > tape, for long-term-archival (they wanna be able to get back everything > somehow) and for disaster-recovery. [...] > Or I would tar the whole BackupPC pool in one archive, but it might be > too big for one tape, and useful only for desabster recovery (Write back > the whole thing, but not easily get out a single file, caused by the > name mangling etc.) It makes disaster recovery a two-stage process, but only if the BackupPC pool (or the machine it is on) is lost. If you can manage it, this is probably one of the better paths, since BackupPC does recovery of data pretty quickly... > Or BackupPC_tarCreate: Let it create a tar archive for each back-upped > machine and dump that to tape. Later we can directly pipe that to the > machine in need for restore. ...but this is the most "traditional" solution. Pull the backup into BackupPC, then write out the tar files and back them up. Version 2.1 has tools to do this automatically, I believe. Daniel -- Nature provides a free lunch, but only if we control our appetites. -- William Ruckelshaus, _Business Week_, 18 June 1990 |
From: Kevin F. <kev...@hc...> - 2004-05-28 14:08:12
|
Oliver Freyd writes: > So I'm investigating BackupPC, so far with great results. > So far I backup a linux server with rsync, and (for testing) > a windows client via smbclient. We're gonna have a pool > of about 320GB, on 2 disks probably connected by LVM, so there's > room to add more disks if needed. First of all, get that backup on some type of redundency. We use a 3Ware Escallade IDE controller in a RAID-5 configuration. If you keep the data drive on LVM from there, you can add drives to the physical raid, attach that additional storage to the data partition, then expand the Linux partition via LVM. Much better system and relatively cheap. > For now the company uses a tape-based backup called Veritas Backup Exec, > and the guy how uses it doesn't like it... EEEEEEKKKKSSSSSSS I agree, nasty program that we could never get to work right. > Now I have to find out how to archive the data on the Backup-PC onto tape, > for long-term-archival (they wanna be able to get back everything somehow) > and for disaster-recovery. We actually have a great solution for this also.... We use a USB based removable hard drive. I connect this up on a regular basis and rsync it with the data drive. I then remove the drive, and place it in the fire-proof vault of the bank in lobby of our building. Now, we have a fire recovery also. In addition, it would be easy to get back either a single file or an entire directory or the entire backup in case of emergency. Hope this Helps Kevin Fries |