From: Les M. <les...@gm...> - 2011-10-26 01:48:21
|
On Tue, Oct 25, 2011 at 8:13 PM, Steve M. Robbins <st...@su...> wrote: >> Unless you have several hosts that hold duplicate data, after you get >> the initial fulls option #1 with rysnc transport over ssh or a vpn >> with compression enabled won't be moving more data than other ways you >> might attempt it. > > At the risk of exposing my ignorance of BackupPC internals, I don't see > how this is possible. For a full back-up, isn't it true that all the > files are transferred to the backup host, then compared to the pool? With tar/smb xfers, that is true. With rsync/rsyncd, both fulls and incrementals walk the tree of the previous full run for that host, transferring only the differences - and after that any duplicates that are found are linked to the pool. The difference between an rsync full and incremental is that the incrementals quickly skip files where the directory timestamp and length match where fulls do a read and block checksum comparison of everything, but that does not use a lot of bandwidth. > One host of mine has 1.4M files totalling 550 GB, but the last full > backup recorded 1400 new files totalling 57GB. Thus option #1 would > transfer all 550 GB, whereas my proposal would transfer a tenth of > that. No? Depends on the xfer mode. The one place where doing some smart copy of the pool might help a lot would be if you have a large number of hosts and do an OS update or something similar that creates duplicate new files on all of them. Even with rsync, a remote run will transfer all the changes separately for each host even though they end up pooled after the copy. -- Les Mikesell les...@gm... |