From: Jeffrey J. K. <bac...@ko...> - 2011-06-07 14:06:56
|
Andrew Schulman wrote at about 09:31:14 -0400 on Tuesday, June 7, 2011: > > If they don't change between runs, backuppc will pool the new instance > > with the previous, although a full backup may still take a long time as > > the block checksum verification is done over the whole file. If they do > > change and you use rsync, only the differences will be transferred (to > > the extent that rsync can find them and resync on the matching parts in > > a huge file), but the server will use the old copy and the differences > > to reconstruct a full-sized copy which is slow and won't be pooled with > > anything else. > > Although this is true generally, I don't think it applies in this case. > What you say is true in the case of a single file that has changes in it. > Then rsync efficiently transfers only the delta. > > But that doesn't apply in this case, because backuppc doesn't change the > existing VM image in the storage pool. Instead, it creates an entire new > file, which then has to be transferred completely. Even if the new file is > 99.99% identical some other file in the pool, it won't help because rsync > isn't comparing that file to every other file in the pool. It's only > comparing the source and target copies of the new file, and the target > doesn't exist yet, so it has to be copied completely from the source. > > Someone please correct me if I'm wrong about that. I believe this is what happens... If the file name & path is unchanged then BackupPC/rsync knows to compare it with the existing pooled file. The file has to be read and checksum'd on both ends (and possibly decompressed on the server side if using the cpool) and if there are *any* changes then a new version is constructed and written to the pool based on the delta and the existing pooled version. However, only the deltas and not the entire file is transferred across the slow WAN link -- which is the point of this thread. So the speed will be limited primarily by the read/write/decompression speed on the client & server plus the overhead of rsync's delta algorithm. If changes are limited, then the WAN speed will generally not be rate limiting. |