Re: [BackupPC-users] trying to improve the speed at which backuppc rsync back up processes a large binary file in incremental backups

[Les M. <les...@gm...>]

On Sun, Jan 8, 2012 at 4:48 AM, John Habermann
<jha...@co...> wrote:
>
> You can see that the backup of the /opt share takes nearly the total
> time of the incremental taking about 8 and half hours to complete while
> the backup of the /opt rsync share in the full backup takes about 3 and
> half hours. The full backup is slightly longer than what it takes if I
> just do a rsync over ssh copy of the file from the client server to the
> backup server.
>
> I have found that rsync seems to always transfer the whole file when
> copying this file from the client server to the backup server:
>
> # rsync -avzh --progress -e ssh
> administrator@isabella:ExchangeDailyBackup.bkf ExchangeDailyBackup.bkf
> Password:
> receiving incremental file list
> ExchangeDailyBackup.bkf
>      54.44G 100%   10.66MB/s    1:21:10 (xfer#1, to-check=0/1)4
>
> sent 3.31M bytes  received 3.27G bytes  486.33K bytes/sec
> total size is 54.44G  speedup is 16.65.

Note that you have used the -z option with native rsync, which
backuppc doesn't support.  You can add the -C option to ssh to get
compression at that layer when you run rsync over ssh, though.

> My questions for the list are:
> 1. Is it reasonable for rsync to transfer the whole file when copying a
> large ntbackup file?

Yes, those files may have little or nothing in common with the
previous copy.   If compression or encryption are used they will
ensure that no blocks match and even if they aren't, the common blocks
may be skewed enough that rsync can't match them up.

> 2. Why does an incremental backup of this file take so much longer than
> a full backup of it or a plain rsync of this file?

That doesn't make sense to me either.  Are you sure that is consistent
and not related to something else that might have been using the link
concurrently?

-- 
   Les Mikesell
     les...@gm...