[BackupPC-users] Rotating log files causing backups to fall behind.

[John R. <rou...@re...>]

Hi all:

I am noticing an issue in our backuppc installation.

Every Monday we have 20-30 hosts (of 84) that were not backed up in
the prior 24 hours. It seems the /var/log filesystem takes much longer
on the sunday/monday backups than it takes during the rest of the
week.

I am using the rsync backup method across a WAN and run 4 backups at a
time rate limited to 1Mb/s to stay under our bandwidth allocation.

The backup time runs from 23:00->12:00 (13 hour window).

I have been able to see only 2 of the 4 slots in use during the backup
window. To try to keep all 4 slots full I changed the wakeup schedule
to:

  $Conf{WakeupSchedule} = [22, 22.5, 23, 23.5, 0, 0.5, 1, 1.5, 2, 2.5,
  3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 9, 10, 11, 12, 13, 14, 15,
  16, 17, 18, 19, 20, 21];

and this seems to have helped somewhat.  The nightly run takes about
30-40 minutes, so I started the WakeupSchedle at 22:00 so it would
complete before the backup window starts at 23:00.

I think the problem is that log files are rotated on Sunday AM, so the
Sunday night/Monday morning backup run has to transfer the entire
contents of these log files because the names change during the log
rotation. Even though I have almost all of /var/log/messages in the
backuppc cpool, on the client machine the file is renamed to
/var/log/messages.1 during the rotation while backupc's copy of that
same file is named /var/log/messages so rsync copies the entire
contents onto a new messages.1. This continues for messages.2 and so
on.

I think I have tuned backuppc as far as I can to try to eliminate dead
time in the backup window.

Does anybody have any ideas on getting rsync to be able to use the
contents of the prior files? I am not using filled backups, but one
thought is to use filled backups and (via a hook) duplicate the
rotation of log file in the backuppc tree before the rsync operation
is done. So the backuppc's /var/log/messages would move to
/var/log/messages.1 and thus match the file layout (and contents) of
the client machine.

Thanks for any ideas.

-- 
				-- rouilj

John Rouillard
System Administrator
Renesys Corporation
603-244-9084 (cell)
603-643-9300 x 111