[BackupPC-users] Troubleshooting a slow backup

[Tony N. <tn...@st...>]

Hi All,

I have been using BackupPC very successfully for several years at 
several sites now.  I'm presently having an issue with a few backups 
running very slowly and I'm looking for help troubleshooting the issue.

BackupPC Server:

backuppc ~ # uname -a
Linux backuppc 2.6.18-gentoo-r2 #5 SMP PREEMPT Tue Nov 21 16:32:02 EST 
2006 i686 Intel(R) Xeon(TM) CPU 2.66GHz GenuineIntel GNU/Linux

backuppc ~ # rsync --version
rsync  version 2.6.9  protocol version 29

backuppc ~ # dmesg | grep mem
Memory: 2069124k/2097088k available (2389k kernel code, 26728k reserved, 
762k data, 200k init, 1179584k highmem)

backuppc ~ # df -h /var/lib/backuppc
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             1.2T  773G  397G  67% /var/lib/backuppc

backuppc proc # ps -ef | grep rsync
apache   14086 14080  1 02:10 ?        00:06:17 /usr/bin/ssh -l root 
target-machine /usr/bin/rsync --server --sender --numeric-ids --perms 
--owner --group -D --links --times --block-size=2048 --recursive 
--one-file-system -v --exclude /proc --exclude /dev --exclude /tmp 
--exclude /mnt --exclude \*.tmp . /ha/

Target Machine:

www2 ~ # uname -a
Linux www2 2.6.22-gentoo-r5 #1 SMP Wed Sep 12 12:09:30 EDT 2007 x86_64 
Intel(R) Xeon(TM) CPU 3.00GHz GenuineIntel GNU/Linux

www2 ~ # rsync --version
rsync  version 2.6.9  protocol version 29

www2 ~ # dmesg | grep -i mem
Memory: 8188692k/9175040k available (2788k kernel code, 199180k 
reserved, 1095k data, 336k init)

www2 ~ # df -h /ha
Filesystem            Size  Used Avail Use% Mounted on
/dev/drbd0            184G   27G  148G  16% /ha



I'm backing up the /ha partition from the target machine to BackupPC. 
The machines are not on the same subnet, but they are in the same 
physical office separated by an (old) 100Mb switch.

As you can see the partition I'm backing up is using 27G, but on the 
same network I wouldn't think that should take upwards of 12-14 hours 
which is what it is currently taking.

The filesystem contains (for the most part) a directory structure where 
files and directories are added much more than they are removed.  There 
are very few files that are actually modified.

For completeness sake, there is a "parent" directory on the /ha share 
that contains 363 (at the moment) subdirectories.  Each of these 
subdirectories contains up to 4 special purpose directories where files 
are stored using a numerical pathname (similar to BackupPCs pool/cpool) 
that is 8 directories deep.  For example:

/ha/ih/data/uploads/Company86/Trade/0/0/0/1/1/1/1/1

Each of these directories has 4 or 5 files in it (2-3 page word 
documents generally) so I would imagine that rsync is spending a lot of 
time just traversing directories.

What I would like to do is figure out the best way of determining if the 
source of the slowness is the target server, the backuppc server or a 
network bottleneck that I just can't imagine.

If there is further information I can send in for evaluation please let 
me know and I will get it together as quickly as possible.

Thank you for your help in advance
Tony Nelson
Starpoint Solutions