Bowie Bailey <Bowie_Bailey@BUC.com> wrote on 02/23/2011
> It seems like something in your setup is causing problems. BackupPC
> should not be that slow. I use rsync on most of my backups and
> see anything like this.
> My largest machine:
> 1.8 million files
> Full backup: 10 hours
> Inc backup: 1 hour
This parallels fairly closely with the numbers I see
(and wrote about separately).
> And these machines are nothing special. The
backup server is an Athlon
> 64 1Ghz w/ 1G ram. The server being backed up is a P4
2.5Ghz w/ 512M
> ram. Of course, I don't have USB drives or filesystem encryption
> slowing me down.
I must have missed the beginning of this thread: is
the original poster using *both* USB drives (which I knew about) *and*
filesystem-level encryption? Wow.
USB is very CPU-intensive. The host CPU must
poll the bus constantly. I strongly suggest "upgrading"
to eSATA. It's still cheap, and at least you won't have the USB overhead.
Encryption is *very* CPU-intensive. For example,
from what I've seen (mostly with scp), 1MB/s is a *good* amount of data
transfer while using encryption! That's great for your average home-drive
or flash drive type storage, but BackupPC lives and dies by throughput...
Are you also using compression? Good luck. Compression
is almost as CPU intensive as encryption! (I forgot to note in my
stats previously that I do *not* use compression on BackupPC: I want my
backups as simple as possible.)
I think you might want to start a *LOT* simpler. Get
rid of a lot of the extra layers, even if temporarily. Start with
a simple server: 1 SATA drive, 1GB RAM, 1GHz processor and 1 Gb interface.
See how that performs--I think you will be pleasantly surprised.
Then start adding layers: move to a USB drive. See how
that changes things. Then add encryption. See where the bottleneck
And remember: there are *lots* of others seeing
results *very* different from yourself. Try to find out why.