Re: [BackupPC-users] Really slow with rsync, even on incrementals

[John G. <jgo...@co...>]

Timothy J Massey <tmassey <at> obscorp.com> writes:

> Encryption is *very* CPU-intensive.  For example,
> from what I've seen (mostly with scp), 1MB/s is a *good* amount of data
> transfer while using encryption!  That's great for your average home-drive

I ran some bonnie++ benchmarks on the filesystem in question (backed by USB disk
and compression).  It showed 25MB/s block write, 4.5MB/s block rewrite, and
6MB/s block read.  So by even a pessimistic benchmark that is 4.5 times as fast
as 1MB/s ;-)

> or flash drive type storage, but BackupPC lives and dies by throughput...
> Are you also using compression?  Good luck.  Compression
> is almost as CPU intensive as encryption!  (I forgot to note in my
> stats previously that I do *not* use compression on BackupPC: I want my
> backups as simple as possible.)

Yes, in backuppc (not any kind of filesystem-level compression), but when most
of the dataset doesn't change in a night -- is it really relevant?

> I think you might want to start a *LOT* simpler.  Get
> rid of a lot of the extra layers, even if temporarily.  Start with
> a simple server: 1 SATA drive, 1GB RAM, 1GHz processor and 1 Gb interface.
>  See how that performs--I think you will be pleasantly surprised.

Well, I did run this on a system with better specs than that (SATA disks in a
RAID, lots of RAM, faster processor).  It worked great with tar but bogged down
terribly on 25GB files with rsync (see the other thread about that).  I believe
the problems I'm seeing now, at least for the incrementals, are unrelated.

-- John