Re: [BackupPC-users] BackupPC_nightly taking very long

[Craig B. <cba...@us...>]

Guus Houtzager writes:

> Some time ago I had performance problems with backuppc, but those were
> solved with a switch from ext3 to reiser 3.
> Now, about a month further along, BackupPC_nightly is taking very long,
> about 8.5 hours. Because that blocks any backup an restore processes,
> this doesn't make me very happy.
> I'm afraid my I/O subsystem just isn't fast enough, but then again, my
> setup isn't that ridiculously large I think. Is there anything I can do
> to speed up this process?

It sounds like BackupPC generally runs faster on reiser3, except
for BackupPC_nightly, which seems to run faster on ext3...

Since a number of people are having problems with BackupPC_nightly
running slow, I propose the following:

  - You could choose to run BackupPC_nightly during the day, if
    your backups generally run at night.  The first entry of
    $Conf{WakeupSchedule} is when BackupPC_nightly runs, so
    simply re-order the entries so that the first one is when you
    want to run BackupPC_nightly.

  - I will look into adding a feature to BackupPC-2.1.0beta1 (maybe
    target for this weekend), which allows BackupPC_nightly to just
    check a part of the pool each night.  The only significant issue
    is that the pool stats (number of files etc) will be harder to
    update correctly.  The only cost here is a slight increase in
    storage since unused pool files are not deleted sooner.

  - Or instead (or in addition) I could set things up so BackupPC_nightly
    only does the pool check say every other night.  This won't reduce
    the run time, but the average impact will be half.

  - In a later version (post 2.1.0) I will look into having
    BackupPC_nightly fork itself a configurable number of times
    so that it runs in parallel on different parts of the pool.
    Again, the tricky issue here is collecting the stats output
    from each child.

Craig