I just noticed one bug that ended up in the release after some last-minute cleanup I was doing. BackupPC_nightly fails to run because of a missing library specification.  The fix is simply adding the missing line back - a patch is attached.

Either apply this with the patch command after you unpack the tarball and before you run configure.pl, or manually add the missing line "use BackupPC::DirOps;" in bin/BackupPC_nightly, either before or after you install the code (if you do it after, you have to edit bin/BackupPC_nightly in the installed location, not where the tarball was unpacked).

If you are already running 4.0.0, the bug is essentially harmless - the next night after you make the fix, BackupPC_nightly will catch up with all its tasks.  If you don't make the fix, the pool stats won't get updated, emails won't be sent, and nothing will be deleted from the pool.


--- bin/BackupPC_nightly        2013-06-23 17:18:00.000000000 -0700
+++ bin/BackupPC_nightly        2013-06-24 22:18:57.000000000 -0700
@@ -81,6 +81,7 @@
 use lib "__INSTALLDIR__/lib";
 use BackupPC::Lib qw( :BPC_DT_ALL );
 use BackupPC::XS;
+use BackupPC::DirOps;
 use Getopt::Std;

 use File::Path;

On Sun, Jun 23, 2013 at 6:10 PM, Craig Barratt <cbarratt@users.sourceforge.net> wrote:
BackupPC community,

I'm pleased to announce that BackupPC 4.0.0alpha0 has been released on SourceForge at:

4.0.0 is a significant upgrade and rewrite.  It should be backward compatible with existing installations.

The release comprises three packages:

 - BackupPC-4.0.0alpha0.tar.gz: the usual BackupPC release tar ball.
 - BackupPC-XS-0.01.tar.gz: a perl XS module with C code that replaces several BackupPC perl libraries for improved performance.
 - rsync-bpc- a modified rsync that runs on the server that has a shim layer that interfaces directly to the BackupPC file system.

I'll also push BackupPC-XS-0.01.tar.gz to CPAN, and at some point I'll push the source to GitHub  (maybe next weekend; I'm starting a new job tomorrow).  Of course, the tar balls above contain all the source too.

As you can guess from the release name, I do not recommend using this release in any production environment.  Having some people try it out in a sandbox in their environment would be very helpful.  One important area to test is upgrading an existing V3 installation, so it would be great to test not just a virgin V4 installation, but also to also first install V3, do some backups, then upgrade to V4.

The documentation is only partially updated.  I'm happy to answer questions - for example, there are some importact changes to the configuration settings. If you reply to this thread please only copy the backuppc-devel list.

I've pasted a short summary of features and changes.  More is in the documentation.


 - No use of hardlinks (except temporarily to do atomic renames).  Reference counting
   is handled at the application level.

 - Backups are stored as "reverse deltas" - the most recent backup is always filled
   and older backups are reconstitued by merging all the deltas starting with the
   nearest future filled backup and working backwards.

   This is the opposite of V3 where incrementals are stored as "forward deltas"
   to a prior backup (typically the last full backup or prior lower-level
   incremental backup, or the last full in the case of rsync).

 - Since the most recent backup is filled, viewing/restoring that backup (which is
   the most common backup used) doesn't require mergning any deltas from other backups.

 - The concepts of incr/full backups and unfilled/filled storage are decoupled.

 - Uses full-file MD5 digests, which are stored in the directory attrib files.
   Each backup directory only contains an attrib file.

 - The Pool layout still support chains to handle md5 collisions.  While collisions
   can be constructed and are now well-known, they are highly unlikely in the wild.
   Pool files are never renamed or moved, unlike V3.

 - Any backup can be deleted (deltas are merged into next older backup if it is
   not filled).

 - The reverse deltas allow "infinte incrementals" - no need for a full backup
   if you are willing to trade speed for the risk that a file change will
   not be detected if the mtime or size don't change.

 - An rsync "full" backup now uses --checksum (instead of --ignore-times),
   which is much more efficient on the server side - the server just needs to
   check the full-file checksum computed by the client, together with the mtime,
   nlinks, size attributes, to see if the file has changed.  If you want a more
   conservative approach, you can change it back to --ignore-times, which
   requires the server to send block checksums to the client.

 - The use of rsync --checksum allows BackupPC to guess a potential match
   anywhere in the pool, even on a first-time backup.  In that case, the usual
   rsync block checksums are still exchanged to make sure the complete file
   is identical.

 - Uses rsync-3.0.9 on the server side (in place of File::RsyncP), with
   a C code layer to interface to the BackupPC storage.  So the whole data
   path for rsync is now in compiled C code, which is generally faster than perl
   (other than compression, which is the same code in either case, and I/O).

 - Due to the use of rsync-3.X, acls and xattrs are supported, and other
   useful options (but not all) are supported.  Rsync protocol 30 supports
   the efficient incremental file list, which significantly improves memory
   usage and startup time.  It also supports MD5 full-file checksums, which
   match BackupPC's new digest.  That allows a full-file digest to be checked
   as easily as an mtime on the server side.

 - Significant portions of the BackupPC code are now compiled C code in a
   new module called BackupPC::XS that is dynamically linked to perl.

 - FTP Xfer method isn't supported yet.