[BackupPC-users] Upcoming release of rsync-bpc-3.4.1.0rc1.

[G.W. H. <ba...@ju...>]

Hi there,

As mentioned in another thread, I think the release of the next
version of rsync-bpc - version 3.4.1.0rc1 - is approaching.

As its version number implies, this version is based on the latest
upstream rsync, version 3.4.1.  That version contains a number of
improvements plus a few fixes for vulnerabilites.  Vulnerabilities
have been found in supporting packages such as zlib and popt.  For
whatever reasons, many packages (not just rsync and BackupPC) bundled
the code for these libraries in their own source distributions.  The
problem with doing that, of course, is that the bundled versions tend
to get out of date which is what happened for rsync, BackupPC-XS and
rsync-bpc although the resulting issues were fortunately not serious.
It could easily have been otherwise.

As for BackupPC-XS, the new version of rsync-bpc does away with the
bundled zlib and popt code, instead relying on the system to provide
these libraries.  If the distribution's security teams and your update
practice are up to snuff, then these paths to vulnerability should be
closed off for good.

This has been quite a journey and I'm asking for volunteers, to begin
with, to sanity-check what I've done.

The changes to the rsync-bpc code have been very extensive (think in
terms of a context diff with a line count well into five digits) so I
really do want to get people to put this thing under the microscope.
It will help enormously if you're a proficient 'C' coder but it isn't
an absolute requirement; just building and testing by anyone will be
more than welcome.  Running here, it's just done its first incremental
backup after four fulls for just one share on just one machine.  I'll
be adding to that population in the coming days.  So far I only built
on one architecture (armhf).  Anything could happen on a 64-bit box,
the data structures in rsync are truly scary.

There are still a couple of outstanding issues which I want to look
into, and I'll be glad of any comments from users:

1. While testing on a box with usually ~3G spare memory, I noticed a
tendency for one of the rsync-bpc processes to use it all up and then
crash.  After I'd removed most of my debugging (gigabyte+ log files)
this seemed to go away, but I'm reminded of Github issue #547 and I
think it needs a bit more investigation.  Also in the coming days I
plan to do that.  I know where to look and what to log, I just don't
yet know if having a bunch of 8 MByte file buffers in RAM is an issue.

2. The latest version of rsync offers extra options for checksumming,
and that seemed to throw off the new rsync-bpc's use of MD5 for file
comparisons.  My very klundgy fix for that has been to add the rsync
'--checksum-choice=md5' option to $Conf{RsyncArgs} but I'd much prefer
to look at modifying the code to force the use of the right checksums
without requiring any changes to the configuration.  At this stage I
don't know what this might mean for people using e.g. 'openrsync' on
the Mac - I don't know if the Mac's rsync recognizes the option - and
I don't know if it will be an issue for people using very old versions
of 'genuine' rsync on other clients.  If there are Mac users Out There
reading, *please* help me with the testing if you can.  If your rsync
version _is_ very old, I promise not to tell.

At this stage, rather than putting something on Github which may quite
possibly crash and burn, I'd prefer to email the source to volunteers.

Any takers?

-- 

73,
Ged.