Re: [Burp-users] Stuck on Phase4 (shuffling)

[Graham K. <gr...@gr...>]

On Mon, Dec 22, 2014 at 02:03:58AM -0700, Moter Pent wrote:
> I've noticed backups for a particular host failing for several days now.
> Logs show either:
> 
> 2014-12-13 10:09:01: burp[27088] Begin phase4 (shuffle files)
> 2014-12-13 10:09:01: burp[27088] previous attempt started not hardlinked
> 2014-12-13 10:09:01: burp[27088]  not doing hardlinked archive
> 2014-12-13 10:09:01: burp[27088]  will generate reverse deltas
> 2014-12-13 10:09:01: burp[27088] Doing the atomic data jiggle...
> 2014-12-13 10:09:01: burp[27088] skipping already present file:
> /pool2/backups/server/finishing/data/t/data/.blksnap/AAData.md
> 2014-12-13 10:09:01: burp[27088] to save log space, skips of other already
> present files will not be logged
> 2014-12-13 10:50:49: burp[27088] zlib_inflate returned: -1
> 2014-12-13 10:50:49: burp[27088] error when inflating old file:
> /pool2/backups/server/finishing/currentdup/data/t/home/someuser/archive/somefile.nsf
> 2014-12-13 10:50:51: burp[27088] could not finish up backup.
> 2014-12-13 10:50:51: burp[27088] Problem with prior backup. Please check
> the client log on the server.
> 
> 
> ...or...
> 
> 2014-12-22 00:49:01: burp[567] Begin phase4 (shuffle files)
> 2014-12-22 00:49:01: burp[567] previous attempt started not hardlinked
> 2014-12-22 00:49:01: burp[567]  not doing hardlinked archive
> 2014-12-22 00:49:01: burp[567]  will generate reverse deltas
> 2014-12-22 00:49:01: burp[567] Doing the atomic data jiggle...
> 2014-12-22 00:49:01: burp[567] skipping already present file:
> /pool2/backups/server/finishing/data/t/data/.blksnap/AAData.md
> 2014-12-22 00:49:01: burp[567] to save log space, skips of other already
> present files will not be logged
> 2014-12-22 01:40:28: burp[567] could not lstat
> /pool2/backups/server/finishing/currentdup/data/t/home/someuser/archive/somefile.nsf
> 2014-12-22 01:40:28: burp[567] error when inflating old file:
> /pool2/backups/server/finishing/currentdup/data/t/home/someuser/archive/somefile.nsf
> 2014-12-22 01:40:30: burp[567] could not finish up backup.
> 2014-12-22 01:40:30: burp[567] Problem with prior backup. Please check the
> client log on the server.
> 
> 
> I've seen a few topics on this previously.  One mentioned this was resolved
> in an update (I'm running the latest 1.4.28 on both server and client).
> Another mentioned trying setting the "directory_tree" option to 0 on the
> server.  I have not tried this.  I have told burp to no longer compress the
> file type in question but to no avail.
> 
> I'm dealing with a fairly large amount of data (4TB+) so each attempt is
> fairly costly in terms of run time.  Any suggestions on untangling this?
> I'd prefer to not to reset given how long it takes to take the initial base.

Hello,

Is it always the same file that it complains about?

When it says 'error when inflating old file', it means that it has a forward
librsync patch to apply to a file that was previously backed up (in order to
create the latest version of the file).

It thinks that the previously backed up file needs to be uncompressed in order
to apply the patch, and 'zlib_inflate returned: -1' means that it had an
error when doing that.
'could not lstat' means that now it could not even find the file at all.

I would see if the file exists in
/pool2/backups/server/finishing/currentdup/data/t/home/someuser/archive/somefile.nsf
and in
/pool2/backups/server/current/data/t/home/someuser/archive/somefile.nsf

They should be hardlinked together. If they are not, you can hard link the
paths by hand. And then check whether the file has been compressed with zlib.
You can use the 'file' command to do this. For example, 'file somefile.nsf'
would print something like:
'somefile.nsf: gzip compressed data, from Unix, max compression'

You can then try resuming the backup by getting the client to connect again.


More strategies I can think of:

*) Edit the manifest fragments in the 'finishing' directory to remove mentions
   of somefile.nsf. Then burp will not try to do anything with it when it
   resumes. The files to edit are called 'phase2' and 'unchanged'. If you
   want to try this, let me know and I will help you with the lines that you
   need to delete.

*) Point the 'current' symlink at the most recent good 'hardlinked' backup,
   delete all the more recent ones, then carry on from there.
   There was a recent (within the last year) mailing list post/thread that
   explained about this, but I can't seem to find it right now. If you
   need/want to try this, let me know and I will try harder to find it.