#11 'internal error: job made no progress' on 25G file
I'm using librsync 9.7, ./configure with shared libraries
enabled (only way to get the librsync.so file on freebsd).
It is a back-end to rdiff-backup which is a python based
backup program in the ports collection.
rdiff-backup is reaching out over a network via ssh to
backup a 250G hard drive on a server to a local backup
drive nightly.
It completed the initial backup run, which was oddly CPU
bound even though ssh was not used with compression.
Took much longer than the bandwidth would have
suggested. That used the 9.6 librsync.
When I ran the program again, which would have only
transferred the very small deltas since the previous day,
it just never completed. I aborted it, tried a few more
times, then upgraded to librsync 9.7.
the file librsync choked on is 25,990,693,888 bytes long
(a big Windows incremental backup of some sort).
here's the diagnostic output
Previous backup seems to have failed, regressing
destination now.
librsync: ERROR: (rs_job_iter) internal error: job made no
progress [orig_in=130842, orig_out=65536,
final_in=130842, final_out=65536]
UpdateError library/Archive/DevOffice.bkf librsync error
107 while in patch cycle
Help appreciated!
(anti spammer) email hcoin at n4comm
and then .com
Harry Coin
Bettendorf, Iowa
Logged In: YES
user_id=1206224
I saw a couple of complile comments:
it didn't like the lack of #include <string.h> in
isprefix.driver.c
and in netint.c in function rs_suck_netint it didn't like
133 breaking anti-aliasing rules. also in netint it didn't
like 179 'integer constant too large for long type'.
didn't like 117 in readsums.c about antialiasing also. I
think that's all I saw in the large comple log. All the
checks and tests passed.
Also I think I'm part of this whole sourceforge website now.
Harry
Logged In: YES
user_id=1370792
Actually, as a follow up, this case is not unique. The
problem is observed for EVERY file over 4Gb on several
(possibly all?) Linux versions and FreeBSD as far as I can
confirm. I have tested it extensively and it only happens to
files over 4Gb in size on at least ext2fs and xfs for as far
as I have tested... 32/64 bit related? This is a major bug.
Hopefully someonce can look at it soon!
Gerard van Dijnsen
Eindhoven, The Netherlands
Logged In: YES
user_id=1450893
Patch Tracker ID: 1439412 is a patch that remedies this
bug.
Don Malcolm
Sydney, Australia
Logged In: YES
user_id=10273
This bug has been fixed in the developer's CVS. There may be some
delay before the fix appears in anonymous CVS.
The fix will be included in the next release of librsync after
the date of this comment (the date this bug was set "Fixed").
This bug will be closed when the fix is included in a release.
Logged In: YES
user_id=335973
This bug is not really fixed!
I did a CVS checkout and the NEWS file says "Fixed bug
#1110812", so I do have an anonymous checkout where it is
supposed to be fixed.
But the bug still occurs.
I'm talking about backing up an smbmounted windows share
with a large file in it.
(Do I need to open a new bug?)
Best regards and thanks for librsync
Matthias
Logged In: YES
user_id=10273
I'm not sure that the smbmounted windows share large file
problem is the same as this bug. Please check that the error
you are getting has the same diagnostic output. Also check
that you are actually running the CVS code.
It may be that you are hitting a smb filesize limitation.
The diagnostic information migt be miss-leading. I would
probably submit a new bug.
Logged In: YES
user_id=335973
The diagnostic output is the same:
python: ERROR: (rs_job_iter) internal error: job made no
progress [orig_in=113293, orig_out=65536, final_in=113293,
final_out=65536]
UpdateError xxxxxxxxx/xxxxxxxxx librsync error 107 while in
patch cycle
I'm running the CVS code.
I switched to CIFS mounting, so the SMB filesize limitation
is not relevant anymore.
Regards
Matthias
Logged In: NO
Just to comment on this, it doesn't happen for *all* files
>4GB. I get this error on two files, each ~5gb, but we also
have files larger than that which complete successfully.
Logged In: YES
user_id=2145255
Originator: NO
The problem is still not solved:
Using Librsync 0.9.7 delivered with Ubuntu 7.10 via rdiff-backup 1.1.14 to backup an encrypted vmware virtual machine of 7.7 GB reproduces the problem:
rdiff-backup /home/ /media/disk/bak/rdiff-backups/home
python: ERROR: (rs_job_iter) internal error: job made no progress [orig_in=1, orig_out=65536, final_in=1, final_out=65536]
UpdateError inet/VirtualMachines/wiki/wiki.vmdk librsync error 107 while in patch cycle
root@lx5:/home/inet# rdiff-backup /home/ /media/disk/bak/rdiff-backups/home
python: ERROR: (rs_job_iter) internal error: job made no progress [orig_in=1, orig_out=65536, final_in=1, final_out=65536]
UpdateError inet/VirtualMachines/wiki/wiki.vmdk librsync error 107 while in patch cycle
For anyone still looking. There is a patch for this:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=355178