#15 with cygwin 'internal error: job made no progress'

[James DuPlain]

This looks like bug 1110812 but in that report there was no mention of it happening on cygwin. So...creating new report to reflect the same/similar issue on cygwin.

Details:
winXP
cygwin and cygwin supplied libs (latest 2007-12-13)
python 2.5.1 (from cygwin)
rdiff-backup-1.1.14
Tried librsync with both (rebuilding/installing rsync-backup each time)
librsync-0.9.7
librsync from anonymous CVS on 2007-12-13
Steps to build/install...
./autogen.sh
./configure
make all check
make install
The last command reported..
Libraries have been installed in:
/usr/local/lib
So, when rsync-backup was installed, it was done so with
python setup.py install --prefix=/usr/local

To reproduce...
1. Create a really big tar file. Mine was....
5304637440 Dec 13 21:35 reallyBigFile.tar

2. Run the backup...
=========== INITIAL BACKUP CMD/OUTPUT =============
$ rdiff-backup -v5 --print-statistics sourceFiles/ /e/destFiles/
Using rdiff-backup version 1.1.14
Unable to import module xattr.
Extended attributes not supported on filesystem at sourceFiles
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at sourceFiles
escape_dos_devices required by filesystem at sourceFiles
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Case sensitivity Off
Escape DOS devices On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import module xattr.
Extended attributes not supported on filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices required by filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing Off
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames Off
Access control lists Off
Extended attributes Off
Case sensitivity Off
Escape DOS devices On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 1
Starting mirror sourceFiles to /e/destFiles
Processing changed file .
Processing changed file reallyBigFile.tar
--------------[ Session statistics ]--------------
StartTime 1197571556.00 (Thu Dec 13 10:45:56 2007)
EndTime 1197572232.64 (Thu Dec 13 10:57:12 2007)
ElapsedTime 676.64 (11 minutes 16.64 seconds)
SourceFiles 2
SourceFileSize 5304637440 (4.94 GB)
MirrorFiles 1
MirrorFileSize 0 (0 bytes)
NewFiles 1
NewFileSize 5304637440 (4.94 GB)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 1
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 0
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 5304637440 (4.94 GB)
Errors 0
--------------------------------------------------

3. Update the tar file (added a small text file to the tar).

4. Re-run the backup....
=========== 2ND BACKUP CMD/OUTPUT =============
$ rdiff-backup -v5 --print-statistics sourceFiles/ /e/destFiles/
Using rdiff-backup version 1.1.14
Unable to import module xattr.
Extended attributes not supported on filesystem at sourceFiles
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at sourceFiles
escape_dos_devices required by filesystem at sourceFiles
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Case sensitivity Off
Escape DOS devices On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import module xattr.
Extended attributes not supported on filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices required by filesystem at /e/destFiles/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing Off
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames Off
Access control lists Off
Extended attributes Off
Case sensitivity Off
Escape DOS devices On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 1
Starting increment operation sourceFiles to /e/destFiles
Processing changed file .
Incrementing mirror file /e/destFiles
Processing changed file reallyBigFile.tar
librsync: ERROR: (rs_job_iter) internal error: job made no progress [orig_in=64400, orig_out=65536, final_in=64400, final_out=65536]
Exception 'librsync error 107 while in patch cycle' raised of class '<class 'rdiff_backup.librsync.librsyncError'>':
File "/usr/lib/python2.5/site-packages/rdiff_backup/robust.py", line 32, in check_common_error
try: return function(*args)
File "/usr/lib/python2.5/site-packages/rdiff_backup/Rdiff.py", line 98, in patch_local
if outrp: return outrp.write_from_fileobj(patchfile)
File "/usr/lib/python2.5/site-packages/rdiff_backup/rpath.py", line 1068, in write_from_fileobj
copyfileobj(fp, outfp)
File "/usr/lib/python2.5/site-packages/rdiff_backup/rpath.py", line 58, in copyfileobj
inbuf = inputfp.read(blocksize)
File "/usr/lib/python2.5/site-packages/rdiff_backup/librsync.py", line 77, in read
self._add_to_outbuf_once()
File "/usr/lib/python2.5/site-packages/rdiff_backup/librsync.py", line 88, in _add_to_outbuf_once
except _librsync.librsyncError, e: raise librsyncError(str(e))

UpdateError reallyBigFile.tar librsync error 107 while in patch cycle
--------------[ Session statistics ]--------------
StartTime 1197572543.00 (Thu Dec 13 11:02:23 2007)
EndTime 1197574457.94 (Thu Dec 13 11:34:17 2007)
ElapsedTime 1914.94 (31 minutes 54.94 seconds)
SourceFiles 1
SourceFileSize 0 (0 bytes)
MirrorFiles 1
MirrorFileSize 0 (0 bytes)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 1
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 1
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 0 (0 bytes)
Errors 1
--------------------------------------------------

[Guru Evi]

This is a bug with files >4G. Use the following patch:
diff -urN librsync-0.9.7/mdfour.h librsync-0.9.7dev/mdfour.h
--- librsync-0.9.7/mdfour.h 2004-02-07 18:17:57.000000000 -0500
+++ librsync-0.9.7dev/mdfour.h 2006-03-06 03:21:46.000000000 -0500
@@ -24,7 +24,7 @@
#include "types.h"

struct rs_mdfour {
- int A, B, C, D;
+ unsigned int A, B, C, D;
#if HAVE_UINT64
uint64_t totalN;
#else
diff -urN librsync-0.9.7/patch.c librsync-0.9.7dev/patch.c
--- librsync-0.9.7/patch.c 2004-09-17 17:35:50.000000000 -0400
+++ librsync-0.9.7dev/patch.c 2006-03-06 03:21:06.000000000 -0500
@@ -214,12 +214,12 @@
void *buf, *ptr;
rs_buffers_t *buffs = job->stream;

- len = job->basis_len;
-
/* copy only as much as will fit in the output buffer, so that we
* don't have to block or store the input. */
- if (len > buffs->avail_out)
+ if (job->basis_len > buffs->avail_out)
len = buffs->avail_out;
+ else
+ len = job->basis_len;

if (!len)
return RS_BLOCKED;