#11 'internal error: job made no progress' on 25G file

open-fixed
7
2006-03-10
2005-01-27
Anonymous
No

I'm using librsync 9.7, ./configure with shared libraries
enabled (only way to get the librsync.so file on freebsd).
It is a back-end to rdiff-backup which is a python based
backup program in the ports collection.

rdiff-backup is reaching out over a network via ssh to
backup a 250G hard drive on a server to a local backup
drive nightly.

It completed the initial backup run, which was oddly CPU
bound even though ssh was not used with compression.
Took much longer than the bandwidth would have
suggested. That used the 9.6 librsync.

When I ran the program again, which would have only
transferred the very small deltas since the previous day,
it just never completed. I aborted it, tried a few more
times, then upgraded to librsync 9.7.

the file librsync choked on is 25,990,693,888 bytes long
(a big Windows incremental backup of some sort).

here's the diagnostic output

Previous backup seems to have failed, regressing
destination now.
librsync: ERROR: (rs_job_iter) internal error: job made no
progress [orig_in=130842, orig_out=65536,
final_in=130842, final_out=65536]
UpdateError library/Archive/DevOffice.bkf librsync error
107 while in patch cycle

Help appreciated!

(anti spammer) email hcoin at n4comm

and then .com

Harry Coin
Bettendorf, Iowa

Discussion

  • Harry Coin
    Harry Coin
    2005-01-27

    Logged In: YES
    user_id=1206224

    I saw a couple of complile comments:

    it didn't like the lack of #include <string.h> in
    isprefix.driver.c

    and in netint.c in function rs_suck_netint it didn't like
    133 breaking anti-aliasing rules. also in netint it didn't
    like 179 'integer constant too large for long type'.

    didn't like 117 in readsums.c about antialiasing also. I
    think that's all I saw in the large comple log. All the
    checks and tests passed.

    Also I think I'm part of this whole sourceforge website now.

    Harry

     
  • Logged In: YES
    user_id=1370792

    Actually, as a follow up, this case is not unique. The
    problem is observed for EVERY file over 4Gb on several
    (possibly all?) Linux versions and FreeBSD as far as I can
    confirm. I have tested it extensively and it only happens to
    files over 4Gb in size on at least ext2fs and xfs for as far
    as I have tested... 32/64 bit related? This is a major bug.
    Hopefully someonce can look at it soon!

    Gerard van Dijnsen
    Eindhoven, The Netherlands

     
  • Don Malcolm
    Don Malcolm
    2006-02-27

    Logged In: YES
    user_id=1450893

    Patch Tracker ID: 1439412 is a patch that remedies this
    bug.

    Don Malcolm
    Sydney, Australia

     
  • Donovan Baarda
    Donovan Baarda
    2006-03-10

    Logged In: YES
    user_id=10273

    This bug has been fixed in the developer's CVS. There may be some
    delay before the fix appears in anonymous CVS.

    The fix will be included in the next release of librsync after
    the date of this comment (the date this bug was set "Fixed").
    This bug will be closed when the fix is included in a release.

     
  • Donovan Baarda
    Donovan Baarda
    2006-03-10

    • labels: 506017 --> Implementation
    • priority: 5 --> 7
    • assigned_to: nobody --> abo
    • status: open --> open-fixed
     
  • Logged In: YES
    user_id=335973

    This bug is not really fixed!
    I did a CVS checkout and the NEWS file says "Fixed bug
    #1110812", so I do have an anonymous checkout where it is
    supposed to be fixed.

    But the bug still occurs.
    I'm talking about backing up an smbmounted windows share
    with a large file in it.

    (Do I need to open a new bug?)

    Best regards and thanks for librsync
    Matthias

     
  • Donovan Baarda
    Donovan Baarda
    2006-04-19

    Logged In: YES
    user_id=10273

    I'm not sure that the smbmounted windows share large file
    problem is the same as this bug. Please check that the error
    you are getting has the same diagnostic output. Also check
    that you are actually running the CVS code.

    It may be that you are hitting a smb filesize limitation.
    The diagnostic information migt be miss-leading. I would
    probably submit a new bug.

     
  • Logged In: YES
    user_id=335973

    The diagnostic output is the same:

    python: ERROR: (rs_job_iter) internal error: job made no
    progress [orig_in=113293, orig_out=65536, final_in=113293,
    final_out=65536]
    UpdateError xxxxxxxxx/xxxxxxxxx librsync error 107 while in
    patch cycle

    I'm running the CVS code.
    I switched to CIFS mounting, so the SMB filesize limitation
    is not relevant anymore.

    Regards
    Matthias

     
  • Logged In: NO

    Just to comment on this, it doesn't happen for *all* files
    >4GB. I get this error on two files, each ~5gb, but we also
    have files larger than that which complete successfully.

     
  • ListigeWitwe
    ListigeWitwe
    2008-07-12

    Logged In: YES
    user_id=2145255
    Originator: NO

    The problem is still not solved:
    Using Librsync 0.9.7 delivered with Ubuntu 7.10 via rdiff-backup 1.1.14 to backup an encrypted vmware virtual machine of 7.7 GB reproduces the problem:

    rdiff-backup /home/ /media/disk/bak/rdiff-backups/home
    python: ERROR: (rs_job_iter) internal error: job made no progress [orig_in=1, orig_out=65536, final_in=1, final_out=65536]
    UpdateError inet/VirtualMachines/wiki/wiki.vmdk librsync error 107 while in patch cycle
    root@lx5:/home/inet# rdiff-backup /home/ /media/disk/bak/rdiff-backups/home
    python: ERROR: (rs_job_iter) internal error: job made no progress [orig_in=1, orig_out=65536, final_in=1, final_out=65536]
    UpdateError inet/VirtualMachines/wiki/wiki.vmdk librsync error 107 while in patch cycle