Menu

#276 Delta backup recovery needs too much memory

areca-3.11
open
nobody
None
1
2013-09-15
2013-09-11
dariomartin
No

Recovering of files that have been backed up with the "delta option" seems to load in memory every part of the file to proceed with reconstruction. This behaviour makes necessary to assign several GB of RAM to Java VM (through the Xmx variable in the Areca's ini file) in order to recover big files, and makes impossible to recover really large files (reconstruction fails and the resultant file is truncated, although no errors are reported in the log).
Since optimizing the backup of big files is the reason to use "delta backups", I would be really nice to optimize the reconstruction process.

Related

Feature Requests: #276

Discussion

  • aventin

    aventin - 2013-09-15

    hi

    Areca doesn't load the whole file into memory during recovery, so that type of problem shouldn't occur
    Could you send me the log file that was written while recovering so I can figure out what went wrong ?

    thanks

     
  • dariomartin

    dariomartin - 2013-09-17

    Thank you very much for your answer. I've attached two log files (one with
    Xmx=256m and one with Xmx=4096m) and the configuration file I used. These
    are the steps I followed:

    • I created a test target to reproduce the error, using "delta" copy. In
      the source there was a .pst file (from outlook) of 491MB.
    • I ran an incremental backup (as it was the first backup, Areca did a full
      backup), modified the source file, and run a second backup.
    • In the "archives" tab I see that the first backup uses 353MB and the
      second backup 1KB.
    • I adjusted the Xmx variable to 256m and restarted Areca.
    • When I tried to recover the file, I got in the destination folder a
      truncated file of 99.805KB and a temporary folder lcpy0 with two subfolders
      inside, containing the two original fragments of the file.
    • I adjusted Xmx variable to 4096m and restarted Areca.
    • This time the recover process worked right, and in the destination folder
      I got only the recovered file (temporary folder and files were deleted). I
      could see in the Windows process monitor how the memory used by Java grew
      up to 2GB more or less during reconstruction.

    No errors are reported in the logs, but you can see how with Xmx=256m only
    the first fragment is processed and then the recovering is reported to be
    completed:

    13-09-17 09:02 - INFO - Recovery filter : archive2.pst
    13-09-17 09:02 - INFO - 2 archives will be processed.
    13-09-17 09:02 - INFO - Files will be recovered in
    C:/Users/administrator/Desktop/Recover/rcv
    13-09-17 09:02 - INFO - 1 files will be recovered.
    13-09-17 09:02 - INFO - Data recovery ...
    13-09-17 09:02 - INFO - Recovering
    C:/Users/administrator/Desktop/target/1857487966/130917 (1 file) ...
    13-09-17 09:02 - INFO - Recovering
    C:/Users/administrator/Desktop/target/1857487966/130917_1 (1 file) ...
    13-09-17 09:02 - INFO - Processing archive 0
    (C:\Users\administrator\Desktop\Recover\rcv\lcpy0\130917) - 1 entry ...
    13-09-17 09:02 - INFO - Recovery of archive2.pst completed.

    With Xmx=4096, the two fragments are processed, and there are some further
    steps before the process is completed:

    13-09-17 09:00 - INFO - Recovery filter : archive2.pst
    13-09-17 09:00 - INFO - 2 archives will be processed.
    13-09-17 09:00 - INFO - Files will be recovered in
    C:/Users/administrator/Desktop/Recover2/rcv
    13-09-17 09:00 - INFO - 1 files will be recovered.
    13-09-17 09:00 - INFO - Data recovery ...
    13-09-17 09:00 - INFO - Recovering
    C:/Users/administrator/Desktop/target/1857487966/130917 (1 file) ...
    13-09-17 09:00 - INFO - Recovering
    C:/Users/administrator/Desktop/target/1857487966/130917_1 (1 file) ...
    13-09-17 09:00 - INFO - Processing archive 0
    (C:\Users\administrator\Desktop\Recover2\rcv\lcpy0\130917) - 1 entry ...
    13-09-17 09:00 - INFO - Processing archive 1
    (C:\Users\administrator\Desktop\Recover2\rcv\lcpy0\130917_1) - 1 entry ...
    13-09-17 09:00 - INFO - Creating missing directories and symbolic links ...
    13-09-17 09:00 - INFO - Applying metadata ...
    13-09-17 09:00 - INFO - Metadata applied.
    13-09-17 09:00 - INFO - Moving
    C:/Users/administrator/Desktop/Recover2/rcv/archive2.pst to
    C:/Users/administrator/Desktop/Recover2/archive2.pst
    13-09-17 09:00 - INFO - Recovery of archive2.pst completed.

    Thank you very much. You have done a great job with Areca. It's a powerful
    and yet simple to use application.

    On Sun, Sep 15, 2013 at 4:13 PM, aventin aventin@users.sf.net wrote:

    hi

    Areca doesn't load the whole file into memory during recovery, so that
    type of problem shouldn't occur
    Could you send me the log file that was written while recovering so I can
    figure out what went wrong ?

    thanks

    Status: open
    Created: Wed Sep 11, 2013 11:10 AM UTC by dariomartin
    Last Updated: Wed Sep 11, 2013 11:10 AM UTC
    Owner: nobody

    Recovering of files that have been backed up with the "delta option" seems
    to load in memory every part of the file to proceed with reconstruction.
    This behaviour makes necessary to assign several GB of RAM to Java VM
    (through the Xmx variable in the Areca's ini file) in order to recover big
    files, and makes impossible to recover really large files (reconstruction
    fails and the resultant file is truncated, although no errors are reported
    in the log).
    Since optimizing the backup of big files is the reason to use "delta
    backups", I would be really nice to optimize the reconstruction process.


    Sent from sourceforge.net because you indicated interest in
    https://sourceforge.net/p/areca/feature-requests/276/

    To unsubscribe from further messages, please visit
    https://sourceforge.net/auth/subscriptions/

     

    Related

    Feature Requests: #276


Log in to post a comment.

MongoDB Logo MongoDB