[Bacula-devel] Better algorithms during interactive restore.

[Jesper K. <je...@kr...>]

Hi Bacula-devel.

I have done some restore-testing recently and there seems to be some
point which (at best) scales linear with the amount of files selected
for restore:

Select FileSet resource (1-2): 1
+--------+-------+-----------+-----------------+---------------------+------------+
| JobId  | Level | JobFiles  | JobBytes        | StartTime           |
VolumeName |
+--------+-------+-----------+-----------------+---------------------+------------+
| 31,988 | F     | 3,565,970 | 664,313,863,429 | 2009-11-14 15:19:46 |
004004L4   |
| 31,988 | F     | 3,565,970 | 664,313,863,429 | 2009-11-14 15:19:46 |
004005L4   |
| 31,988 | F     | 3,565,970 | 664,313,863,429 | 2009-11-14 15:19:46 |
004006L4   |
| 31,988 | F     | 3,565,970 | 664,313,863,429 | 2009-11-14 15:19:46 |
004007L4   |
| 32,535 | D     |   477,540 |  54,432,173,459 | 2009-11-30 08:31:14 |
004026L4   |
| 32,564 | I     |    43,215 |  25,105,348,665 | 2009-12-01 00:28:30 |
004023L4   |
| 32,594 | I     |   113,647 |  30,349,923,561 | 2009-12-01 23:46:04 |
004024L4   |
| 32,625 | I     |    44,610 |  26,696,212,439 | 2009-12-02 23:28:41 |
004024L4   |
| 32,652 | I     |    39,555 |  34,361,481,765 | 2009-12-03 23:46:18 |
004022L4   |
| 32,687 | I     |    62,945 |  36,001,296,657 | 2009-12-05 03:24:00 |
004020L4   |
| 32,716 | I     |     9,733 |  27,153,771,551 | 2009-12-05 23:12:57 |
004020L4   |
+--------+-------+-----------+-----------------+---------------------+------------+
You have selected the following JobIds:
31988,32535,32564,32594,32625,32652,32687,32716

Building directory tree for JobId 31988 ...
++++++++++++++++++++++++++++++++++++++++++++++++
Building directory tree for JobId 32535 ...  ++++
Building directory tree for JobId 32564 ...
Building directory tree for JobId 32594 ...
Building directory tree for JobId 32625 ...
Building directory tree for JobId 32652 ...  +
Building directory tree for JobId 32687 ...
Building directory tree for JobId 32716 ...
8 Jobs, 3,798,738 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /
$ cd /mnt/backup
cwd is: /mnt/backup/
$ mark cache
2,872,501 files marked.
$ done
Bootstrap records written to /var/lib/bacula/bacula-dir.restore.1.bsr

Just after done, the system waited for around 2.5 hours before getting
onto the actual restore. Seen from the system side it was pure cpu-load,
having one thread sitting at 100% CPU and absolutly no database-activity
and a decent (not growing) memory usage (~512MB).

Most of the time it actually never got to done but somehow the thread
taking care of the job just got killed (a watchdog timeout perhaps?)

I'm still on Bacula 2.4, so just let me know if there has been looked
into this in 3.0.

Jesper
-- 
Jesper