Menu
▾
▴
[Bacula-users] incremental larger than full?
|
From: <pbd...@pi...> - 2010-12-05 13:28:41
|
Hello.
I'm rather new to bacula, coming from years of doing dump/tar backups.
As the environment grows more complex, I need something more flexible
and automated, thus trying to switch to bacula.
Basically I managed to work out the setup and do backups. But with one
thing I just can't figure out what I'm doing wrong and would be glad for
some pointer.
The incremental backups are _way_ too large (output is attached further
down). I do understand the concept of doing a full instead of an
incremental if a previous full backup is not available or too old. The
increpmental a couple of hours after the full is even way too large.
Example 1:
turtle-010 is doing a full backup with 37'465 files
and 370 MB (manually run with the run command) and the next incremental a
couple of hours later counts 1'399 files with 866 MB. There's nothing
going on on that machine that would install additional files or
anything. The incremental is twice the size of the full!
Example 2:
turtle-121 is doing a full backup with 289'971 files and 22 GB (again,
run manually with the run command). Except for a few logfiles, there is
not much changing on this machine. A couple of hours later, the
automatic incremental kicks in and basically backs up the whole machine
again, 269'108 files counting 22 GB.
This, of course, eats up my backup storage capacity far too quickly. The
only non-full backup, which looks reasonable to me below, is the two
differentials.
The output below is from a fresh startover, I threw away all backups,
cleaned out all files written by the storage daemon, removed everything
under /var/spool/bacula/ and /var/log/bacula/, deleted the whole
database and recreated it empty from scratch.
The setup:
location pbdhome location pbdbuchs location pbdhetzner
------------------- ------------------------ --------------------
dir-pbdhome - -
sd-pbdhome sd-pbdbuchs sd-pbdhetzner
local disk via nfs local disk direct attach local disk via sshfs
fd-h1,fd-h2...fd-hn fd-b1,fd-b2...fd-bn fd-e1,fd-e2...fd-en
- One director at home is managing everything.
- Each of the three locations has a storage daemon.
- All the file daemons at each location, controlled by the central
director, are sending their backups to the storage daemon at their own
location.
- Each of the clients has _one_ job with a stacked schedule, doing one
full once a month, a differential every week and an incremental every
day.
- The jobs do have Accurate = yes
*list jobs
Automatically selected Catalog: Catalog-pbdhome
Using Catalog "Catalog-pbdhome"
+-------+---------------------------+---------------------+------+-------+----------+----------------+-----------+
| JobId | Name | StartTime | Type | Level | JobFiles | JobBytes | JobStatus |
+-------+---------------------------+---------------------+------+-------+----------+----------------+-----------+
| 1 | BackupClient-turtle | 2010-12-04 19:50:06 | B | F | 79,094 | 449,663,696 | T |
| 2 | BackupClient-grasshopper | 2010-12-04 19:50:16 | B | F | 88,007 | 4,002,316,384 | T |
| 3 | BackupClient-weasel | 2010-12-04 19:50:42 | B | F | 170,503 | 1,983,609,840 | T |
| 4 | BackupClient-turtle-010 | 2010-12-04 19:57:58 | B | F | 37,465 | 370,969,616 | T |
| 5 | BackupClient-turtle-011 | 2010-12-04 20:04:44 | B | F | 38,031 | 335,572,848 | T |
| 8 | BackupClient-turtle-033 | 2010-12-04 20:10:20 | B | F | 64,028 | 447,847,840 | T |
| 9 | BackupClient-turtle-119 | 2010-12-04 20:18:24 | B | F | 125,963 | 3,129,319,856 | T |
| 10 | BackupClient-turtle-120 | 2010-12-04 20:41:03 | B | F | 37,423 | 477,046,384 | T |
| 12 | BackupClient-rabbit | 2010-12-04 21:01:24 | B | F | 177,292 | 2,023,360,064 | T |
| 14 | BackupClient-weasel | 2010-12-04 22:19:47 | B | I | 4,473 | 4,881,982,486 | T |
| 15 | BackupClient-turtle-121 | 2010-12-04 22:24:15 | B | F | 289,971 | 22,271,267,808 | T |
| 28 | BackupClient-hedgehog-011 | 2010-12-04 22:30:31 | B | F | 41,646 | 484,639,376 | T |
| 29 | BackupClient-hedgehog-010 | 2010-12-04 22:38:04 | B | F | 37,461 | 376,538,256 | T |
| 31 | ErrorMigrateTemp | 2010-12-04 22:40:38 | g | F | 0 | 0 | T |
| 30 | BackupClient-hedgehog-007 | 2010-12-04 22:42:48 | B | F | 48,016 | 465,592,512 | T |
| 32 | BackupClient-hedgehog | 2010-12-04 22:46:03 | B | F | 39,128 | 403,276,240 | T |
| 33 | BackupClient-blackbird | 2010-12-04 22:53:19 | B | F | 54,604 | 496,077,344 | T |
| 34 | BackupClient-bear | 2010-12-04 23:00:41 | B | F | 254,799 | 5,516,363,088 | T |
| 36 | BackupClient-bear-nfs | 2010-12-05 00:55:06 | B | F | 140,757 | 72,794,092,336 | T |
| 37 | BackupClient-turtle-121 | 2010-12-05 02:06:03 | B | I | 269,108 | 22,270,624,064 | T |
| 39 | BackupClient-bear | 2010-12-05 05:05:02 | B | D | 120 | 4,064,688 | T |
| 40 | BackupClient-hedgehog-010 | 2010-12-05 08:06:04 | B | I | 1,396 | 869,839,438 | T |
| 41 | BackupClient-turtle-002 | 2010-12-05 09:04:03 | B | F | 71,184 | 517,900,992 | T |
| 42 | BackupClient-bear-nfs | 2010-12-05 09:05:03 | B | D | 221 | 155,519,568 | T |
| 43 | BackupClient-turtle-033 | 2010-12-05 10:06:03 | B | I | 62,799 | 447,632,320 | T |
| 44 | BackupClient-turtle-010 | 2010-12-05 11:06:03 | B | I | 1,399 | 866,036,798 | T |
| 45 | BackupClient-turtle-121 | 2010-12-05 11:14:08 | B | I | 0 | 0 | R |
+-------+---------------------------+---------------------+------+-------+----------+----------------+-----------+
*llist jobid=4
JobId: 4
Job: BackupClient-turtle-010.2010-12-04_19.57.55_06
Name: BackupClient-turtle-010
PurgedFiles: 0
Type: B
Level: F
ClientId: 13
Name: bacula-fd-turtle-010
JobStatus: T
SchedTime: 2010-12-04 19:57:51
StartTime: 2010-12-04 19:57:58
EndTime: 2010-12-04 20:04:42
RealEndTime: 2010-12-04 20:04:42
JobTDate: 1,291,493,082
VolSessionId: 2
VolSessionTime: 1,291,490,553
JobFiles: 37,465
JobErrors: 0
JobMissingFiles: 0
PoolId: 6
PooLname: Pool-File-pbdhetzner
PriorJobId: 0
FileSetId: 2
FileSet: DefaultFileset
*llist jobid=44
JobId: 44
Job: BackupClient-turtle-010.2010-12-05_11.06.00_45
Name: BackupClient-turtle-010
PurgedFiles: 0
Type: B
Level: I
ClientId: 13
Name: bacula-fd-turtle-010
JobStatus: T
SchedTime: 2010-12-05 11:06:00
StartTime: 2010-12-05 11:06:03
EndTime: 2010-12-05 11:08:23
RealEndTime: 2010-12-05 11:08:23
JobTDate: 1,291,547,303
VolSessionId: 5
VolSessionTime: 1,291,501,310
JobFiles: 1,399
JobErrors: 0
JobMissingFiles: 0
PoolId: 6
PooLname: Pool-File-pbdhetzner
PriorJobId: 0
FileSetId: 2
FileSet: DefaultFileset
*llist jobid=15
JobId: 15
Job: BackupClient-turtle-121.2010-12-04_22.24.12_04
Name: BackupClient-turtle-121
PurgedFiles: 0
Type: B
Level: F
ClientId: 18
Name: bacula-fd-turtle-121
JobStatus: T
SchedTime: 2010-12-04 22:24:06
StartTime: 2010-12-04 22:24:15
EndTime: 2010-12-04 23:54:16
RealEndTime: 2010-12-04 23:54:16
JobTDate: 1,291,506,856
VolSessionId: 1
VolSessionTime: 1,291,501,310
JobFiles: 289,971
JobErrors: 1
JobMissingFiles: 0
PoolId: 6
PooLname: Pool-File-pbdhetzner
PriorJobId: 0
FileSetId: 4
FileSet: turtle-121Fileset
*llist jobid=37
JobId: 37
Job: BackupClient-turtle-121.2010-12-05_02.06.00_37
Name: BackupClient-turtle-121
PurgedFiles: 0
Type: B
Level: I
ClientId: 18
Name: bacula-fd-turtle-121
JobStatus: T
SchedTime: 2010-12-05 02:06:00
StartTime: 2010-12-05 02:06:03
EndTime: 2010-12-05 03:50:42
RealEndTime: 2010-12-05 03:50:42
JobTDate: 1,291,521,042
VolSessionId: 2
VolSessionTime: 1,291,501,310
JobFiles: 269,108
JobErrors: 1
JobMissingFiles: 0
PoolId: 6
PooLname: Pool-File-pbdhetzner
PriorJobId: 0
FileSetId: 4
FileSet: turtle-121Fileset
*stat dir [ used to control everything ]
bacula-dir-pbdhome Version: 5.0.2 (28 April 2010) x86_64-redhat-linux-gnu redhat
# uname -r
2.6.34.7-61.fc13.x86_64
# cat /etc/redhat-release
Fedora release 13 (Goddard)
*stat storage=Storage-File-pbdhome [ used for weasel/rabbit/bear/hedgehog* ]
bacula-sd-pbdhome Version: 5.0.2 (28 April 2010) x86_64-redhat-linux-gnu redhat
# uname -r
2.6.34.7-61.fc13.x86_64
# cat /etc/redhat-release
Fedora release 13 (Goddard)
*stat storage=Storage-File-pbdbuchs [ used for grasshopper/butterfly ]
bacula-sd-pbdbuchs Version: 3.0.3 (18 October 2009) i386-redhat-linux-gnu redhat
# uname -r
2.6.32.10-90.fc12.i686.PAE
# cat /etc/redhat-release
Fedora release 12 (Constantine)
*stat storage=Storage-File-pbdhetzner [ used for all turtle* ]
bacula-sd-pbdhetzner Version: 5.0.2 (28 April 2010) x86_64-redhat-linux-gnu redhat
# uname -r
2.6.35.6-48.fc14.x86_64
# cat /etc/redhat-release
Fedora release 14 (Laughlin)
*stat client=bacula-fd-turtle-010 [ going to Storage-File-pbdbuchs ]
bacula-fd-turtle-010 Version: 5.0.2 (28 April 2010) x86_64-redhat-linux-gnu redhat
# uname -r
2.6.34.7-56.fc13.x86_64
# cat /etc/redhat-release
Fedora release 13 (Goddard)
*stat client=bacula-fd-turtle-121 [ going to Storage-File-pbdbuchs ]
bacula-fd-turtle-121 Version: 3.0.3 (18 October 2009) x86_64-redhat-linux-gnu redhat
# uname -r
2.6.31.9-174.fc12.x86_64
# cat /etc/redhat-release
Fedora release 12 (Constantine)
*stat client=bacula-fd-bear [ going to Storage-File-pbdhome ]
bacula-fd-bear Version: 2.4.4 (28 December 2008) x86_64-redhat-linux-gnu redhat Enterprise release
# uname -r
2.6.18-92.1.6.el5
# cat /etc/redhat-release
CentOS release 5.2 (Final)
--
----------------------------------------------------------------------
: Ku...@pi... http://www.pinboard.com/ business :
: http://kurt.www.pinboard.com/ private :
----------------------------------------------------------------------
: Unix and Internet Specialist :
: PGP fingerprint 5C88 FAD4 E111 F33E DA3E 2EC5 DF0E 220F FFD6 2FF1 :
----------------------------------------------------------------------
|
