[Bacula-users] filesize storage in catalog versus estimate commands

[Olaf Z. <ol...@ar...>]

Dear List,

I am a bit puzzled after I decided to check if a database is actually in 
the backup made. I use the FIFO method to backup PostgreSQL databases as 
described on the wiki: 
http://wiki.bacula.org/doku.php?id=application_specific_backups:postgresql
A proper test would include:
- check if files are in the backup
- check if filesize > 0 and are of a size something about right
- retrieve files
- import data into a database
- check if database is ok
After all, we are backing up not for the sake of making backups, but to 
be able to restore.
Like in most articles and notes the author has a focus on making a 
backup and not on restoring the data. Same for the wiki article. So I 
wanted to test stuff and optionally expand the wiki article.

My questions are:
1) what exactly does the command "estimate" when in restore-mode
2) can I get the same result(s) from SQL, if so: how
3) related to 2) : which field in which table of the catalog db holds 
the filesize
4) how are FIFO "results" stored in a Bacula backup. Is it normal to 
size filesize 0 and if so: does that mean that the backup failed or is 
the actual data stored elsewhere.

And now a hands-on approach:

So, first things first:
- find the JobID
jobid turned out to be 1054
next issue on bconsole the command "restore"
from the menu I selected option 3: Enter list of comma separated JobIds 
to select
and entered 1054
This resulted in:

Enter JobId(s), comma separated, to restore: 1054
You have selected the following JobId: 1054

Building directory tree for JobId(s) 1054 ...  
++++++++++++++++++++++++++++++++++++++++++++++
431,198 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /
$ cd /home/system/backupdump/postgresql/gaz/fifo/
cwd is: /home/system/backupdump/postgresql/gaz/fifo/
$ ls
postgres.data.dump
postgres.schema.dump
template1.data.dump
template1.schema.dump
$ mark *
4 files marked.
$ estimate
462396 total files; 4 marked to be restored; 0 bytes.
$ help
  Command    Description
  =======    ===========
  add        add dir/file to be restored recursively, wildcards allowed
  cd         change current directory
  count      count marked files in and below the cd
  delete     delete dir/file to be restored recursively in dir
  dir        long list current directory, wildcards allowed
  done       leave file selection mode
  estimate   estimate restore size
  exit       same as done command
  find       find files, wildcards allowed
  help       print help
  ls         list current directory, wildcards allowed
  lsmark     list the marked files in and below the cd
  mark       mark dir/file to be restored recursively, wildcards allowed
  markdir    mark directory name to be restored (no files)
  pwd        print current working directory
  unmark     unmark dir/file to be restored recursively in dir
  unmarkdir  unmark directory name only no recursion
  quit       quit and do not do restore
  ?          print help


Dan Langille and I got into a discussion on IRC. Mainly because I was 
unaware that there are 2 "estimate" commands. One is described in the 
docs and let's the FD do an estimation on the size of the data to be in 
the backup. However the estimate command in "restore" mode seems to give 
the filesize(s) of file(s) in the backup made by the selected job.
You can test this youself for instance on editing  
/etc/bacula/bconsole.conf (assuming it is being part of a backup):
- estimate in bconsole will give you a filesize
- add a space to the file or something
- estimate in bconsole will give you a different (+1) filesize
- now enter "restore" mode in a job containing this file 
(/etc/bacula/bconsole.conf)
- mark the file to be restored
- estimate command now gives the original filesize (you can change it on 
disk as often as you want, the size remains the same according to this 
command)

This lead me to believe that the filesize is actually stored in the 
catalog DB. Which is not a very surprising thing when you think about it.
Until.....
http://bacula.org/5.0.x-manuals/en/developers/developers/Database_Tables.html
No where is described that it is incorporated in the database and where 
(unless I overlooked something).
The table "File" is a good contender for where it could be stored, but 
as there is no separate field for it I presume it is shuffled into one 
of the blobs (lstat & md5) or .....
Anyway I would like to know:
* can I query the DB to check for filesizes
* how is FIFO data stored
* is a size of 0 a reason to think that the FIFO backup failed

Any thoughts appreciated,

Olaf


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.