Re: [Bacula-devel] Framework for backing up non-filesystems (MSSQL, Exchange, etc)

[James H. <jam...@be...>]

I'm not sure I was very clear when I described my final idea, and it
changed a bit along the way too, so here it is again:

I thought your idea of using helper scripts/executables/whatever was
pretty neat, but could be extended to be very generic such that the user
need not actually do anything but (possibly) add some credentials to a
config file somewhere if required, everything else would be done in
bacula-dir.conf

To back up MSSQL under Windows, we would have a helper called
'mssql.bat' that lives in "C:\Program Files\bacula\helper\mssql.bat".

mssql.bat takes command line arguments that tell it how it should
behave:

1. "dir <relative path>" asks it to give a listing of the items that it
knows how to back up (eg it describes a directory structure that appears
under "C:\Program Files\bacula\helper\mssql.bat\")
2. "backup <filename>" tells it to back up the item (write it to stdout)
3. "restore <filename>" tells it to restore the item (read it from
stdin)

As a simplified version of the example that I've given before,
"mssql.bat dir /" writes a list of items to be backed up to stdout, eg:
/master
/msdb
/model
/james_db

"mssql.bat backup master" would back up the master database and write it
to stdout

To roughly follow a backup through, if the fileset was:

FileSet {
  Name =3D "james"
  Include {
    Options {
      # whatever normal options go here
    }
    File =3D "C:/"
  }
  Include {
    Options {
      portable=3Dyes # not sure that this matters
    }
    File =3D "C:/Program Files/bacula/helper/mssql.bat/james_db"
  }
}

The backup of C:/ happens as normal per the first Include.

In the second Include, bacula-fd gets to mssql.bat, and notices that we
have specified a regular file as if it were a directory (this is what
invokes the new functionality). bacula-fd then bpipe's ".../mssql.bat
dir /james_db", and mssql.bat returns "/james_db" (if we had specified
".../mssql.bat/" in the fileset, instead of ".../mssql.bat/james_db"
then it would return the complete list of databases).

Then bacula-fd calls mssql.bat for each item returned above, eg
".../mssql.bat backup /james_db". mssql.bat does whatever is required to
get the backup done, and writes the resulting data to stdout, which
bacula-fd in turn sends to bacula-sd. The attributes are dummied up by
either bacula-fd or (if it makes sense to do so) a separate call to
mssql.bat.

The resulting backup on tape would look (approximately) like:
C:/
C:/WINDOWS
...
C:/Program Files/
...
C:/Program Files/bacula/helper/mssql.bat/james_db

Am I making any more sense now?

I've had a look at what is required implementation-wise, and it doesn't
appear unpossible... I've implemented the beginnings of the 'dir'
function in as far as that bacula-fd can now detect that we want to
invoke a helper, and can read the resulting directly listing (but not
yet do anything with it).

I think the fundamental difference between your idea and mine is that
you prefer to keep bacula simple and push the configuration complexity
into the scripts (eg 1 for each database), while I would prefer a
simpler configuration at the expense of extra complexity in bacula
itself. Does that sound right?

I think where my idea has the advantage is where you want back up 20
different databases. To do this nicely (eg so that they could be
restored individually from within bacula) your way would require 20
different script pairs and fifo's, while my way would just need a =
File=3D
and maybe some excludes, and also it implicitly backs up any new
databases created without needing any extra configuration work. Also, my
way scales right up to being able to do individual mailbox item restores
in an Exchange database (assuming someone writes a helper, which would
be no small feat! :)

James