[Bacula-devel] Re: File daemon enhancements

[Kern S. <ke...@si...>]

Hello Johanes,

In response to your question about where to publish ideas:

- for development ideas please copy the bacula-devl list.
- for how to do or user questions please copy the bacula-users
  list.

Yes, Bacula is designed to permit as many different kinds of
data streams as we need.  Along the lines you mention, I have
two ideas: The first:. FSM (filesystem modules), which would be built
in code within Bacula that would handle special files or
other such things. (one day, they might even be plugins).  
For example, we need a FSM to handle Mac files with their
code and data forks.
The second idea: allow Bacula to exec an external program
with a filename as an argument. This program would then read
the file and send the output to Bacula, which would store
it on tape.  There must of course be the counter part: a
program that gets raw data from Bacula from STDIN and
writes it to a file.  I think this is the most general
way of handling the problem, and probably the most
flexible.

The big problem with idea 1 is that it requires compiling
code into Bacula, but it will be well integrated and
efficient.

The big problem with idea 2 is that I have not figured
out how to link the read/write program names to the FileSet 
to be backed up.  In general, the name of these programs
(or the path to them) must also be stored on tape so that
any program reading the tape without the database and without
the original bacula-dir.conf file can figure out how to
get the data off the tape correctly.

Both of these ideas are briefly mentioned in my kernstodo
file in the Bacula source directory.

At the current time, I am focusing on completing the
Roadmap that I laid out in August (also in kernstodo).

I'd be very happy to hear any ideas you have on how to
do this and how important you think it is relative to
the Roadmap projects. 


Commenting more directly on what you wrote. It would
seem to me that executing the external program needs
to be done in the code that reads the filesystem, and
in the code that writes the filesystem. This code is
currently is currently both in the File daemon
and in the standalone Storage daemon tools. So, it 
probably isn't really necessary to "split" the File
daemon into two programs, rather make it smarter 
so that it can either read/write the filesystem directly,
or read/write STDIN/STDOUT after execing a program.

I would be very happy to see you get your spare PC 
up and running and doing a little programming. :-)

Best regards,

Kern


On Thu, 2002-10-10 at 12:16, Johanes Kalbus wrote:
> Hello Kern,
> 
> yesterday evening I was playing around with bacula - and I got an idea for the 
> File Daemon. How could it be handled if you want to backup a database or 
> something else - how could you send a "data stream" with different data than 
> filesystem.
> 
> I think it should be possible to specify a command which creates a data stream.
> an example would be: dump a database online and send that dump including the 
> information about the tables to the storage daemon. That would it make possible 
> to backup a database without stopping it. Or give an application an API to send 
> the data to the storage daemon as if it were a locally attached drive.
> 
> There it would be necessary to split the file daemon in a communication daemon 
> and a program which is spawned each time a backup starts. Thats in a rough 
> description what networker does.
> 
> Is there another way to publish ideas, requests, wishes other than sending you 
> a mail? And I think with my ideas I should get my spare PC up and running just 
> to do a little development...
> 
> Yours
>   Johanes Kalbus
> 
> -----------------------------------------------------
> !  Johanes Kalbus        !  http://www.kalbus.de/   !
> !  Danziger Str. 32      !  Tel.: 09187 / 959 343   !
> !  90518 Altdorf         !  Fax.: 09187 / 959 344   !
> -----------------------------------------------------