On Thursday 28 December 2006 01:04, Dan Trainor wrote:
> Kern Sibbald wrote:
> > On Tuesday 26 December 2006 20:23, Dan Trainor wrote:
> >> Kern Sibbald wrote:
> >>> On Tuesday 26 December 2006 11:02, James Harper wrote:
> >>>> Assuming that the user would be responsible for the initial
> >>>> etc, is there any reason that a generic 'bare metal' restore CD could
> >>>> not be made? It looks like the catalogs and bootstrap files can be
> >>>> recreated with the bscan etc tools, so unless I'm missing something, it
> >>>> should be possible to create a CD with the client daemon (for restoring
> >>>> where the director is on another machine and is still up) and the
> >>>> storage daemon (for bextract where the director is not available).
> >>>> Or maybe this already exists? I looked but couldn't find anything...
> >>>> currently tinkering with dfsbuild to see what it can put together.
> >>>> Also, if a pre-backup operation could create a system config file with
> >>>> all the partitioning and other bootstrap info, then even that could be
> >>>> restored off of a tape at the start to automate the process.
> >>>> The reason I'm so interested in this is that the existing documented
> >>>> disaster recovery procedure seems to include a lot of pre-disaster work
> >>>> and ongoing maintenance (update your DR CD when updating kernels
> >>>> If there was a (more or less) generic ISO that could be downloaded and
> >>>> run, then the whole process might be a lot simpler, and therefore
> >>>> attractive...
> >>> Well, interestingly enough, I am just today working on the Bacula rescue
> >>> CDROM, and I have just about given up the idea of making a generic
> >>> CDROM for a number of reasons that I'll describe below. First, here is
> > what
> >>> I consider the Bacula CDROM to be:
> >>> 1. A snapshot of your hard disk configuration.
> >>> 2. A copy of your current Bacula file daemon that can be run on
> >>> a rescue system (i.e. probably statically linked).
> >>> 3. A bunch of scripts that can be used to do various recovery tasks
> >>> (bring up the network, repartition your hard disks as they were,
> > your
> >>> hard disks, ...). Obviously you would use only those scripts that are
> > really
> >>> necessary.
> >>> 4. All sorts of binaries to make recovery easy.
> >>> 5. All this put together with your current kernel on a CDROM that can be
> >>> booted.
> >>> Now items 1-4 already exist and are more or less straight foward and
> > or
> >>> less Linux distro independent. (If I am not mistaken, these already
> > address
> >>> what you were asking for above).
> >>> Unfortunately, item 5 is rather important, but I've now concluded that
> > is
> >>> far from being distro independent, and way more complicated than
> > I
> >>> want to work on. The problem is that without item 5 (your kernel), you
> >>> cannot be sure that your environment will be setup correctly when
> > the
> >>> rescue disk. This is because there are *major* issues with raid,
> >>> that must be addressed to get a sysstem back up and running without even
> >>> considering the problem of reloading the files.
> >>> Three or four years ago, creating a boot CDROM used to be more or less
> >>> straight forward, but today it is not -- for example, the SuSE 10.2
> > mkinitrd
> >>> script, which sets up a generic boot environment based on your system is
> >>> 3,377 lines of shell script. Needless to say, each distro has a
> >>> mkinitrd script.
> >>> The first iteration of the Bacula rescue disk implement its own code to
> > make
> >>> the rescue boot (item 5). As we started seeing 2.6 kernels, this
> >>> implementation failed more and more often. The second iteration (not yet
> >>> released) uses mkcdrec to actually do the booting. This seemed to work
> >>> pretty well until I moved to SuSE 10.2, and now it no longer boots. So,
> > the
> >>> bottom line is that I give up on trying to implement item 5 -- it is
> > to
> >>> complicated, too distro dependent, and a constantly moving target.
> >>> I have not yet found a satisfactory solution. Every distro has a rescue
> > disk.
> >>> However, most of them boot up without mounting the disks and leave you
> >>> sitting at a raw command prompt. If you are like me, you are not even
> >>> capable of mounting a CDROM because I cannot (refuse) to remember all
> >>> silly options and the syntax needed to mount a CDROM (it is trivial if
> >>> have the right fstab). So, we are (I am) sort of dead in the water.
> >>> The path I am exploring for the moment is simply packaging the output
> >>> items 1-4 onto tar file that the user can save to a floppy or a CDROM.
> > am
> >>> also considering the possibility of remastering rescue disks and adding
> > the
> >>> Bacula data, but that is probably also a black hole of distro dependent
> >>> code ...
> >>> If anyone has some better ideas, I would appreciate it to hear them ...
> >>> Best regards,
> >>> Kern
> > Hello,
> >> Good afternoon, Kern -
> >> I've not yet read up on the bare metal recovery process, except for a
> >> quick fly-by just to get the basic setup in my head. I'm still very new
> >> to Bacula, and I think that a bare-metal restore of anything is quite
> >> out of the question at this point.
> > Well, from a Bacula standpoint, it is a piece of cake compared to the boot
> > process (initramdisk, ramfs, lvm, raids of all sorts, ...).
> >> However, I have ample experience in rolling my own CentOS/RHEL-based
> >> boot CDs for a job that I used to work at, and I think that a single
> >> generic CD, which is to be booted on a bare metal machine, would work
> >> just fine.
> > Those are *exactly* the qualifications that the Bacula rescue project
> > this point.
> >> It's my understanding that a spin-off could be created to support a very
> >> large amount of generic hardware, and for the recovery process, the
> >> kernel version would not matter in the slightest. The kernel and
> >> associated modules would only be used in 'live' mode, so they would not
> >> even be touching the disk.
> > Uh, you are sort of right and sort of wrong. Yes, the *exact* kernel
> > and the Linux distro are probably not critical. However, you are never
> > to be able to do a restore with a 2.0 kernel to a 2.6 kernel based system.
> > Even a 2.4 kernel might have problems doing a 2.6 restore -- this is
> > the kernel APIs do subtly changing over time and there are lots of new
> > features being added such as lvm and raid (and other 3 letter terms that I
> > don't understand at all -- evms, dasd, zfcp, lvm2, dmraid, ...), which are
> > not likely to be supported by older kernels. Anyway, you probably get the
> > idea.
> > However, I agree, that it is really inmaterial whether or not the distro
> > FC6, RHEL4.x or SuSE 10.2 since they all have very recent kernels.
> >> I'd like to think of it as a live distribution (Barecula? har har har)
> >> which simply contains support for the hardware - via modules, etc etc -,
> >> has a file daemon on it, network support, and knows where to look for
> >> partition information to restore. Assuming bare-metal recovery implies
> >> that the machine should have absolutely nothing on it to begin with, I
> >> don't see why we would need to be concerned about what kernel we use in
> >> such a beast - however, it of course would be as current as it can be to
> >> support hardware which might be a little awkward, such as RAID and
> >> Network and such.
> > Yes, bare-metal assumes you have everything you need on the CDROM (well,
> > the file data comes to the Bacula File daemon across the network). The
> > I am having problems with is that I need a rescue boot disk that has:
> > - Additional binaries in /bin
> > - Additional libriaries in /lib (so everything doesn't need to be static)
> > - At least one additional directory on the CD that will be loaded into a
> > ramfs.
> > - A second directory on the CD that could contain additional data to be
> > in memory if needed.
> > - A very simple process for allowing the user to add the information above
> > and then create the iso.
> > - The kernel and all the boot process could potential already be packaged,
> > there needs to be a relatively straight forward way that someone in the
> > Bacula project could update it from time to time (i.e. between major
> > releases).
> > - Ideally (and this probably not a problem that you or I can solve) each
> > Linux distro would supply a mkrescue script that would allow me to
> > the two directories mentioned above, it would then package everything
> > together into an iso -- basically it is a modest extention of the
> > script that each distro has, but modified for a rescue disk (i.e.
> > attempt to mount the disks) rather than doing a full boot.
> >> As far as specifics go re: partitioning, if we do replace say a failing
> >> disk with a bigger/smaller one, like you said earlier, a template could
> >> be used. A script could facilitate this template, which would allow one
> >> to either let the restore process "expand" on the template and fill up
> >> remaining space, or the user can define the space. As I understand,
> >> Bacula wouldn't give a damn about what an actual partition is, unless
> >> you're doing a backup or snapshot of the drive or partition itself - not
> >> just the mount point. The name by which Bacula calls these escapes me
> >> right now, but I hope that you understand where I'm going with this.
> > Bacula just restores files to directories. It doesn't concern itself with
> > partitions and mount points. These must all be setup prior to running
> > Bacula.
> > I think the rescue disk as I have described above would be a giant step
> > forward as it would allow experienced users to rather easily restore their
> > systems from bare metal. The old Bacula rescue used to do this with 2.4
> > kernels -- it worked quite fine, then 2.6 came along and things got
> > but I got it working, and now with all the above mentioned 3 letter words,
> > no longer works.
> > I think over time, if all the Linux distros supplied a mkrescue script
> > had the appropriate hooks, we would see a lot of improvements -- e.g.
> > interfaces rather than command line, more automatic scripts, i.e.
> > more like the distro installers but rather than installing a distro they
> > attempt to repair it or get it into a state where a restore program can
> >> Anyway, I'd be happy to tread this thread and get some ideas, perhaps
> >> being able to deliver something that might achieve this goal. WHat are
> >> your additional thoughts on the subject?
> > Hopefully the above gives you an idea of what I envision. For someone who
> > a whiz at creating special boot disks (or rescue disks), supplying the
> > may not be too difficult. As for the Bacula end, that is something that
> > already exists, it just needs to be repackaged appropriately.
> > Thanks for your interest in this.
> > Best regards,
> > Kern
> Hi, Kern -
> I'll go ahead and see what I can do about getting a basic CD rolled.
> It's been a while since I have done it, so I will be a bit rusty, for
> sure - and for that reason, I cannot give you a timeframe.
What would help me the most would be to checkout the current Bacula "rescue"
module from the CVS and run it on your machine and see if it works. I think
it would be a good starting point.
> However, I'll put a fair amount of work into it. I completely see what
> you're saying about the differences, not necessarily between kernel
> versions, but other things such as LVM. Which makes me wonder - would
> it be possible/wise to investigate Bacula's ability to back all that
> information up, as well?
At the current stage of evolution of Bacula, we attempt wherever possible to
avoid doing too much system dependent code (I don't exclude it in the
future). As such, it would be much better as I indicated in my previous
email to run the "make bacula" in the rescue directory to generate that
information, then backup the appropriate directory. It is not as automatica
as it could be, but it is a good place to start and doesn't require modifying
Bacula source code.
> I might have misunderstood how Bacula worked a little bit there in
> regards to setting up things such as partitions and LVM. I believe the
> ideal solution for a good restore would include Bacula's ability to back
> up not only files and directories, but also partition info, layouts,
> LVM, SCSI interfaces, blah blah blah.
Yes, please see my previous comment on how to attack this problem in the
immediate future. Longer term, perhaps it will be possible to integrate some
of that directly into Bacula -- however, I am skeptical on this point given
the complexity and the rapidity at which I have seen Linux filesystems and
partition technology advance -- as well as the vendor (Distro) specific
> See we could spend a good amount
> of time making an all-in-one boot CD to allow for the re-initialization
> of said devices, or Bacula could learn how to restore this information.
> Of course, I'm in no position to dictate how the development process
> should work, so I guess what I'm asking here is - is that sort of thing
> planned for the near future?
Yes, it is planned as a sort of add-on through the rescue disk. However, even
in that state, I am finding it next to impossible to maintain since I am not
really interested in being a filesystem, raid, lvm, ... expert for even one
distro much less all the distros.
> Will Bacula eventually be able to back up
> all that data in addition to just files and directories?
Probably never directly, but by running appropriate machine dependent scripts,
it is completely possible today. It just needs someone to organize and
> None the less, I'll get rolling on this later this evening for sure, and
> see if I can make even a little contribution.