Bacula

Hello Phil,

Those are good questions that a lot of people might
be asking.

Concerning the access permissions on WinNT: if you know some way
I can read these files, please let me know. Apparently WinNT has
some sort of exclusive lock that blocks even read-only access to
the files. If there is some way around it, I'll be happy to fix
the code, but at the present time, I don't know how.

Concerning the ' in the path. This is a bug that I introduced
between version 1.27 and 1.28.  I'll attach the quote patch
to this email -- if I am not mistaken, I sent it in a previous
mail, but that points out the need to have some sort of patch
repository -- I'll look into it.  By the way, I was very tempted
to make a release 1.28c with this patch included, but 
decided against it for fear of having too many releases.
Any comments from you or anyone else on what you prefer
patches or release would be helpful.

Concerning the fact that the SD will not accept the tape:
The problem is that the number of files on the tape do not
agree with what is in the catalog.  It is not always safe
simply to change them as you may in some cases lose some
data, as a consequence the SD simply refuses to write on
the tape. The simplest and best solution is to use a
different tape. The next best is to "purge" the tape,
and make sure it is marked for recycle, then Bacula
will reuse it.

I've done several things to make Bacula more flexible.
Both will be available in 1.29:

1. When the SD is terminating (e.g. from a cancel), it
   attempts to set the proper file position.  This now
   works much better than before, but obviously won't
   help for a Director crash.
2. In the current version 1.29 in the CVS I've added
   a new option to the Console 'update' command that 
   permits resetting the VolFiles variable in the 
   database. However, I consider this somewhat dangerous.

Again, on recovering from "oops" situations like this,
I have for the default taken the conservative approach
to ensure that no data is ever lost. I'm now in the process
of improving the recovery techniques, and have provided
a way to shoot oneself in the foot (with warning
messages included).  

I'll take another look at opening Windows files to see
if there isn't some magic incantation that allows read-only
access ...

Best regards,

Kern

PS: Apply the quote.patch to version 1.27b with
   
  cd <bacula-source>
  patch -p0 <quote.patch
  make
  make install

On Wed, 2003-01-15 at 04:16, Phil Stracchino wrote:
> Kern,
> 
> I just did my first WinBacula test runs, and ran into trouble.  A small
> test backup of c:/Bacula worked perfectly, so I tried to back up c:/ and
> hit problems.  First of all, I got access permissions problems on a
> bunch of files:
> 
> whitestar-fd: Cannot open c:/Documents and Settings/Alaric/Local Settings/Application Data/Microsoft/Windows/UsrClass.dat: ERR=Permission denied.
> whitestar-fd: Cannot open c:/Documents and Settings/Alaric/Local Settings/Application Data/Microsoft/Windows/UsrClass.dat.LOG: ERR=Permission denied.
> whitestar-fd: Cannot open c:/Documents and Settings/Alaric/NTUSER.DAT: ERR=Permission denied.
> whitestar-fd: Cannot open c:/Documents and Settings/Alaric/ntuser.dat.LOG: ERR=Permission denied.
> whitestar-fd: Cannot open c:/Documents and Settings/All Users.WINNT/Application Data/Microsoft/Network/Downloader/qmgr0.dat: ERR=Permission denied.
> whitestar-fd: Cannot open c:/Documents and Settings/All Users.WINNT/Application Data/Microsoft/Network/Downloader/qmgr1.dat: ERR=Permission denied.
> 
> Surely this shouldn't happen for LocalSystem?
> 
> Continuing on from there, Bacula-1.28a appears to have problems with
> filenames containing ' characters coming from win32 file daemons:
> 
> babylon5-dir: Fatal error: sql_create.c:561 sql_create.c:561 insert INSERT INTO Path (Path)  VALUES ('c:/Documents and Settings/All Users.WINNT/Start Menu/Programs/Games/Sid Meier's Alpha Centauri/') failed: You have an error in your SQL syntax near 's Alpha Centauri/')' at line 1
> babylon5-dir: Error: sql_create.c:563 Create db Path record INSERT INTO Path (Path)  VALUES ('c:/Documents and Settings/All Users.WINNT/Start Menu/Programs/Games/Sid Meier's Alpha Centauri/') failed. ERR=You have an error in your SQL syntax near 's Alpha Centauri/')' at line 1
> babylon5-dir: Error in sql_create.c:578 Failed ASSERT: ar->PathId
> babylon5-dir: Fatal Error because: Bacula interrupted by signal 11: Segmentation violation
> 
> So then after the director crashed, I stopped bacula, saved a few tweaks
> to the config files I'd been working on in the meantime, restarted
> bacula, and tried to run a smaller test job.  However:
> 
> babylon5-dir: Start Backup JobId 7, Job=Test.2003-01-14_22.05.57
> babylon5-sd: Volume VXAtape-0001 previously written, moving to end of data.
> babylon5-sd: Test.2003-01-14_22.05.57 Error: I canot write on this volume because:
> The number of files mismatch! Volume=6 Catalog=5
> 
> So now I somehow need to repair this too.  There's nothing crucial on
> the tape yet, it's all tests, but there needs to be a way to recover
> from "oops" events like this.
> 
> 
> 
>

Hello Phil,

Concerning bug reports and patches, after a bit of
playing around and learning, I find that the features
offered by the SourceForge bug tracking and patch 
tracking facilities are quite adequate, so I will
make an effort to keep them updated with any bugs
that I am aware of and any patches that I develop.

Best regards,

Kern

On Wed, Jan 15, 2003 at 12:56:15PM +0100, Kern Sibbald wrote:
> Hello Phil,
> 
> Concerning bug reports and patches, after a bit of
> playing around and learning, I find that the features
> offered by the SourceForge bug tracking and patch 
> tracking facilities are quite adequate, so I will
> make an effort to keep them updated with any bugs
> that I am aware of and any patches that I develop.

OK.  Would you prefer I use sourceforge to submit bug reports in future?

Any insight on the filename failure? I thought I recalled you just fixed
this recently.  Since I don't know what exactly you did to fix it, is it
possible it's fixed only for the *nix fd?


-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

Hello Phil,



On Wed, 2003-01-15 at 17:04, Phil Stracchino wrote:
> On Wed, Jan 15, 2003 at 12:56:15PM +0100, Kern Sibbald wrote:
> > Hello Phil,
> > 
> > Concerning bug reports and patches, after a bit of
> > playing around and learning, I find that the features
> > offered by the SourceForge bug tracking and patch 
> > tracking facilities are quite adequate, so I will
> > make an effort to keep them updated with any bugs
> > that I am aware of and any patches that I develop.
> 
> OK.  Would you prefer I use sourceforge to submit bug reports in future?

Administratively for both of us, unless you are 100% sure it is
a bug, I suggest you use the list as you are now, and I'll submit
the bugs myself. However, feel completely free to use the SourceForge
interface, which ever is easier for you.

The above advice goes for everyone else too.  

> 
> Any insight on the filename failure? I thought I recalled you just fixed
> this recently.  Since I don't know what exactly you did to fix it, is it
> possible it's fixed only for the *nix fd?

I'm not sure what you mean by the filename failure.  If you 
are referring to the crash you reported in your previous
email, that was in fact a quote (') in a path, and I sent
you the patch for it. The problem occurred on any system
with a quote in a path -- generally only Windows is stupid
enough to create such monstrosities, but it is perfectly
legal on Unix systems as well.

I'll also be posting a buffer overrun patch for 
long filenames -- it should be on SourceForge within
several hours.

Best regards,

Kern

On Wed, Jan 15, 2003 at 05:51:02PM +0100, Kern Sibbald wrote:
> I'm not sure what you mean by the filename failure.  If you 
> are referring to the crash you reported in your previous
> email, that was in fact a quote (') in a path, and I sent
> you the patch for it.

I'm sorry, I misunderstood your mail.  I assume the patch is on
sourceforge; I'll go find it there.  I was well aware it was the ' that
caused the failure, but I'd thought that was already fixed in 1.28a.


-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

On Wed, Jan 15, 2003 at 10:47:41AM +0100, Kern Sibbald wrote:
> Hello Phil,
> 
> Those are good questions that a lot of people might
> be asking.
> 
> Concerning the access permissions on WinNT: if you know some way
> I can read these files, please let me know. Apparently WinNT has
> some sort of exclusive lock that blocks even read-only access to
> the files. If there is some way around it, I'll be happy to fix
> the code, but at the present time, I don't know how.


A friend who's an experienced win32 coder told me this:

<chad> you have to use the flag to the CreateFile API (the open call)
<chad> that tells it that you're backing it up, and then it will take a
<chad> system state image of that file.
<chad> if you're trying to cheat and do a file archive for a backup, stop
<chad> right there because it won't work. 




-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

Oops, hit "send" too soon ...

<chad> CreateFile, BackupRead, BackupWrite are the APIs. there is an
<chad> article in MSDN that describes how to build a backup application,
<chad> too.



-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

Continuing the win32 testing, a full backup of the win32 box failed when
the storage daemon croaked.  I don't have a lot of information; here's
what I have:

(on stderr)

Kaboom! bacula-sd, babylon5-sd got signal 11. Attempting traceback.
execv: /opt/bacula/bin//btraceback failed: ERR=No such file or directory
Traceback complete, attempting cleanup ...
 M  j  09 00 00 00 00 00 4  0C 08 00 05 00 00 00
 08 00 00 00 00 80 00 00 8F #  00 00 00 00 00 00
 =  00 00 00 00 00 00 00 00 00 00 00 00 E9 07 00


(console log file)

15-Jan-2003 15:39 whitestar-fd: Whitestar_Save.2003-01-15_14.13.11
Error: bnet.c:322 Write error sending to Storage daemon:babylon5:9103:
ERR=Connection reset by peer

<a bunch of not-really-interesting "cannot open source file, permission
denied by Win2K" errors skipped>

15-Jan-2003 15:43 babylon5-dir: Whitestar_Save.2003-01-15_14.13.11
Error: Bacula 1.28b (06Jan03): 15-Jan-2003 15:43
JobId:                  3
Job:                    Whitestar_Save.2003-01-15_14.13.11
FileSet:                Windows Full Set
Backup Level:           Full
Client:                 Whitestar
Start time:             15-Jan-2003 14:13
End time:               15-Jan-2003 15:43
Files Written:          42,809
Bytes Written:          11,738,950,968
Rate:                   2178.3 KB/s
Software Compression:   2.1 %
Volume names(s):        
Volume Session Id:      1
Volume Session Time:    1042657938
Last Volume Bytes:      6,485,239,879
Termination:            *** Backup Error ***



The tape drive is an Ecrix VXA-1, 66GB rated capacity (33GB uncompressed
native), with hardware compression enabled.  The tape is known good and
should have been slightly over half full.  The tape drive continued to
write *something* for almost ten minutes after the fd crashed;
apparently, it finished writing the CD image file that it was writing at
the time the sd crash occurred.  The director process was apparently not
adversely impacted.



-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

Hello Phil,

I've never seen the SD crash, so this is unusual.
Can you run it again, but under the debugger?
The Kaboom chapter of the manual explains how.

Thanks for the WinAPI tips -- I'll look into it.

Best regards,

Kern

On Wed, 2003-01-15 at 21:56, Phil Stracchino wrote:
> Continuing the win32 testing, a full backup of the win32 box failed when
> the storage daemon croaked.  I don't have a lot of information; here's
> what I have:
> 
> (on stderr)
> 
> Kaboom! bacula-sd, babylon5-sd got signal 11. Attempting traceback.
> execv: /opt/bacula/bin//btraceback failed: ERR=No such file or directory
> Traceback complete, attempting cleanup ...
>  M  j  09 00 00 00 00 00 4  0C 08 00 05 00 00 00
>  08 00 00 00 00 80 00 00 8F #  00 00 00 00 00 00
>  =  00 00 00 00 00 00 00 00 00 00 00 00 E9 07 00
> 
> 
> (console log file)
> 
> 15-Jan-2003 15:39 whitestar-fd: Whitestar_Save.2003-01-15_14.13.11
> Error: bnet.c:322 Write error sending to Storage daemon:babylon5:9103:
> ERR=Connection reset by peer
> 
> <a bunch of not-really-interesting "cannot open source file, permission
> denied by Win2K" errors skipped>
> 
> 15-Jan-2003 15:43 babylon5-dir: Whitestar_Save.2003-01-15_14.13.11
> Error: Bacula 1.28b (06Jan03): 15-Jan-2003 15:43
> JobId:                  3
> Job:                    Whitestar_Save.2003-01-15_14.13.11
> FileSet:                Windows Full Set
> Backup Level:           Full
> Client:                 Whitestar
> Start time:             15-Jan-2003 14:13
> End time:               15-Jan-2003 15:43
> Files Written:          42,809
> Bytes Written:          11,738,950,968
> Rate:                   2178.3 KB/s
> Software Compression:   2.1 %
> Volume names(s):        
> Volume Session Id:      1
> Volume Session Time:    1042657938
> Last Volume Bytes:      6,485,239,879
> Termination:            *** Backup Error ***
> 
> 
> 
> The tape drive is an Ecrix VXA-1, 66GB rated capacity (33GB uncompressed
> native), with hardware compression enabled.  The tape is known good and
> should have been slightly over half full.  The tape drive continued to
> write *something* for almost ten minutes after the fd crashed;
> apparently, it finished writing the CD image file that it was writing at
> the time the sd crash occurred.  The director process was apparently not
> adversely impacted.
> 
>

On Wed, Jan 15, 2003 at 10:11:33PM +0100, Kern Sibbald wrote:
> Hello Phil,
> 
> I've never seen the SD crash, so this is unusual.
> Can you run it again, but under the debugger?
> The Kaboom chapter of the manual explains how.

I'll try, but I don't know if I can reproduce it.  We'll see, I guess.


-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :

On Wed, Jan 15, 2003 at 10:11:33PM +0100, Kern Sibbald wrote:
> Hello Phil,
> 
> I've never seen the SD crash, so this is unusual.
> Can you run it again, but under the debugger?
> The Kaboom chapter of the manual explains how.

OK, I did the necessary setup to make traceback work and ran the same
backup job again, but I couldn't reproduce the crash.  Now, though, I'll
be able to get a traceback if it happens again.


-- 
 .*********  Fight Back!  It may not be just YOUR life at risk.  *********.
 : phil stracchino : unix ronin : renaissance man : mystic zen biker geek :
 :  alaric@... : alaric-ruthven@... : phil@...  :
 :   2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)   :
 :    Linux Now!   ...Because friends don't let friends use Microsoft.    :