From: Kern S. <ke...@si...> - 2002-04-14 21:13:06
|
Hello, Below is a comment by a perspective user ( prefixed with >>) with the response that was given by John Walker. I'm sending this to the list for archiving and because I found John's perspective very interesting. Kern ====== included email ========= >> Just a note that the major competition of Bacula seems to be Amanda, >> and I am being asked why folks should switch over from that to >> Bacula. If they're happy with Amanda, there's no reason they shouldn't stay with it. However, Amanda and Bacula take a very different approach toward backup. Amanda is a high-level wrapper for existing backup utilities such as dump. It manages the tape library, but it doesn't allow simultaneous backups of multiple hosts (essential in the large site environment), nor hierarchical administration, nor multiple storage nodes, and numerous other things. A small to medium size Unix site can do very well with Amanda, but there's a point beyond which it doesn't scale gracefully. That's where Bacula comes in. Eventually, I expect that Bacula will be the easiest solution to install even if you have only a single machine with a tape drive attached; it isn't there now, but there are no forbidding obstacles between where we are now and that goal. Bacula is the skeleton of a complete end to end solution to the backup problem. I say skeleton since there remains a lot to do, which qualified volunteers are welcome to contribute towards. Public key encryption of backups. multiple backup sites, handling of 256 bit file addresses, restoring of bizarre Macintosh multi-fork files as Unix directory trees, etc. all remain on the to do list. Amanda doesn't begin to understand the issues involved in backing up to a remote site over a secure Internet connection to a silo with a dozen simultaneously writable DLT drives. Well, Bacula doesn't handle this either, at the moment, but we've had that goal in mind since the beginning and we know how to get from here to there, and a large part of the infrastructure required is already in place. Also, in the Real World (as much as we might wish we didn't live in it), it is almost trivial to set up the backup of any number of Windows clients to a Linux server with Bacula. This is something *lots* of people need to do, and Amanda doesn't do it. We should have www.bacula.org up within the next few days. |
From: Phil S. <al...@ba...> - 2002-04-19 01:48:33
Attachments:
make_mysql_tables_patch
|
Kern, I just got back to working on setting up bacula (it's been a hectic week), and I get the following error when executing make_mysql_tables: babylon5:root:/opt/bacula/etc:6 # ./make_mysql_tables ERROR 1064 at line 17: You have an error in your SQL syntax near ')' at line 13 ERROR 1064 at line 32: You have an error in your SQL syntax near '' at line 5 ERROR 1064 at line 37: You have an error in your SQL syntax near 'Type BINARY(1) NOT NULL, Level BINARY(1) NOT NULL, ClientId INTEGER NOT NU' at line 1 Creation of Bacula tables succeeded. babylon5:root:/opt/bacula/etc:7 # If I examine the bacula database, I have the following tables: mysql> show tables; +------------------+ | Tables_in_bacula | +------------------+ | Client | | FileSet | | Filename | | JobMedia | | Media | | Path | | Pool | +------------------+ 7 rows in set (0.00 sec) The File and Job tables don't appear to be being created. I've traced the problems to a surplus comma at line 13 of the CREATE TABLE File call, and a semicolon that should be a comma at line 5 of CREATE TABLE Job. A patch is attached. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-19 01:56:26
|
A minor issue with gnome-console: It has no switch to tell it where gnome-console.conf is located, appears not to default to $(sysconfdir), and cannot find it unless it is located in the current directory. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-19 02:03:47
|
On Thu, Apr 18, 2002 at 06:56:25PM -0700, Phil Stracchino wrote: > A minor issue with gnome-console: It has no switch to tell it where > gnome-console.conf is located, appears not to default to $(sysconfdir), > and cannot find it unless it is located in the current directory. Further checking reveals that console also fails to default to $(sysconfdir) for its conf file, though it does honor the -c switch. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-19 06:34:51
|
Hello Phil, If I am not mistaken, all the daemons and programs default their configuration file to the current directory. They all permit the -c option (with the exception of the gnome_init() problem you found with gnome-console). I'd be happy to consider changing the default, but find the current default (current directory) quite convenient for developers. For production, it is no real problem to specify the -c. Let me know what you think. I've noted this item for documentation in my todo list. Best regards, Kern On Fri, 2002-04-19 at 04:03, Phil Stracchino wrote: > On Thu, Apr 18, 2002 at 06:56:25PM -0700, Phil Stracchino wrote: > > A minor issue with gnome-console: It has no switch to tell it where > > gnome-console.conf is located, appears not to default to $(sysconfdir), > > and cannot find it unless it is located in the current directory. > > Further checking reveals that console also fails to default to > $(sysconfdir) for its conf file, though it does honor the -c switch. > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Phil S. <al...@ba...> - 2002-04-19 06:41:50
|
On Fri, Apr 19, 2002 at 08:34:32AM +0200, Kern Sibbald wrote: > Hello Phil, > > If I am not mistaken, all the daemons and programs > default their configuration file to the current > directory. They all permit the -c option (with the > exception of the gnome_init() problem you found > with gnome-console). > > I'd be happy to consider changing the default, but > find the current default (current directory) > quite convenient for developers. For production, > it is no real problem to specify the -c. Let me > know what you think. Hmm, seems counter-intuitive to create a directory for config scripts etc and then require a command-line option to use it. Would it be workable to have it use the config file specified with -c if used, else a config file in the current directory if present, else ${sysconfdir}? Even so, this could lead to unanticipated results if you ran it from a directory that you forgot there was an old config file in. I'd think the safe behavior would be to always look for the config script in ${sysconfdir} by default, unless specified otherwise with the -c option. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-19 07:18:23
|
Hello Phil, Yes, I think you are right. It is (was) logical for me, but certainly not for someone just approaching the package. I'm going to think about it for a day or two. I would like to change the default to look in $(sysconfdir) only (i.e. your "safe behavior"), but here is my problem. I'd like Bacula to have no paths or other information linked into it so that the same binary can work anywhere. This is particularly important on platforms like Windows where generally the user loads a binary, and he might want to install Bacula on drive D rather than C. That means that when Bacula starts, it knows nothing about where files are (except perhaps the local directory). Now to get around that problem and try to make Bacula respect the sysconfdir, I hammer sysconfdir into all the production startup scripts (the developers startit/stopit scripts are excluded). Perhaps a bit more documentation would help? Let me know what you think. Best regards, Kern PS: if all goes well, I'll be releasing an update today now that it has done two nightly saves here (total of 12 "production" jobs). It will include all your patches. On Fri, 2002-04-19 at 08:41, Phil Stracchino wrote: > On Fri, Apr 19, 2002 at 08:34:32AM +0200, Kern Sibbald wrote: > > Hello Phil, > > > > If I am not mistaken, all the daemons and programs > > default their configuration file to the current > > directory. They all permit the -c option (with the > > exception of the gnome_init() problem you found > > with gnome-console). > > > > I'd be happy to consider changing the default, but > > find the current default (current directory) > > quite convenient for developers. For production, > > it is no real problem to specify the -c. Let me > > know what you think. > > Hmm, seems counter-intuitive to create a directory for config scripts etc > and then require a command-line option to use it. Would it be workable to > have it use the config file specified with -c if used, else a config file > in the current directory if present, else ${sysconfdir}? Even so, this > could lead to unanticipated results if you ran it from a directory that > you forgot there was an old config file in. I'd think the safe behavior > would be to always look for the config script in ${sysconfdir} by default, > unless specified otherwise with the -c option. > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Phil S. <al...@ba...> - 2002-04-19 02:40:40
|
A quick question: The example FileSet contains the following note: # Note: / backs up everything There's an important question here. How much does "everything" mean? What I'm specifically asking is, does bacula descend into remotely-mounted (NFS and SMB) filesystems, requiring network-mounted filesystems to be specifically excluded, or are they automatically skipped unless explicitly included? Also, if I have a remote (NFS, let's say) filesystem mounted at /mount/foo, and I exclude /mount/foo, presumably I lose the mountpoint as well. If manual exclusion has to be used, is there a way to backup the mountpoint but not the filesystem mounted on it? -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-19 05:31:00
Attachments:
sql-level-nulls-patch
|
*run job=NightlySave babylon5-dir: ua_run.c:82 Doing arg 1 = job babylon5-dir: ua_run.c:89 Got keyword=job Run Backup job JobName: NightlySave FileSet: Full Set Level: 0 Client: babylon5-fd Storage: VXA1 OK to run? (yes/mod/no): yes 18-Apr-2002 21:59 babylon5-dir: Fatal Error at sql_create.c:94 because: sql_create.c:94 insert INSERT INTO Job (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES (NULL, "NightlySave.2002-04-18.21:59:45", "NightlySave", "B", " failed: You have an error in your SQL syntax near '"' at line 1 18-Apr-2002 21:59 babylon5-dir: Job NightlySave.2002-04-18.21:59:45 Cancelled because: sql_create.c:95 Create DB Job record INSERT INTO Job (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES (NULL, "NightlySave.2002-04-18.21:59:45", "NightlySave", "B", " failed. ERR=You have an error in your SQL syntax near '"' at line 1 The source code here is: Mmsg(&mdb->cmd, "INSERT INTO Job (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES \ (%s, \"%s\", \"%s\", \"%c\", \"%c\", \"%s\", %d)", JobId, jr->Job, jr->Name, (char)(jr->Type), (char)(jr->Level), dt, StartDay); if (!INSERT_DB(mdb, mdb->cmd)) { Mmsg2(&mdb->errmsg, _("Create DB Job record %s failed. ERR=%s\n"), mdb->cmd, sql_strerror(mdb)); jr->JobId = 0; stat = 0; } else { jr->JobId = sql_insert_id(mdb); stat = 1; } Now, here's the question: Are you really sure you want to put jr->Level into that query string as a char, not as an int? If you put it in as a char, then every time you do a level 0 backup you zero-terminate the string in the middle, just the way it's happened here. I'd say that %c format needs to be a %d. The same problem also occurs in sql_update.c and sql_find.c. Now, in theory, Level 0 is apparently an unimplemented backup level. But I couldn't get to that message and find that out until I'd fixed the problem above. (I note that level 0 is listed as "unimplemented" rather than "unsupported", leading me to believe that it's meant to be supported in the future, at which point the SQL command issue above would bite you.) This leads to a logic error: if level 0 is unimplemented, why does "run job=<foo>" offer level 0 as the default level? Patch is attached for the SQL command null-termination problem. The logic issue of an unimplemented backup level being offered as a default by the run command is up to you to solve. I'm posting these patches on sourceforge as well. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-19 07:03:55
|
Hello Phil, Well, the problem is that your backup level is set to zero. This is probably because you did not specify a level in the Job resource (and I always did and never thought of checking for a zero). This bug only occurs when you start from the Console program (I've noted to fix it). There are two solutions: 1. In your Job resource, add "level=Incremental", which will then be the default if nothing else is specified. This is my recommended solution. 2. Reply "mod" to the "OK to run? (yes/mod/no):" and select level, then choose a level. Thanks for finding and reporting this. Best regards, Kern On Fri, 2002-04-19 at 07:30, Phil Stracchino wrote: > *run job=NightlySave > babylon5-dir: ua_run.c:82 Doing arg 1 = job > babylon5-dir: ua_run.c:89 Got keyword=job > Run Backup job > JobName: NightlySave > FileSet: Full Set > Level: 0 <========== this is the problem !!!!!!!!! > Client: babylon5-fd > Storage: VXA1 > OK to run? (yes/mod/no): yes > 18-Apr-2002 21:59 babylon5-dir: Fatal Error at sql_create.c:94 because: > sql_create.c:94 insert INSERT INTO Job (JobId, Job, Name, Type, Level, > SchedTime, StartDay) VALUES (NULL, "NightlySave.2002-04-18.21:59:45", > "NightlySave", "B", " failed: > You have an error in your SQL syntax near '"' at line 1 > 18-Apr-2002 21:59 babylon5-dir: Job NightlySave.2002-04-18.21:59:45 > Cancelled because: sql_create.c:95 Create DB Job record INSERT INTO Job > (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES (NULL, > "NightlySave.2002-04-18.21:59:45", "NightlySave", "B", " failed. ERR=You > have an error in your SQL syntax near '"' at line 1 > > > The source code here is: > > Mmsg(&mdb->cmd, > "INSERT INTO Job (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES \ > (%s, \"%s\", \"%s\", \"%c\", \"%c\", \"%s\", %d)", > JobId, jr->Job, jr->Name, (char)(jr->Type), (char)(jr->Level), dt, > StartDay); > > if (!INSERT_DB(mdb, mdb->cmd)) { > Mmsg2(&mdb->errmsg, _("Create DB Job record %s failed. ERR=%s\n"), > mdb->cmd, sql_strerror(mdb)); > jr->JobId = 0; > stat = 0; > } else { > jr->JobId = sql_insert_id(mdb); > stat = 1; > } > > > Now, here's the question: Are you really sure you want to put jr->Level > into that query string as a char, not as an int? If you put it in as a > char, then every time you do a level 0 backup you zero-terminate the > string in the middle, just the way it's happened here. > > I'd say that %c format needs to be a %d. The same problem also occurs in > sql_update.c and sql_find.c. Now, in theory, Level 0 is apparently an > unimplemented backup level. But I couldn't get to that message and find > that out until I'd fixed the problem above. (I note that level 0 is > listed as "unimplemented" rather than "unsupported", leading me to believe > that it's meant to be supported in the future, at which point the SQL > command issue above would bite you.) > > This leads to a logic error: if level 0 is unimplemented, why does "run > job=<foo>" offer level 0 as the default level? > > > Patch is attached for the SQL command null-termination problem. The logic > issue of an unimplemented backup level being offered as a default by the > run command is up to you to solve. I'm posting these patches on > sourceforge as well. > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > ---- > > --- sql_create.c.orig Thu Mar 21 07:16:11 2002 > +++ sql_create.c Thu Apr 18 22:10:18 2002 > @@ -87,8 +87,8 @@ > /* Must create it */ > Mmsg(&mdb->cmd, > "INSERT INTO Job (JobId, Job, Name, Type, Level, SchedTime, StartDay) VALUES \ > -(%s, \"%s\", \"%s\", \"%c\", \"%c\", \"%s\", %d)", > - JobId, jr->Job, jr->Name, (char)(jr->Type), (char)(jr->Level), dt, > +(%s, \"%s\", \"%s\", \"%c\", \"%d\", \"%s\", %d)", > + JobId, jr->Job, jr->Name, (char)(jr->Type), jr->Level, dt, > StartDay); > > if (!INSERT_DB(mdb, mdb->cmd)) { > --- sql_update.c.orig Tue Mar 12 03:59:56 2002 > +++ sql_update.c Thu Apr 18 22:17:26 2002 > @@ -110,9 +110,9 @@ > date_encode(2000, 1, 1)); > > P(mdb->mutex); > - Mmsg(&mdb->cmd, "UPDATE Job SET Level='%c', StartTime=\"%s\", \ > + Mmsg(&mdb->cmd, "UPDATE Job SET Level='%d', StartTime=\"%s\", \ > ClientId=%d, StartDay=%d WHERE JobId=%d", > - (char)(jr->Level), dt, jr->ClientId, StartDay, jr->JobId); > + jr->Level, dt, jr->ClientId, StartDay, jr->JobId); > stat = UPDATE_DB(mdb, mdb->cmd); > V(mdb->mutex); > return stat; > --- sql_find.c.orig Thu Mar 21 07:10:59 2002 > +++ sql_find.c Thu Apr 18 22:18:41 2002 > @@ -72,14 +72,14 @@ > if (jr->Level == L_DIFFERENTIAL) { > Mmsg(&mdb->cmd, > "SELECT JobId from Job WHERE JobStatus='T' and Type='%c' and \ > -Level='%c' and Name=\"%s\" and ClientId=%d and FileSetId=%d \ > +Level='%d' and Name=\"%s\" and ClientId=%d and FileSetId=%d \ > ORDER by StartTime DESC LIMIT 1", > jr->Type, L_FULL, jr->Name, jr->ClientId, jr->FileSetId); > /* Incremental is since last Full, Incremental, or Differential */ > } else if (jr->Level == L_INCREMENTAL) { > Mmsg(&mdb->cmd, > "SELECT JobId from Job WHERE JobStatus='T' and Type='%c' and \ > -(Level='%c' or Level='%c' or Level='%c') and Name=\"%s\" and ClientId=%d \ > +(Level='%d' or Level='%d' or Level='%d') and Name=\"%s\" and ClientId=%d \ > ORDER by StartTime DESC LIMIT 1", > jr->Type, L_INCREMENTAL, L_DIFFERENTIAL, L_FULL, jr->Name, > jr->ClientId); > @@ -144,11 +144,11 @@ > /* Find last full */ > P(mdb->mutex); > if (jr->Level != L_VERIFY_CATALOG) { > - Emsg2(M_FATAL, 0, _("Expecting Level=%c, got %c\n"), L_VERIFY_CATALOG, jr->Level); > + Emsg2(M_FATAL, 0, _("Expecting Level=%d, got %d\n"), L_VERIFY_CATALOG, jr->Level); > return 0; > } > Mmsg(&mdb->cmd, > -"SELECT JobId from Job WHERE Type='%c' and Level='%c' and Name=\"%s\" and \ > +"SELECT JobId from Job WHERE Type='%c' and Level='%d' and Name=\"%s\" and \ > ClientId=%d ORDER by StartTime DESC LIMIT 1", > JT_VERIFY, L_VERIFY_INIT, jr->Name, jr->ClientId); > |
From: Phil S. <al...@ba...> - 2002-04-20 05:19:23
|
On Fri, Apr 19, 2002 at 09:03:34AM +0200, Kern Sibbald wrote: > There are two solutions: > 1. In your Job resource, add "level=Incremental", > which will then be the default if nothing else > is specified. This is my recommended solution. There turns out to be a problem with this solution: Job { Name = "Test" Backup = Client=babylon5-fd FileSet="Test" Schedule = "WeeklyCycle" Storage = VXA1 Messages = Standard Level = Incremental Pool = Default } .... Starting the Director daemon bacula-dir ABORTING due to ERROR in parse_conf.c:809 Config error: Keyword level not permitted in this resource, : Line 33, col 8 of file /opt/bacula/etc/bacula-dir.conf Level = Incremental /opt/bacula/sbin/bacula: line 189: 8618 Aborted /opt/bacula/sbin/bacula-dir $2 -c /opt/bacula/etc/bacula-dir.conf -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-20 05:42:35
|
Suppose I'm writing a FileSet resource, and I want to back up (for example) the /tmp directory itself, but not its contents. (Note this specific example doesn't apply to my case, since all my tmp dirs are tmpfs, but it's a probably-common example.) My /tmp directory currently contains the following: babylon5:alaric:~:2 $ ls -A /tmp .ICE-unix/ icbrc mutt-babylon5-9279-1 .X11-unix/ ksocket-alaric/ mysql.sock= .crond_running mcop-alaric/ screens/ .font-unix/ mutt-babylon5-28767-1 session_mm_apache0.sem .vmware/ mutt-babylon5-28767-2 ssh-LsI15410/ elvis1.ses mutt-babylon5-28767-2.asc fileEfdBtr mutt-babylon5-28767-3 Do I: exclude { /tmp } Or do I: exclude { /tmp/* } And, if the latter (which I'm assuming to be the case), will this *also* skip /tmp/.ICE-unix/, /tmp/.X11-unix/, /tmp/.crond_running, /tmp/.font-unix/, and /tmp/.vmware/, or do I also need to exclude /tmp/.??* as well? -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-20 07:45:02
|
Hello Phil, Gee, I cannot answer this one. Sorry. I'm not sure that it is possible to backup just a directory entry with Bacula. I never really thought about doing that. I'd recommend that you build and play with <bacula>/src/findlib/testfind to see what it actually does. I'll be interested to see your findings and to accept any RFCs. Best regards, Kern On Sat, 2002-04-20 at 07:42, Phil Stracchino wrote: > Suppose I'm writing a FileSet resource, and I want to back up (for > example) the /tmp directory itself, but not its contents. (Note this > specific example doesn't apply to my case, since all my tmp dirs are > tmpfs, but it's a probably-common example.) My /tmp directory currently > contains the following: > > > babylon5:alaric:~:2 $ ls -A /tmp > .ICE-unix/ icbrc mutt-babylon5-9279-1 > .X11-unix/ ksocket-alaric/ mysql.sock= > .crond_running mcop-alaric/ screens/ > .font-unix/ mutt-babylon5-28767-1 session_mm_apache0.sem > .vmware/ mutt-babylon5-28767-2 ssh-LsI15410/ > elvis1.ses mutt-babylon5-28767-2.asc > fileEfdBtr mutt-babylon5-28767-3 > > > Do I: > > exclude { > /tmp > } > > Or do I: > > exclude { > /tmp/* > } > > And, if the latter (which I'm assuming to be the case), will this *also* > skip /tmp/.ICE-unix/, /tmp/.X11-unix/, /tmp/.crond_running, > /tmp/.font-unix/, and /tmp/.vmware/, or do I also need to exclude > /tmp/.??* as well? > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Phil S. <al...@ba...> - 2002-04-21 18:33:44
|
...Right now, the OK to run? (yes/mod/no): prompt *only* accepts literally yes/mod/no (case insensitive), but won't accept y/m/n. Why not accept y/m/n as well? -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-21 18:47:40
|
Bacula's unwillingness to erase or re-label an already-labelled tape is becoming a problem. I don't have a large pool of tapes, and I can only spare one for bacula testing. The console *really needs* a command to relabel a tape and treat it as a new tape. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-21 19:10:57
|
Hello Phil, Yes, this is the next big item on my list. I sent an email on this subject recently. In addition to tape labeling, a more nasty problem is that the database continues to grow unless you manually apply a retention period, which is a bit painful. For testing, the procedure for "relabeling" a tape is: In Console: delete media (type tape name to be deleted) (type "pretty please" without quotes) unmount (specify which drive) In another command window: mt -f /dev/xxxx rewind mt -f /dev/xxxx weof In Console: label ======== Another way to "relabel" a tape is to use the btape program: btape /dev/xxxx label (answer the questions) (it blasts over any existing label) In Console (possibly delete the media as described above) or use the add command to add the Volume to the Catalog without doing the label. ============ Now my question to you is what do you want the console relabel command to do? Do you really mean Recycle the tape (i.e. purge all Catalog File entries for that tape, but not the Media record) and then mark the tape recycle (so it gets reused)? Another form of relabel, would be to Purge ALL Catalog entries including the Media record, and then relabel the tape with a new Volume name. For me (and John Walker), what is important is that we have some fool-proof way of ensuring that an operator cannot accidentally overwrite a valid tape (unless the tape is defined as being Recyclable). On Sun, 2002-04-21 at 20:47, Phil Stracchino wrote: > Bacula's unwillingness to erase or re-label an already-labelled tape is > becoming a problem. I don't have a large pool of tapes, and I can only > spare one for bacula testing. The console *really needs* a command to > relabel a tape and treat it as a new tape. > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Phil S. <al...@ba...> - 2002-04-21 21:49:04
|
On Sun, Apr 21, 2002 at 09:10:36PM +0200, Kern Sibbald wrote: > ============ > Now my question to you is what do you want the console > relabel command to do? Do you really mean Recycle the > tape (i.e. purge all Catalog File entries for that > tape, but not the Media record) and then mark the > tape recycle (so it gets reused)? > > Another form of relabel, would be to Purge ALL > Catalog entries including the Media record, and > then relabel the tape with a new Volume name. > > For me (and John Walker), what is important is that > we have some fool-proof way of ensuring that an > operator cannot accidentally overwrite a valid > tape (unless the tape is defined as being Recyclable). Excellent points, and good questions. The first part is easy -- both Recycle and Relabel, as described above, have their place. As for the second .... well, as the saying goes, "It's hard to make anything foolproof, because fools are so ingenious." I suppose one way that this could be done is to permit *scheduled* recycling of tapes that have passed their retention date automatically, unless they hold "Archive" jobs, but make the *manual* recycle or relabel operation require a separate console tool -- you have console/gconsole for doing all regular tasks *except* manual recycling and relabelling, and a separate recycle/grecycle which ONLY does recycling and relabelling, and is the only tool that does manual recycling and relabelling. That probably provides about the best protection you're ever likely to be able to have against an operator accidentally overwriting a tape that's still in use, short of disallowing manual recycle/relabel altogether or requiring a nuclear-launch style of authorization like requiring two people to enter a recycle-authorization password simultaneously on two different machines. It won't protect against *malicious* destruction of backup tapes .... but heck, the fastest way to do that is a strong magnet anyway. Note: I'm not seriously suggesting nuclear-launch style auth. :-) -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-22 01:57:58
|
I've found the FileId problem on inserting MD5sums with MySQL. You're using mysql_insert_id() in cats/sql_create.c:db_create_file_record() to get the FileId for the File record that's just been created. This function is documented as being unreliable, and in particular, it is documented (see http://www.php.net/manual/en/function.mysql-insert-id.php) that if the auto_increment field is of type BIGINT (which File:FileId is), then mysql_insert_id() consistently returns incorrect values. The documented workaround is to perform a SELECT LAST_INSERT_ID() instead. I'm working on code to do this right now. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-22 05:58:46
Attachments:
sql-fileid-patch
|
On Sun, Apr 21, 2002 at 06:57:56PM -0700, Phil Stracchino wrote: > I've found the FileId problem on inserting MD5sums with MySQL. You're > using mysql_insert_id() in cats/sql_create.c:db_create_file_record() to > get the FileId for the File record that's just been created. This > function is documented as being unreliable, and in particular, it is > documented (see http://www.php.net/manual/en/function.mysql-insert-id.php) > that if the auto_increment field is of type BIGINT (which File:FileId is), > then mysql_insert_id() consistently returns incorrect values. > > The documented workaround is to perform a SELECT LAST_INSERT_ID() instead. > I'm working on code to do this right now. Turned out there were three problems. The first is that mysql_insert_id() was returning garbage. The second is that the garbage was being further trashed by using the wrong format string. (On this architecture at least, %ld appears to be correct for a unit32_t.) The third problem is that catalog_update() is called separately to write the file attributes and to add the MD5sum to the record, and nothing was being done to preserve the FileId between the two calls, so even if the format string *was* correct and mysql_insert_id() *did* work on a BIGINT field, ar.FileId would still be incorrect by the time catalog_update() is called the second time and calls db_add_MD5_to_file_record(), because it never gets initialized on the second call. (Yes, it *was* being used uninitialized, but the manner in which it was being used uninitialized was such that gcc didn't catch it.) The patch attached to this message fixes all three of these problems. I also note that get_attributes_and_put_in_catalog() in fd_cmds.c shares much common code with catalog_update(), and almost certainly suffers from the same problem. I have therefore applied the same corrections to get_attributes_and_put_in_catalog(). With these changes, I get correct FileId values out of MySQL, and I get MD5sums written into my File table. I *do not guarantee* that this change will not break functionality with SQLite, and I do not guarantee its portability to other architectures as written. It may need massaging to make it portable. Both of these need to be tested. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-22 07:44:46
|
Hello Phil, First, sorry for all the problems you are running into. Second, many thanks for digging into it. Could you try doing what I suggested in my previous email -- change FileId from BIGINT to INTEGER. I believe that it will solve all your problems. See my comments below: On Mon, 2002-04-22 at 07:58, Phil Stracchino wrote: > > The first is that mysql_insert_id() was returning garbage. This is unfortunate, but I'm sure you are right. Unfortunately, I don't think I can take your fix as is because it seems to me that it implements non-standard SQL. The final solution will be to move the code into the sql_insert_id() subroutine, which is specifically SQL dependent. Actually, for MySQL, sql_insert_id() may be a simple define, but that is easily changed to point to a MySQL specific subroutine. Until I can sort this out properly in an SQL independent way (not always so easy), I recommend changing FileId back to INTEGER and recreating your database. > > The second is that the garbage was being further trashed by using the > wrong format string. (On this architecture at least, %ld appears to be > correct for a unit32_t.) This surprises me A LOT!!!! %d on all but the most weird architectures is specifically designed for 32 bit integers. The problem with using ld is that on the Alpha native compiler, it assumes a 64 bit integer. The same is probably true on other architectures. > > The third problem is that catalog_update() is called separately to write > the file attributes and to add the MD5sum to the record, and nothing was > being done to preserve the FileId between the two calls, so even if the > format string *was* correct and mysql_insert_id() *did* work on a BIGINT > field, ar.FileId would still be incorrect by the time catalog_update() is > called the second time and calls db_add_MD5_to_file_record(), because it > never gets initialized on the second call. (Yes, it *was* being used > uninitialized, but the manner in which it was being used uninitialized was > such that gcc didn't catch it.) Yes, in looking at the code, you are correct. For me, it was working because it remained unchanged on the stack -- not very good!! However, we cannot solve the problem using static variables (temporarily is OK) because we have lots of threads running around and multiple jobs may be running at the same time. The solution is to move the information into the JCR, which is in effect the job "global context". I'll take a careful at what you did. Thanks again for finding this nasty bug. Best regards, Kern |
From: Phil S. <al...@ba...> - 2002-04-22 15:16:26
|
On Mon, Apr 22, 2002 at 09:44:30AM +0200, Kern Sibbald wrote: > Hello Phil, > > First, sorry for all the problems you are running > into. Second, many thanks for digging into it. > > Could you try doing what I suggested in my previous > email -- change FileId from BIGINT to INTEGER. I > believe that it will solve all your problems. > > See my comments below: > > > On Mon, 2002-04-22 at 07:58, Phil Stracchino wrote: > > > > The first is that mysql_insert_id() was returning garbage. > > This is unfortunate, but I'm sure you are right. Unfortunately, > I don't think I can take your fix as is because it seems to me > that it implements non-standard SQL. The final solution will be > to move the code into the sql_insert_id() subroutine, which is > specifically SQL dependent. Actually, for MySQL, sql_insert_id() > may be a simple define, but that is easily changed to point > to a MySQL specific subroutine. > > Until I can sort this out properly in an SQL independent way (not > always so easy), I recommend changing FileId back to INTEGER and > recreating your database. I'm not sure that solves the problem, and there's two reasons why. The first is that mysql_insert_id() is volatile across queries. It is reset by any subsequent query to the query that generates the auto_increment, regardless of whether the intervening query generates an auto_increment or not. So if there is *any query* between creating the file record and adding the MD5sum to it, mysql_insert_id() will return the wrong result. This means that if there's multiple jobs running at the same time, as you mention below, then using mysql_insert_id() will work or not work at random. The second is that, from what I can tell from the documentation, even in an INTEGER field, mysql_insert_id() is unreliable. It works only on the latest versions of MySQL, and it doesn't work reliably even then. It's not the case that it's known to work so long as you don't use BIGINT, but rather that if you use BIGINT, it's *guaranteed* not to work. > > The second is that the garbage was being further trashed by using the > > wrong format string. (On this architecture at least, %ld appears to be > > correct for a unit32_t.) > > This surprises me A LOT!!!! %d on all but the most weird architectures > is specifically designed for 32 bit integers. The problem with using > ld is that on the Alpha native compiler, it assumes a 64 bit integer. > The same is probably true on other architectures. Well, in the course of tracing this, I inserted debugging probes in various places to watch what was happening to FileId, and since you had said there was a question of the correct format, I had the probes dump FileId three times as %lld, %ld and %d. The correct value was dumped by %ld. However, I was trying to maximize the length of the variable since you noted that you had originally had FileId as a uint64_t, so I was using sscanf(%ld) and strtoul (I tried and compared both) to extract it in the first place. If I recall correctly, I tried %d and %lld in the sscanf as well, but got incorrect results from them. When I first started managing to get FileId values out (even though the values were incorrect) from mysql_insert_id(), those values appeared to be dumped correctly by %d. > > The third problem is that catalog_update() is called separately to write > > the file attributes and to add the MD5sum to the record, and nothing was > > being done to preserve the FileId between the two calls, so even if the > > format string *was* correct and mysql_insert_id() *did* work on a BIGINT > > field, ar.FileId would still be incorrect by the time catalog_update() is > > called the second time and calls db_add_MD5_to_file_record(), because it > > never gets initialized on the second call. (Yes, it *was* being used > > uninitialized, but the manner in which it was being used uninitialized was > > such that gcc didn't catch it.) > > Yes, in looking at the code, you are correct. For me, it was working > because it remained unchanged on the stack -- not very good!! However, > we cannot solve the problem using static variables (temporarily is OK) > because we have lots of threads running around and multiple jobs > may be running at the same time. The solution is to move > the information into the JCR, which is in effect the job "global > context". I'll take a careful at what you did. That would be a better solution, yes, but there is still the issue of getting a correct FileId in the first place, particularly if (as you point out) multiple jobs are running. I think the real answer to this problem may be to hold off on writing the file attribute record into the File table until the MD5sum of the stream is known, and write the complete record atomically in a single operation. This avoids the whole problem of having to retrieve the FileId of the last inserted record at all. What do you think about this approach? -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Phil S. <al...@ba...> - 2002-04-23 19:03:26
|
Kern, Is there a way to put commands such as "automount on", "autodisplay on", or "setdebug" in the console.conf file so that they execute automatically when starting the console? -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-23 19:33:16
|
Hello Phil, No, not without hard coding them. In my own 1.19 stream, I've added "autodisplay on" to the gnome-console but I'm not sure it works because I think (as perhaps with readline() turned on, it blocks rather than polling Bacula every 15 seconds as intended). I'm planning to add Tcl as scripting capability both to the console programs and internally to Bacula in the near future. This is needed for flexibility, and because I am getting really anxious to have some sort of regression script capability. Hopefully that will solve the problem you are having. If it really annoys you, we could write a small piece of code that finds ~.consolerc and feeds all the commands to the director at startup. I'm currently trying to get the 1.18 code into CVS -- having some ssh problems that I will work out. My current Bacula programming project is implementing automatic recycling of volumes but I won't have much time until this weekend. Best regards, Kern On Tue, 2002-04-23 at 21:03, Phil Stracchino wrote: > Kern, > > Is there a way to put commands such as "automount on", "autodisplay on", > or "setdebug" in the console.conf file so that they execute automatically > when starting the console? > > > -- > ********* Fight Back! It may not be just YOUR life at risk. ********* > phil stracchino :: al...@ba... :: hal...@so... > unix ronin :::: renaissance man :::: mystic zen biker geek > 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) > Linux Now! ...because friends don't let friends use Microsoft. > > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel |
From: Phil S. <al...@ba...> - 2002-04-23 23:14:08
|
On Tue, Apr 23, 2002 at 09:32:53PM +0200, Kern Sibbald wrote: > Hello Phil, > > No, not without hard coding them. In my own 1.19 > stream, I've added "autodisplay on" to the gnome-console > but I'm not sure it works because I think (as perhaps > with readline() turned on, it blocks rather than > polling Bacula every 15 seconds as intended). > > I'm planning to add Tcl as scripting capability both > to the console programs and internally to Bacula in > the near future. This is needed for flexibility, and > because I am getting really anxious to have some sort > of regression script capability. > Hopefully that will solve the problem you are having. > If it really annoys you, we could write a small > piece of code that finds ~.consolerc and feeds all > the commands to the director at startup. I'm not finding it annoying particularly, I just wondered if there was a way to do it. I had noticed the way autodisplay blocks, though. There actually used to be a smiliar problem in cicb, and it took pretty extensive code changes to fix it. As for Tcl, the CICB code uses it extensively ... they keep changing the Tcl APIs though. I have some experience working with it, but not enough to know offhand how to add support in from scratch. -- ********* Fight Back! It may not be just YOUR life at risk. ********* phil stracchino :: al...@ba... :: hal...@so... unix ronin :::: renaissance man :::: mystic zen biker geek 2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold) Linux Now! ...because friends don't let friends use Microsoft. |
From: Kern S. <ke...@si...> - 2002-04-24 06:33:07
|
On Wed, 2002-04-24 at 01:14, Phil Stracchino wrote: > On Tue, Apr 23, 2002 at 09:32:53PM +0200, Kern Sibbald wrote: > > Hello Phil, > > > > No, not without hard coding them. In my own 1.19 > > stream, I've added "autodisplay on" to the gnome-console > > but I'm not sure it works because I think (as perhaps > > with readline() turned on, it blocks rather than > > polling Bacula every 15 seconds as intended). > > > > I'm planning to add Tcl as scripting capability both > > to the console programs and internally to Bacula in > > the near future. This is needed for flexibility, and > > because I am getting really anxious to have some sort > > of regression script capability. > > Hopefully that will solve the problem you are having. > > If it really annoys you, we could write a small > > piece of code that finds ~.consolerc and feeds all > > the commands to the director at startup. > > I'm not finding it annoying particularly, I just wondered if there was a > way to do it. I had noticed the way autodisplay blocks, though. There > actually used to be a smiliar problem in cicb, and it took pretty > extensive code changes to fix it. For Console using readline(), I'm not sure what the solution is, but in any case, it isn't too serious. For gnome-console, it is really simple, I just need to set a timer. I'll do that shortly. Last night I succeeded in getting 1.18 into the SourceForge CVS (at least I think), and I did a checkout. Now I just need to merge in my 1.19 changes (not too many yet) and see if I can make commit work. > > As for Tcl, the CICB code uses it extensively ... they keep changing the > Tcl APIs though. I have some experience working with it, but not enough > to know offhand how to add support in from scratch. I fiddled with Tcl for a while creating an apcupsd graphic display, so I don't think it will be a big problem. At the time I gave up on Tcl/Tk when I realized that they didn't have a combobox (drop down box). In this case, I'm not very interested in the Tk (graphical) part. Best regards, Kern |