Menu
▾
▴
[Bacula-commits] SF.net SVN: bacula: [4183] branches/Branch-2.0/bacula
|
From: <ke...@us...> - 2007-02-15 18:57:56
|
Revision: 4183
http://bacula.svn.sourceforge.net/bacula/?rev=4183&view=rev
Author: kerns
Date: 2007-02-15 10:57:55 -0800 (Thu, 15 Feb 2007)
Log Message:
-----------
15Feb07
kes Add quick disconnect FD code from 2.1.4 to 2.0.3. This code
causes the SD to release the FD as soon as the FD has sent
all the data to the SD. After that the SD will do any final
despooling (data and attributes) that are necessary. This
allows laptops to disconnect much quicker from the network
after a backup.
13Feb07
kes Apply Eric's ClientRunScriptAfter patch to 2.0 and 2.1.
11Feb07
kes Optimize the use of the database a bit in the Status dir command.
Only open it when needed, ensure that if any previous database
was opened, it is closed.
10Feb07
kes Modify dbcheck to handle orphaned JobMedia, Path, Filename,
and File records in 300K chunks to be more efficient. This
idea came from Juan Luis Frances (if I remember right).
09Feb07
kes Update projects list.
Modified Paths:
--------------
branches/Branch-2.0/bacula/projects
branches/Branch-2.0/bacula/src/dird/catreq.c
branches/Branch-2.0/bacula/src/dird/job.c
branches/Branch-2.0/bacula/src/dird/msgchan.c
branches/Branch-2.0/bacula/src/dird/protos.h
branches/Branch-2.0/bacula/src/dird/ua_cmds.c
branches/Branch-2.0/bacula/src/dird/ua_output.c
branches/Branch-2.0/bacula/src/dird/ua_status.c
branches/Branch-2.0/bacula/src/dird/ua_update.c
branches/Branch-2.0/bacula/src/dird/verify.c
branches/Branch-2.0/bacula/src/filed/job.c
branches/Branch-2.0/bacula/src/filed/verify.c
branches/Branch-2.0/bacula/src/jcr.h
branches/Branch-2.0/bacula/src/stored/acquire.c
branches/Branch-2.0/bacula/src/stored/append.c
branches/Branch-2.0/bacula/src/stored/fd_cmds.c
branches/Branch-2.0/bacula/src/stored/protos.h
branches/Branch-2.0/bacula/src/tools/dbcheck.c
branches/Branch-2.0/bacula/src/version.h
branches/Branch-2.0/bacula/technotes-2.0
Modified: branches/Branch-2.0/bacula/projects
===================================================================
--- branches/Branch-2.0/bacula/projects 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/projects 2007-02-15 18:57:55 UTC (rev 4183)
@@ -4,6 +4,11 @@
Status updated 26 January 2007
After re-ordering in vote priority
+Items Completed:
+Item: 18 Quick release of FD-SD connection after backup.
+Item: 40 Include JobID in spool file name
+Item: 25 Implement huge exclude list support using dlist
+
Summary:
Item: 1 Accurate restoration of renamed/deleted files
Item: 2 Implement a Bacula GUI/management tool.
@@ -22,14 +27,14 @@
Item: 15 Multiple threads in file daemon for the same job
Item: 16 Add Plug-ins to the FileSet Include statements.
Item: 17 Restore only file attributes (permissions, ACL, owner, group...)
-Item: 18 Quick release of FD-SD connection after backup.
+Item: 18* Quick release of FD-SD connection after backup.
Item: 19 Implement a Python interface to the Bacula catalog.
Item: 20 Archive data
Item: 21 Split documentation
Item: 22 Implement support for stacking arbitrary stream filters, sinks.
Item: 23 Implement from-client and to-client on restore command line.
Item: 24 Add an override in Schedule for Pools based on backup types.
-Item: 25 Implement huge exclude list support using hashing.
+Item: 25* Implement huge exclude list support using hashing.
Item: 26 Implement more Python events in Bacula.
Item: 27 Incorporation of XACML2/SAML2 parsing
Item: 28 Filesystem watch triggered backup.
@@ -44,7 +49,7 @@
Item: 37 Add an item to the restore option where you can select a pool
Item: 38 Include timestamp of job launch in "stat clients" output
Item: 39 Message mailing based on backup types
-Item: 40 Include JobID in spool file name
+Item: 40* Include JobID in spool file name
Item 1: Accurate restoration of renamed/deleted files
@@ -82,7 +87,7 @@
Item 2: Implement a Bacula GUI/management tool.
Origin: Kern
Date: 28 October 2005
- Status:
+ Status: In progress
What: Implement a Bacula console, and management tools
probably using Qt3 and C++.
@@ -569,6 +574,7 @@
Notes: If the file is here, we skip restore and we change rights.
If the file isn't here, we can create an empty one and apply
rights or do nothing.
+
Item 18: Quick release of FD-SD connection after backup.
Origin: Frank Volf (frank at deze dot org)
Date: 17 November 2005
@@ -746,10 +752,12 @@
backups, then the Full job would use the proper Storage device, which
has more capacity (i.e. a 8TB tape library.
-Item 25: Implement huge exclude list support using hashing.
+Item 25: Implement huge exclude list support using hashing (dlists).
Date: 28 October 2005
Origin: Kern
- Status:
+ Status: Done in 2.1.2 but was done with dlists (doubly linked lists
+ since hashing will not help. The huge list also supports
+ large include lists).
What: Allow users to specify very large exclude list (currently
more than about 1000 files is too many).
Modified: branches/Branch-2.0/bacula/src/dird/catreq.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/catreq.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/catreq.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -280,7 +280,7 @@
mr.VolWriteTime = sdmr.VolWriteTime;
mr.VolParts = sdmr.VolParts;
bstrncpy(mr.VolStatus, sdmr.VolStatus, sizeof(mr.VolStatus));
- if (jcr->wstore->StorageId) {
+ if (jcr->wstore && jcr->wstore->StorageId) {
mr.StorageId = jcr->wstore->StorageId;
}
Modified: branches/Branch-2.0/bacula/src/dird/job.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/job.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/job.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -156,6 +156,10 @@
* Create Job record
*/
init_jcr_job_record(jcr);
+ if (!get_or_create_client_record(jcr)) {
+ goto bail_out;
+ }
+
if (!db_create_job_record(jcr, jcr->db, &jcr->jr)) {
Jmsg(jcr, M_FATAL, 0, "%s", db_strerror(jcr->db));
goto bail_out;
@@ -164,10 +168,6 @@
Dmsg4(100, "Created job record JobId=%d Name=%s Type=%c Level=%c\n",
jcr->JobId, jcr->Job, jcr->jr.JobType, jcr->jr.JobLevel);
- if (!get_or_create_client_record(jcr)) {
- goto bail_out;
- }
-
generate_daemon_event(jcr, "JobStart");
if (job_canceled(jcr)) {
@@ -554,7 +554,7 @@
jcr->JobStatus);
}
Dmsg3(800, "MaxWaitTime result: %scancel JCR %p (%s)\n",
- cancel ? "" : "do not ", jcr, jcr->job);
+ cancel ? "" : "do not ", jcr, jcr->Job);
#endif
return cancel;
}
@@ -602,7 +602,7 @@
}
Dmsg3(200, "MaxRunTime result: %scancel JCR %p (%s)\n",
- cancel ? "" : "do not ", jcr, jcr->job);
+ cancel ? "" : "do not ", jcr, jcr->Job);
#endif
return true;
}
@@ -886,6 +886,10 @@
db_close_database(jcr, jcr->db);
jcr->db = NULL;
}
+ if (jcr->db_batch) {
+ db_close_database(jcr, jcr->db_batch);
+ jcr->db_batch = NULL;
+ }
if (jcr->stime) {
Dmsg0(200, "Free JCR stime\n");
free_pool_memory(jcr->stime);
Modified: branches/Branch-2.0/bacula/src/dird/msgchan.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/msgchan.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/msgchan.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -302,7 +302,7 @@
* Start a thread to handle Storage daemon messages and
* Catalog requests.
*/
-int start_storage_daemon_message_thread(JCR *jcr)
+bool start_storage_daemon_message_thread(JCR *jcr)
{
int status;
pthread_t thid;
@@ -318,9 +318,12 @@
/* Wait for thread to start */
while (jcr->SD_msg_chan == 0) {
bmicrosleep(0, 50);
+ if (job_canceled(jcr) || jcr->sd_msg_thread_done) {
+ return false;
+ }
}
Dmsg1(100, "SD msg_thread started. use=%d\n", jcr->use_count());
- return 1;
+ return true;
}
extern "C" void msg_thread_cleanup(void *arg)
Modified: branches/Branch-2.0/bacula/src/dird/protos.h
===================================================================
--- branches/Branch-2.0/bacula/src/dird/protos.h 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/protos.h 2007-02-15 18:57:55 UTC (rev 4183)
@@ -141,7 +141,7 @@
extern bool connect_to_storage_daemon(JCR *jcr, int retry_interval,
int max_retry_time, int verbose);
extern bool start_storage_daemon_job(JCR *jcr, alist *rstore, alist *wstore);
-extern int start_storage_daemon_message_thread(JCR *jcr);
+extern bool start_storage_daemon_message_thread(JCR *jcr);
extern int bget_dirmsg(BSOCK *bs);
extern void wait_for_storage_daemon_termination(JCR *jcr);
@@ -195,9 +195,12 @@
int get_num_drives_from_SD(UAContext *ua);
void update_slots(UAContext *ua);
+/* ua_update.c */
+void update_vol_pool(UAContext *ua, char *val, MEDIA_DBR *mr, POOL_DBR *opr);
+
/* ua_output.c */
void prtit(void *ctx, const char *msg);
-int complete_jcr_for_job(JCR *jcr, JOB *job, POOL *pool);
+bool complete_jcr_for_job(JCR *jcr, JOB *job, POOL *pool);
RUN *find_next_run(RUN *run, JOB *job, time_t &runtime, int ndays);
/* ua_restore.c */
Modified: branches/Branch-2.0/bacula/src/dird/ua_cmds.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/ua_cmds.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/ua_cmds.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -384,10 +384,6 @@
JCR *jcr = NULL;
char JobName[MAX_NAME_LENGTH];
- if (!open_client_db(ua)) {
- return 1;
- }
-
for (i=1; i<ua->argc; i++) {
if (strcasecmp(ua->argk[i], NT_("jobid")) == 0) {
uint32_t JobId;
@@ -434,28 +430,38 @@
* throw up a list and ask the user to select one.
*/
char buf[1000];
+ int tjobs = 0; /* total # number jobs */
/* Count Jobs running */
foreach_jcr(jcr) {
if (jcr->JobId == 0) { /* this is us */
continue;
}
+ tjobs++; /* count of all jobs */
if (!acl_access_ok(ua, Job_ACL, jcr->job->name())) {
continue; /* skip not authorized */
}
- njobs++;
+ njobs++; /* count of authorized jobs */
}
endeach_jcr(jcr);
- if (njobs == 0) {
- bsendmsg(ua, _("No Jobs running.\n"));
+ if (njobs == 0) { /* no authorized */
+ if (tjobs == 0) {
+ bsendmsg(ua, _("No Jobs running.\n"));
+ } else {
+ bsendmsg(ua, _("None of your jobs are running.\n"));
+ }
return 1;
}
+
start_prompt(ua, _("Select Job:\n"));
foreach_jcr(jcr) {
char ed1[50];
if (jcr->JobId == 0) { /* this is us */
continue;
}
+ if (!acl_access_ok(ua, Job_ACL, jcr->job->name())) {
+ continue; /* skip not authorized */
+ }
bsnprintf(buf, sizeof(buf), _("JobId=%s Job=%s"), edit_int64(jcr->JobId, ed1), jcr->Job);
add_prompt(ua, buf);
}
@@ -472,7 +478,7 @@
sscanf(buf, "JobId=%d Job=%127s", &njobs, JobName);
jcr = get_jcr_by_full_name(JobName);
if (!jcr) {
- bsendmsg(ua, _("Job %s not found.\n"), JobName);
+ bsendmsg(ua, _("Job \"%s\" not found.\n"), JobName);
return 1;
}
}
Modified: branches/Branch-2.0/bacula/src/dird/ua_output.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/ua_output.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/ua_output.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -1,16 +1,7 @@
/*
- *
- * Bacula Director -- User Agent Output Commands
- * I.e. messages, listing database, showing resources, ...
- *
- * Kern Sibbald, September MM
- *
- * Version $Id$
- */
-/*
Bacula® - The Network Backup Solution
- Copyright (C) 2000-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -34,6 +25,15 @@
(FSFE), Fiduciary Program, Sumatrastrasse 25, 8006 Zürich,
Switzerland, email:ft...@fs....
*/
+/*
+ *
+ * Bacula Director -- User Agent Output Commands
+ * I.e. messages, listing database, showing resources, ...
+ *
+ * Kern Sibbald, September MM
+ *
+ * Version $Id$
+ */
#include "bacula.h"
#include "dird.h"
@@ -602,7 +602,7 @@
* Fill in the remaining fields of the jcr as if it
* is going to run the job.
*/
-int complete_jcr_for_job(JCR *jcr, JOB *job, POOL *pool)
+bool complete_jcr_for_job(JCR *jcr, JOB *job, POOL *pool)
{
POOL_DBR pr;
@@ -611,6 +611,10 @@
if (pool) {
jcr->pool = pool; /* override */
}
+ if (jcr->db) {
+ db_close_database(jcr, jcr->db);
+ jcr->db = NULL;
+ }
jcr->db = jcr->db=db_init_database(jcr, jcr->catalog->db_name, jcr->catalog->db_user,
jcr->catalog->db_password, jcr->catalog->db_address,
jcr->catalog->db_port, jcr->catalog->db_socket,
@@ -620,8 +624,10 @@
jcr->catalog->db_name);
if (jcr->db) {
Jmsg(jcr, M_FATAL, 0, "%s", db_strerror(jcr->db));
+ db_close_database(jcr, jcr->db);
+ jcr->db = NULL;
}
- return 0;
+ return false;
}
bstrncpy(pr.Name, jcr->pool->hdr.name, sizeof(pr.Name));
while (!db_get_pool_record(jcr, jcr->db, &pr)) { /* get by Name */
@@ -633,13 +639,13 @@
db_close_database(jcr, jcr->db);
jcr->db = NULL;
}
- return 0;
+ return false;
} else {
Jmsg(jcr, M_INFO, 0, _("Pool %s created in database.\n"), pr.Name);
}
}
jcr->jr.PoolId = pr.PoolId;
- return 1;
+ return true;
}
Modified: branches/Branch-2.0/bacula/src/dird/ua_status.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/ua_status.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/ua_status.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -1,15 +1,7 @@
/*
- *
- * Bacula Director -- User Agent Status Command
- *
- * Kern Sibbald, August MMI
- *
- * Version $Id$
- */
-/*
Bacula® - The Network Backup Solution
- Copyright (C) 2001-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2001-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -33,6 +25,14 @@
(FSFE), Fiduciary Program, Sumatrastrasse 25, 8006 Zürich,
Switzerland, email:ft...@fs....
*/
+/*
+ *
+ * Bacula Director -- User Agent Status Command
+ *
+ * Kern Sibbald, August MMI
+ *
+ * Version $Id$
+ */
#include "bacula.h"
@@ -100,9 +100,6 @@
CLIENT *client;
int item, i;
- if (!open_client_db(ua)) {
- return 1;
- }
Dmsg1(20, "status:%s:\n", cmd);
for (i=1; i<ua->argc; i++) {
Modified: branches/Branch-2.0/bacula/src/dird/ua_update.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/ua_update.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/ua_update.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -290,7 +290,7 @@
}
/* Modify the Pool in which this Volume is located */
-static void update_vol_pool(UAContext *ua, char *val, MEDIA_DBR *mr, POOL_DBR *opr)
+void update_vol_pool(UAContext *ua, char *val, MEDIA_DBR *mr, POOL_DBR *opr)
{
POOL_DBR pr;
POOLMEM *query;
Modified: branches/Branch-2.0/bacula/src/dird/verify.c
===================================================================
--- branches/Branch-2.0/bacula/src/dird/verify.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/dird/verify.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -756,7 +756,7 @@
}
if (!jcr->fn_printed) {
Jmsg(jcr, M_INFO, 0, "\n");
- Jmsg(jcr, M_INFO, 0, _("The following files are missing:\n"));
+ Jmsg(jcr, M_INFO, 0, _("The following files are in the Catalog but not on disk:\n"));
jcr->fn_printed = true;
}
Jmsg(jcr, M_INFO, 0, " %s%s\n", row[0]?row[0]:"", row[1]?row[1]:"");
Modified: branches/Branch-2.0/bacula/src/filed/job.c
===================================================================
--- branches/Branch-2.0/bacula/src/filed/job.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/filed/job.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -244,16 +244,14 @@
}
}
- if (!jcr->runscript_after) {
- jcr->runscript_after=1;
- run_scripts(jcr, jcr->RunScripts, "ClientAfterJob");
- }
-
/* Inform Storage daemon that we are done */
if (jcr->store_bsock) {
bnet_sig(jcr->store_bsock, BNET_TERMINATE);
}
+ /* Run the after job */
+ run_scripts(jcr, jcr->RunScripts, "ClientAfterJob");
+
generate_daemon_event(jcr, "JobEnd");
dequeue_messages(jcr); /* send any queued messages */
@@ -1387,17 +1385,9 @@
bnet_suppress_error_messages(sd, 1);
bget_msg(sd); /* Read final response from append_data */
Dmsg0(110, "Error in blast_data.\n");
- /* run shortly after end of data transmission */
- run_scripts(jcr, jcr->RunScripts, "ClientAfterJob");
- jcr->runscript_after=1;
-
} else {
set_jcr_job_status(jcr, JS_Terminated);
- /* run shortly after end of data transmission */
- run_scripts(jcr, jcr->RunScripts, "ClientAfterJob");
- jcr->runscript_after=1;
-
if (jcr->JobStatus != JS_Terminated) {
bnet_suppress_error_messages(sd, 1);
goto cleanup; /* bail out now */
Modified: branches/Branch-2.0/bacula/src/filed/verify.c
===================================================================
--- branches/Branch-2.0/bacula/src/filed/verify.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/filed/verify.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -1,15 +1,7 @@
/*
- * Bacula File Daemon verify.c Verify files.
- *
- * Kern Sibbald, October MM
- *
- * Version $Id$
- *
- */
-/*
Bacula® - The Network Backup Solution
- Copyright (C) 2000-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -33,6 +25,14 @@
(FSFE), Fiduciary Program, Sumatrastrasse 25, 8006 Zürich,
Switzerland, email:ft...@fs....
*/
+/*
+ * Bacula File Daemon verify.c Verify files.
+ *
+ * Kern Sibbald, October MM
+ *
+ * Version $Id$
+ *
+ */
#include "bacula.h"
#include "filed.h"
@@ -102,6 +102,7 @@
Dmsg2(30, "FT_LNK saving: %s -> %s\n", ff_pkt->fname, ff_pkt->link);
break;
case FT_DIRBEGIN:
+ jcr->num_files_examined--; /* correct file count */
return 1; /* ignored */
case FT_DIREND:
Dmsg1(30, "FT_DIR saving: %s\n", ff_pkt->fname);
@@ -145,7 +146,8 @@
return 1;
case FT_NORECURSE:
Jmsg(jcr, M_SKIPPED, 1, _(" Recursion turned off. Directory skipped: %s\n"), ff_pkt->fname);
- return 1;
+ ff_pkt->type = FT_DIREND; /* directory entry was backed up */
+ break;
case FT_NOFSCHG:
Jmsg(jcr, M_SKIPPED, 1, _(" File system change prohibited. Directory skipped: %s\n"), ff_pkt->fname);
return 1;
Modified: branches/Branch-2.0/bacula/src/jcr.h
===================================================================
--- branches/Branch-2.0/bacula/src/jcr.h 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/jcr.h 2007-02-15 18:57:55 UTC (rev 4183)
@@ -184,6 +184,7 @@
bool cached_attribute; /* set if attribute is cached */
POOLMEM *attr; /* Attribute string from SD */
B_DB *db; /* database pointer */
+ B_DB *db_batch; /* database pointer for batch insert */
ATTR_DBR *ar; /* DB attribute record */
/* Daemon specific part of JCR */
@@ -294,7 +295,6 @@
int32_t pki_session_encoded_size; /* Size of DER-encoded pki_session */
POOLMEM *crypto_buf; /* Encryption/Decryption buffer */
DIRRES* director; /* Director resource */
- bool runscript_after; /* Don't run After Script twice */
#endif /* FILE_DAEMON */
Modified: branches/Branch-2.0/bacula/src/stored/acquire.c
===================================================================
--- branches/Branch-2.0/bacula/src/stored/acquire.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/stored/acquire.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -463,9 +463,9 @@
if (dev->can_read()) {
dev->clear_read(); /* clear read bit */
+ Dmsg0(100, "dir_update_vol_info. Release0\n");
+// dir_update_volume_info(dcr, false); /* send Volume info to Director */
- /******FIXME**** send read volume usage statistics to director */
-
} else if (dev->num_writers > 0) {
/*
* Note if WEOT is set, we are at the end of the tape
Modified: branches/Branch-2.0/bacula/src/stored/append.c
===================================================================
--- branches/Branch-2.0/bacula/src/stored/append.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/stored/append.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -7,7 +7,7 @@
/*
Bacula® - The Network Backup Solution
- Copyright (C) 2000-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -38,6 +38,7 @@
/* Responses sent to the File daemon */
static char OK_data[] = "3000 OK data\n";
+static char OK_append[] = "3000 OK append data\n";
/* Forward referenced functions */
@@ -261,6 +262,14 @@
}
}
+ /* Create Job status for end of session label */
+ set_jcr_job_status(jcr, ok?JS_Terminated:JS_ErrorTerminated);
+
+ /* Terminate connection with FD */
+ bnet_fsend(ds, OK_append);
+ do_fd_commands(jcr); /* finish dialog with FD */
+
+
time_t job_elapsed = time(NULL) - jcr->run_time;
if (job_elapsed <= 0) {
@@ -271,8 +280,6 @@
job_elapsed / 3600, job_elapsed % 3600 / 60, job_elapsed % 60,
edit_uint64_with_suffix(jcr->JobBytes / job_elapsed, ec));
- /* Create Job status for end of session label */
- set_jcr_job_status(jcr, ok?JS_Terminated:JS_ErrorTerminated);
Dmsg1(200, "Write EOS label JobStatus=%c\n", jcr->JobStatus);
@@ -303,6 +310,8 @@
Pmsg0(000, _("NULL Volume name. This shouldn't happen!!!\n"));
}
+
+
if (!ok) {
discard_data_spool(dcr);
} else {
Modified: branches/Branch-2.0/bacula/src/stored/fd_cmds.c
===================================================================
--- branches/Branch-2.0/bacula/src/stored/fd_cmds.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/stored/fd_cmds.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -15,7 +15,7 @@
/*
Bacula® - The Network Backup Solution
- Copyright (C) 2000-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -95,7 +95,6 @@
static char OK_end[] = "3000 OK end\n";
static char OK_close[] = "3000 OK close Status = %d\n";
static char OK_open[] = "3000 OK open ticket = %d\n";
-static char OK_append[] = "3000 OK append data\n";
static char ERROR_append[] = "3903 Error append data\n";
static char OK_bootstrap[] = "3000 OK bootstrap\n";
static char ERROR_bootstrap[] = "3904 Error bootstrap\n";
@@ -107,6 +106,7 @@
/*
* Run a File daemon Job -- File daemon already authorized
+ * Director sends us this command.
*
* Basic task here is:
* - Read a command from the File daemon
@@ -115,14 +115,9 @@
*/
void run_job(JCR *jcr)
{
- int i;
- bool found, quit;
- BSOCK *fd = jcr->file_bsock;
BSOCK *dir = jcr->dir_bsock;
char ec1[30];
-
- fd->jcr = jcr;
dir->jcr = jcr;
Dmsg1(120, "Start run Job=%s\n", jcr->Job);
bnet_fsend(dir, Job_start, jcr->Job);
@@ -130,6 +125,27 @@
jcr->run_time = jcr->start_time;
set_jcr_job_status(jcr, JS_Running);
dir_send_job_status(jcr); /* update director */
+ do_fd_commands(jcr);
+ jcr->end_time = time(NULL);
+ dequeue_messages(jcr); /* send any queued messages */
+ set_jcr_job_status(jcr, JS_Terminated);
+ generate_daemon_event(jcr, "JobEnd");
+ bnet_fsend(dir, Job_end, jcr->Job, jcr->JobStatus, jcr->JobFiles,
+ edit_uint64(jcr->JobBytes, ec1));
+ bnet_sig(dir, BNET_EOD); /* send EOD to Director daemon */
+ return;
+}
+
+/*
+ * Now talk to the FD and do what he says
+ */
+void do_fd_commands(JCR *jcr)
+{
+ int i;
+ bool found, quit;
+ BSOCK *fd = jcr->file_bsock;
+
+ fd->jcr = jcr;
for (quit=false; !quit;) {
int stat;
@@ -160,17 +176,8 @@
}
}
bnet_sig(fd, BNET_TERMINATE); /* signal to FD job is done */
- jcr->end_time = time(NULL);
- dequeue_messages(jcr); /* send any queued messages */
- set_jcr_job_status(jcr, JS_Terminated);
- generate_daemon_event(jcr, "JobEnd");
- bnet_fsend(dir, Job_end, jcr->Job, jcr->JobStatus, jcr->JobFiles,
- edit_uint64(jcr->JobBytes, ec1));
- bnet_sig(dir, BNET_EOD); /* send EOD to Director daemon */
- return;
}
-
/*
* Append Data command
* Open Data Channel and receive Data for archiving
@@ -185,7 +192,7 @@
Dmsg1(110, "<bfiled: %s", fd->msg);
jcr->JobType = JT_BACKUP;
if (do_append_data(jcr)) {
- return bnet_fsend(fd, OK_append);
+ return true;
} else {
bnet_suppress_error_messages(fd, 1); /* ignore errors at this point */
bnet_fsend(fd, ERROR_append);
Modified: branches/Branch-2.0/bacula/src/stored/protos.h
===================================================================
--- branches/Branch-2.0/bacula/src/stored/protos.h 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/stored/protos.h 2007-02-15 18:57:55 UTC (rev 4183)
@@ -6,7 +6,7 @@
/*
Bacula® - The Network Backup Solution
- Copyright (C) 2000-2006 Free Software Foundation Europe e.V.
+ Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
The main author of Bacula is Kern Sibbald, with contributions from
many others, a complete list can be found in the file AUTHORS.
@@ -144,7 +144,8 @@
/* From fd_cmds.c */
void run_job(JCR *jcr);
-bool get_bootstrap_file(JCR *jcr, BSOCK *bsock);
+bool get_bootstrap_file(JCR *jcr, BSOCK *bsock);
+void do_fd_commands(JCR *jcr);
/* From job.c */
void stored_free_jcr(JCR *jcr);
Modified: branches/Branch-2.0/bacula/src/tools/dbcheck.c
===================================================================
--- branches/Branch-2.0/bacula/src/tools/dbcheck.c 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/tools/dbcheck.c 2007-02-15 18:57:55 UTC (rev 4183)
@@ -702,136 +702,156 @@
static void eliminate_orphaned_jobmedia_records()
{
- const char *query;
+ const char *query = "SELECT JobMedia.JobMediaId,Job.JobId FROM JobMedia "
+ "LEFT OUTER JOIN Job ON (JobMedia.JobId=Job.JobId) "
+ "WHERE Job.JobId IS NULL LIMIT 300000";
printf(_("Checking for orphaned JobMedia entries.\n"));
- query = "SELECT JobMedia.JobMediaId,Job.JobId FROM JobMedia "
- "LEFT OUTER JOIN Job ON (JobMedia.JobId=Job.JobId) "
- "WHERE Job.JobId IS NULL";
if (!make_id_list(query, &id_list)) {
exit(1);
}
- printf(_("Found %d orphaned JobMedia records.\n"), id_list.num_ids);
- if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
- for (int i=0; i < id_list.num_ids; i++) {
- char ed1[50];
- bsnprintf(buf, sizeof(buf),
+ /* Loop doing 300000 at a time */
+ while (id_list.num_ids != 0) {
+ printf(_("Found %d orphaned JobMedia records.\n"), id_list.num_ids);
+ if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
+ for (int i=0; i < id_list.num_ids; i++) {
+ char ed1[50];
+ bsnprintf(buf, sizeof(buf),
"SELECT JobMedia.JobMediaId,JobMedia.JobId,Media.VolumeName FROM JobMedia,Media "
"WHERE JobMedia.JobMediaId=%s AND Media.MediaId=JobMedia.MediaId",
- edit_int64(id_list.Id[i], ed1));
- if (!db_sql_query(db, buf, print_jobmedia_handler, NULL)) {
- printf("%s\n", db_strerror(db));
+ edit_int64(id_list.Id[i], ed1));
+ if (!db_sql_query(db, buf, print_jobmedia_handler, NULL)) {
+ printf("%s\n", db_strerror(db));
+ }
}
}
- }
- if (quit) {
- return;
- }
+ if (quit) {
+ return;
+ }
- if (fix && id_list.num_ids > 0) {
- printf(_("Deleting %d orphaned JobMedia records.\n"), id_list.num_ids);
- delete_id_list("DELETE FROM JobMedia WHERE JobMediaId=%s", &id_list);
+ if (fix && id_list.num_ids > 0) {
+ printf(_("Deleting %d orphaned JobMedia records.\n"), id_list.num_ids);
+ delete_id_list("DELETE FROM JobMedia WHERE JobMediaId=%s", &id_list);
+ }
+ if (!make_id_list(query, &id_list)) {
+ exit(1);
+ }
}
}
static void eliminate_orphaned_file_records()
{
- const char *query;
+ const char *query = "SELECT File.FileId,Job.JobId FROM File "
+ "LEFT OUTER JOIN Job ON (File.JobId=Job.JobId) "
+ "WHERE Job.JobId IS NULL LIMIT 300000";
printf(_("Checking for orphaned File entries. This may take some time!\n"));
- query = "SELECT File.FileId,Job.JobId FROM File "
- "LEFT OUTER JOIN Job ON (File.JobId=Job.JobId) "
- "WHERE Job.JobId IS NULL";
if (verbose > 1) {
printf("%s\n", query);
}
if (!make_id_list(query, &id_list)) {
exit(1);
}
- printf(_("Found %d orphaned File records.\n"), id_list.num_ids);
- if (name_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
- for (int i=0; i < id_list.num_ids; i++) {
- char ed1[50];
- bsnprintf(buf, sizeof(buf),
+ /* Loop doing 300000 at a time */
+ while (id_list.num_ids != 0) {
+ printf(_("Found %d orphaned File records.\n"), id_list.num_ids);
+ if (name_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
+ for (int i=0; i < id_list.num_ids; i++) {
+ char ed1[50];
+ bsnprintf(buf, sizeof(buf),
"SELECT File.FileId,File.JobId,Filename.Name FROM File,Filename "
"WHERE File.FileId=%s AND File.FilenameId=Filename.FilenameId",
- edit_int64(id_list.Id[i], ed1));
- if (!db_sql_query(db, buf, print_file_handler, NULL)) {
- printf("%s\n", db_strerror(db));
+ edit_int64(id_list.Id[i], ed1));
+ if (!db_sql_query(db, buf, print_file_handler, NULL)) {
+ printf("%s\n", db_strerror(db));
+ }
}
}
+ if (quit) {
+ return;
+ }
+ if (fix && id_list.num_ids > 0) {
+ printf(_("Deleting %d orphaned File records.\n"), id_list.num_ids);
+ delete_id_list("DELETE FROM File WHERE FileId=%s", &id_list);
+ }
+ if (!make_id_list(query, &id_list)) {
+ exit(1);
+ }
}
- if (quit) {
- return;
- }
- if (fix && id_list.num_ids > 0) {
- printf(_("Deleting %d orphaned File records.\n"), id_list.num_ids);
- delete_id_list("DELETE FROM File WHERE FileId=%s", &id_list);
- }
}
static void eliminate_orphaned_path_records()
{
- const char *query;
+ const char *query = "SELECT DISTINCT Path.PathId,File.PathId FROM Path "
+ "LEFT OUTER JOIN File ON (Path.PathId=File.PathId) "
+ "WHERE File.PathId IS NULL LIMIT 300000";
printf(_("Checking for orphaned Path entries. This may take some time!\n"));
- query = "SELECT DISTINCT Path.PathId,File.PathId FROM Path "
- "LEFT OUTER JOIN File ON (Path.PathId=File.PathId) "
- "WHERE File.PathId IS NULL";
if (verbose > 1) {
printf("%s\n", query);
}
if (!make_id_list(query, &id_list)) {
exit(1);
}
- printf(_("Found %d orphaned Path records.\n"), id_list.num_ids);
- if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
- for (int i=0; i < id_list.num_ids; i++) {
- char ed1[50];
- bsnprintf(buf, sizeof(buf), "SELECT Path FROM Path WHERE PathId=%s",
- edit_int64(id_list.Id[i], ed1));
- db_sql_query(db, buf, print_name_handler, NULL);
+ /* Loop doing 300000 at a time */
+ while (id_list.num_ids != 0) {
+ printf(_("Found %d orphaned Path records.\n"), id_list.num_ids);
+ if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
+ for (int i=0; i < id_list.num_ids; i++) {
+ char ed1[50];
+ bsnprintf(buf, sizeof(buf), "SELECT Path FROM Path WHERE PathId=%s",
+ edit_int64(id_list.Id[i], ed1));
+ db_sql_query(db, buf, print_name_handler, NULL);
+ }
}
+ if (quit) {
+ return;
+ }
+ if (fix && id_list.num_ids > 0) {
+ printf(_("Deleting %d orphaned Path records.\n"), id_list.num_ids);
+ delete_id_list("DELETE FROM Path WHERE PathId=%s", &id_list);
+ }
+ if (!make_id_list(query, &id_list)) {
+ exit(1);
+ }
}
- if (quit) {
- return;
- }
- if (fix && id_list.num_ids > 0) {
- printf(_("Deleting %d orphaned Path records.\n"), id_list.num_ids);
- delete_id_list("DELETE FROM Path WHERE PathId=%s", &id_list);
- }
}
static void eliminate_orphaned_filename_records()
{
- const char *query;
+ const char *query = "SELECT Filename.FilenameId,File.FilenameId FROM Filename "
+ "LEFT OUTER JOIN File ON (Filename.FilenameId=File.FilenameId) "
+ "WHERE File.FilenameId IS NULL LIMIT 300000";
printf(_("Checking for orphaned Filename entries. This may take some time!\n"));
- query = "SELECT Filename.FilenameId,File.FilenameId FROM Filename "
- "LEFT OUTER JOIN File ON (Filename.FilenameId=File.FilenameId) "
- "WHERE File.FilenameId IS NULL";
if (verbose > 1) {
printf("%s\n", query);
}
if (!make_id_list(query, &id_list)) {
exit(1);
}
- printf(_("Found %d orphaned Filename records.\n"), id_list.num_ids);
- if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
- for (int i=0; i < id_list.num_ids; i++) {
- char ed1[50];
- bsnprintf(buf, sizeof(buf), "SELECT Name FROM Filename WHERE FilenameId=%s",
- edit_int64(id_list.Id[i], ed1));
- db_sql_query(db, buf, print_name_handler, NULL);
+ /* Loop doing 300000 at a time */
+ while (id_list.num_ids != 0) {
+ printf(_("Found %d orphaned Filename records.\n"), id_list.num_ids);
+ if (id_list.num_ids && verbose && yes_no(_("Print them? (yes/no): "))) {
+ for (int i=0; i < id_list.num_ids; i++) {
+ char ed1[50];
+ bsnprintf(buf, sizeof(buf), "SELECT Name FROM Filename WHERE FilenameId=%s",
+ edit_int64(id_list.Id[i], ed1));
+ db_sql_query(db, buf, print_name_handler, NULL);
+ }
}
+ if (quit) {
+ return;
+ }
+ if (fix && id_list.num_ids > 0) {
+ printf(_("Deleting %d orphaned Filename records.\n"), id_list.num_ids);
+ delete_id_list("DELETE FROM Filename WHERE FilenameId=%s", &id_list);
+ }
+ if (!make_id_list(query, &id_list)) {
+ exit(1);
+ }
}
- if (quit) {
- return;
- }
- if (fix && id_list.num_ids > 0) {
- printf(_("Deleting %d orphaned Filename records.\n"), id_list.num_ids);
- delete_id_list("DELETE FROM Filename WHERE FilenameId=%s", &id_list);
- }
}
static void eliminate_orphaned_fileset_records()
Modified: branches/Branch-2.0/bacula/src/version.h
===================================================================
--- branches/Branch-2.0/bacula/src/version.h 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/src/version.h 2007-02-15 18:57:55 UTC (rev 4183)
@@ -4,8 +4,8 @@
#undef VERSION
#define VERSION "2.0.3"
-#define BDATE "06 February 2007"
-#define LSMDATE "06Feb07"
+#define BDATE "15 February 2007"
+#define LSMDATE "15Feb07"
#define PROG_COPYRIGHT "Copyright (C) %d-2007 Free Software Foundation Europe e.V.\n"
#define BYEAR "2007" /* year for copyright messages in progs */
@@ -53,8 +53,12 @@
#define USE_BSNPRINTF 1
-/* Disk file seeking is now controled by CAP_POSITIONBLOCKS */
+/* Turn on the following flag to enable batch attribute inserts
+ * in the catalog. This gives a large speedup.
+ */
+/* #define HAVE_BATCH_FILE_INSERT 1 */
+
/* Debug flags not normally turned on */
/* #define TRACE_JCR_CHAIN 1 */
Modified: branches/Branch-2.0/bacula/technotes-2.0
===================================================================
--- branches/Branch-2.0/bacula/technotes-2.0 2007-02-15 18:53:37 UTC (rev 4182)
+++ branches/Branch-2.0/bacula/technotes-2.0 2007-02-15 18:57:55 UTC (rev 4183)
@@ -1,6 +1,25 @@
Technical notes on version 2.0
General:
+15Feb07
+kes Add quick disconnect FD code from 2.1.4 to 2.0.3. This code
+ causes the SD to release the FD as soon as the FD has sent
+ all the data to the SD. After that the SD will do any final
+ despooling (data and attributes) that are necessary. This
+ allows laptops to disconnect much quicker from the network
+ after a backup.
+13Feb07
+kes Apply Eric's ClientRunScriptAfter patch to 2.0 and 2.1.
+11Feb07
+kes Optimize the use of the database a bit in the Status dir command.
+ Only open it when needed, ensure that if any previous database
+ was opened, it is closed.
+10Feb07
+kes Modify dbcheck to handle orphaned JobMedia, Path, Filename,
+ and File records in 300K chunks to be more efficient. This
+ idea came from Juan Luis Frances (if I remember right).
+09Feb07
+kes Update projects list.
08Feb07
kes Fix dird/ua_cmds.c so that a cancel command checks if the
console is authorized to cancel the job. This fixes bug
@@ -46,6 +65,9 @@
rblist and add them to be built in src/lib. Include some new
methods written by Rudolf Cejka that make the code more readable
(hides some of the ugly casting).
+26Jan07 (back port)
+kes Implement item #12 on project list -- quick release of FD by
+ the SD. This is noted in more detail above.
Version 2.0.2 released: 30 January 2007
28Jan08
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|