Thread: SF.net SVN: safekeep:[784] safekeep/trunk (Page 2)

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Revision: 784
          http://safekeep.svn.sourceforge.net/safekeep/?rev=784&view=rev
Author:   fcrawford
Date:     2011-12-30 09:27:22 +0000 (Fri, 30 Dec 2011)
Log Message:
-----------
Add the ability to have the script file on either the client or the
server.  Note that it is still only run on the client, just that for
a server located one, a temporary copy is sent to the client to run.
At present, it is designed around passing scripts, not binaries, as it
passes the file line by line.

The protocol version has been bumped to 1.2, due to an addition to rund
the script after it is copied.

Modified Paths:
--------------
    safekeep/trunk/doc/safekeep.backup.txt
    safekeep/trunk/safekeep

Modified: safekeep/trunk/doc/safekeep.backup.txt
===================================================================

--- safekeep/trunk/doc/safekeep.backup.txt	2011-12-30 08:50:25 UTC (rev 783)
+++ safekeep/trunk/doc/safekeep.backup.txt	2011-12-30 09:27:22 UTC (rev 784)
@@ -47,7 +47,7 @@
     <snapshot device="/path/to/volume" size="500M" />
 
     <!-- location of a script to be executed at different stages of the run --> 
-    <script path="/path/to/script" />
+    <script path="server:/path/to/script" />
 
   </setup>
 
@@ -248,15 +248,21 @@
 	tag.
 	Optional for a `<snapshot>` element.
 
-/backup/setup/script/@path::
-       Execute the script specified path on the client at certain steps
-       of the backup process.
-       This script is executed with three arguments:
-       - Backup id (/backup/@id)
-       - Backup step
-       - Backup root directory (valid after creation of a snapshot)
-       See the `CLIENT SCRIPT` section for more information.
-       Mandatory for a `<script>` element.
+/backup/setup/script/@location::
+	Execute the script specified path on the client at certain steps
+	of the backup process.
+	This script is executed with three arguments:
+	- Backup id (/backup/@id)
+	- Backup step
+	- Backup root directory (valid after creation of a snapshot)
+	The `location` optionally consists of an optional `host` and
+	a mandatory `path`, separated by a ":", where the host part is
+	either "client" or "server".  If no host part is specified then
+	it is first looked for on the server, and if not found, then is
+	looked for on the client.  If it not found on either, then a
+	warning is raised.
+	See the `CLIENT SCRIPT` section for more information.
+	Mandatory for a `<script>` element.
 
 /backup/data/@exclude-devices::
        One of "true" or "false". If "true", the dump file will
@@ -343,6 +349,15 @@
 `safekeep(1)` support the optional execution of a script or program
 on the client system at different steps during execution of the backup.
 
+The script may be located on either the server or the client.  If it is
+located on the server, then the file is copied to the client into a
+temporary directory before execution.  Note that this directory is located
+where ever the system normally creates temporary files, and it is possible
+that the execution of scripts are disallowed.  In that case it is recommended
+that a client based script is used.  In addition, the script is copied from
+the server on a line by line basis, and so it is not suitable to pass binary
+files.
+
 Note: specification of a script which does not exist is not considered an
 error, and is treated as the same as not specifying a script.  However, if
 the specified path does match a file or directory, the security tests listed

Modified: safekeep/trunk/safekeep
===================================================================
--- safekeep/trunk/safekeep	2011-12-30 08:50:25 UTC (rev 783)
+++ safekeep/trunk/safekeep	2011-12-30 09:27:22 UTC (rev 784)
@@ -68,7 +68,7 @@
 default_bandwidth = {}
 cmd = "<Missing>"
 
-PROTOCOL = "1.1"
+PROTOCOL = "1.2"
 VERSION = "1.3.3"
 VEBOSITY_BY_CLASS = {'DBG': 3, 'INFO': 2, 'WARN': 1, 'ERR': 0}
 
@@ -431,6 +431,8 @@
         script_el = setup_el[0].getElementsByTagName('script')
         if len(script_el) == 1:
             script = script_el[0].getAttribute('path')
+            if not ':' in script and os.path.isfile(script):
+                script = 'server:' + script
         elif len(script_el) > 1:
             raise ConfigException('Can not have more than one setup script element')
 
@@ -771,6 +773,19 @@
     finally:
         dom.unlink()
 
+def do_client_remote_script(script_file, cfg, cmd):
+    (cfg_cmd, server_file, cnt_str) = cmd.split(':', 2)
+    debug("do_client_remote_script: %s -> %s: cnt_str = %s" % (server_file, cfg['script'], cnt_str.strip()))
+    try:
+        for i in xrange(int(cnt_str)):
+            line = sys.stdin.readline()
+            if not line: raise Exception('Unexpected end of file')
+            script_file.write(line)
+    finally:
+        script_file.close()
+
+    os.chmod(cfg['script'], stat.S_IXUSR | stat.S_IRUSR)
+
 def do_client_setup(cfg):
     debug('Do setup of %s' % cfg['host'])
 
@@ -892,7 +907,7 @@
                 warn('Can not tear down snapshot: %s' % device)
             scrubbed = True
 
-        # Now cleanup any safekeep directories still hanging around
+        # Now cleanup any safekeep directories and script files still hanging around
 
         debug("Cleaning up remaining safekeep directories")
         if os.path.isdir('/mnt'):
@@ -905,6 +920,33 @@
                     except OSError, e:
                         warn('Failed to remove: %s: %s' % (mountpoint, e))
 
+        debug("Cleaning up remaining safekeep script files")
+        script_dir = tempfile.gettempdir()
+        if os.path.isdir(script_dir):
+            for ent in os.listdir(script_dir):
+                if fnmatch.fnmatch(ent, 'safekeep-[0-9][0-9]*-') and not fnmatch.fnmatch(ent, 'safekeep-%d-' % current_pid):
+                    script_file = '%s/%s' % (script_dir, ent)
+                    if os.path.isdir(script_file):
+                        temp_dir = script_file
+                        for ent in os.listdir(temp_dir):
+                            script_file = '%s/%s' % (temp_dir, ent)
+                            info("Removing script file %s" % script_file)
+                            try:
+                                os.remove('%s' % script_file)
+                            except OSError, e:
+                                warn('Failed to remove: %s: %s' % (script_file, e))
+                        info("Removing script directory %s" % temp_dir)
+                        try:
+                            os.rmdir(temp_dir)
+                        except OSError, e:
+                            warn('Failed to remove: %s: %s' % (temp_dir, e))
+                    else:
+                        info("Removing script file %s" % script_file)
+                        try:
+                            os.remove('%s' % script_file)
+                        except OSError, e:
+                            warn('Failed to remove: %s: %s' % (script_file, e))
+
         if not scrubbed:
             info('No cleanup required')
 
@@ -921,6 +963,8 @@
     bdir = '/'
     cfg = do_client_config_parse('<backup/>', 'def')
     ex = None
+    script_file = None
+    script_dir = None
     try:
         while True:
             try:
@@ -930,6 +974,29 @@
                     send('OK %s, %s' % (PROTOCOL, VERSION))
                 elif line.startswith('CONFIG'):
                     cfg = do_client_config(line)
+                    if ':' in cfg['script']:
+                        (script_loc, script) = cfg['script'].split(':', 1)
+                    else:
+                        (script_loc, script) = ('client', cfg['script'])
+                    if script_loc == 'server':
+                        if not script_dir:
+                            script_dir = tempfile.mkdtemp(prefix="safekeep-%d-" % current_pid)
+                        script = os.path.basename(script)
+                        (fd, cfg['script']) = tempfile.mkstemp(prefix="%s-" % script, dir=script_dir)
+                        script_file = os.fdopen(fd, 'w')
+                        send('OK %s' % cfg['script'])
+                    elif script_loc == 'client':
+                        cfg['script'] = script
+                        ret = client_side_script('STARTUP', cfg, bdir)
+                        if ret:
+                            send('ERROR Client-side setup script failed: %s' % ret)
+                        else:
+                            send('OK')
+                    else:
+                        warn('Unknown script location %s for script %s' % (script_loc, script))
+                        send('OK')
+                elif line.startswith('SCRIPT'):
+                    do_client_remote_script(script_file, cfg, line)
                     ret = client_side_script('STARTUP', cfg, bdir)
                     if ret:
                         send('ERROR Client-side setup script failed: %s' % ret)
@@ -961,6 +1028,14 @@
     finally:
         if should_cleanup:
             do_client_cleanup(cfg, bdir)
+        if script_file:
+            if not script_file.closed: script_file.close()
+            os.remove(cfg['script'])
+        if script_dir:
+            try:
+                os.rmdir(script_dir)
+            except OSError, e:
+                warn('Failed to remove: %s: %s' % (script_dir, e))
 
     if ex:
         send('TRACEBACK ' + str(ex)  + '>>>' + stacktrace().replace('\n', '###'))
@@ -1156,7 +1231,17 @@
             cin.write('CONFIG: %d: %s\n' % (len(cfg['text'].splitlines()), id))
             cin.write(cfg['text'] + '\n')
             cin.flush()
-            do_server_getanswer(cout)
+            remote_script = do_server_getanswer(cout)
+            if cfg['script'].startswith('server:') and remote_script:
+                local_script = cfg['script'].split(':', 1)[1]
+                debug("Transferring script: %s -> %s" % (local_script, remote_script))
+                fscript = open(local_script)
+                lines = fscript.readlines()
+                fscript.close()
+                cin.write('SCRIPT: %s: %d\n' % (local_script, len(lines)))
+                cin.writelines(lines)
+                cin.flush()
+                do_server_getanswer(cout)
             if cleanup:
                 cleaned_up = False
                 cin.write('SCRUB\n')

This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.





Thread: SF.net SVN: safekeep:[784] safekeep/trunk (Page 2)

safekeep-commit