Menu
▾
▴
SF.net SVN: safekeep:[725] safekeep/tags
|
From: <di...@us...> - 2010-11-23 04:13:11
|
Revision: 725
http://safekeep.svn.sourceforge.net/safekeep/?rev=725&view=rev
Author: dimi
Date: 2010-11-23 04:13:03 +0000 (Tue, 23 Nov 2010)
Log Message:
-----------
Tag safekeep 1.3.1
Added Paths:
-----------
safekeep/tags/Release-safekeep-1_3_1/
safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE
safekeep/tags/Release-safekeep-1_3_1/Makefile
safekeep/tags/Release-safekeep-1_3_1/safekeep
safekeep/tags/Release-safekeep-1_3_1/safekeep.spec.in
Removed Paths:
-------------
safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE
safekeep/tags/Release-safekeep-1_3_1/Makefile
safekeep/tags/Release-safekeep-1_3_1/safekeep
safekeep/tags/Release-safekeep-1_3_1/safekeep.spec.in
Deleted: safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE
===================================================================
--- safekeep/trunk/ANNOUNCE 2010-11-22 04:32:48 UTC (rev 713)
+++ safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE 2010-11-23 04:13:03 UTC (rev 725)
@@ -1,41 +0,0 @@
-This is release 1.3.0 of SafeKeep, a centralized and easy to use
-backup application that combines the best features of a mirror
-and an incremental backup.
-
-What's new in this release:
- - Add ionice(1) support for limiting IO on server side.
- - Allow control of nice value on both server and client
- - Fix password handling when dumping MySQL databases.
- - Fix MySQL dumps when passing a username (for newer versions).
- - Handle correctly Unicode strings, such as localized DB names.
- - Fix a bug when dumping a specific Postgresql database.
- - Prepare the code for newer Python versions
- - Try to remove a snapshot up to 10 times in a row to workaround
- silly udev bug: https://bugzilla.redhat.com/show_bug.cgi?id=577798
- - Multiple internal cleanups, and minor bugs fixes.
-
-Thanks go to Frank Crawford and Bertrand Lecervoisier for providing
-patches and reports for the problems addressed in this release.
-
-Sources and binaries are available from the following locations:
-
- - RedHat EL/CentOS 3,4,5,6 Fedora 8,9,10,11,12,13,14:
- http://prdownloads.sourceforge.net/safekeep/safekeep-common-1.3.0-1.noarch.rpm
- http://prdownloads.sourceforge.net/safekeep/safekeep-client-1.3.0-1.noarch.rpm
- http://prdownloads.sourceforge.net/safekeep/safekeep-server-1.3.0-1.noarch.rpm
-
- - Ubuntu Edgy, Dapper, Breezy, Hardy, Karmic, Lucid, Maverick, and Natty:
- http://prdownloads.sourceforge.net/safekeep/safekeep-common_1.3.0_all.deb
- http://prdownloads.sourceforge.net/safekeep/safekeep-client_1.3.0_all.deb
- http://prdownloads.sourceforge.net/safekeep/safekeep-server_1.3.0_all.deb
-
- - Source:
- http://prdownloads.sourceforge.net/safekeep/safekeep-1.3.0.tar.gz
- http://prdownloads.sourceforge.net/safekeep/safekeep-1.3.0-1.src.rpm
-
-To find out more about the project visit on our website:
- http://safekeep.sourceforge.net
-
---
-Dimi Paun <di...@la...>
-Lattica, Inc.
Copied: safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE (from rev 723, safekeep/trunk/ANNOUNCE)
===================================================================
--- safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE (rev 0)
+++ safekeep/tags/Release-safekeep-1_3_1/ANNOUNCE 2010-11-23 04:13:03 UTC (rev 725)
@@ -0,0 +1,30 @@
+This is release 1.3.1 of SafeKeep, a centralized and easy to use
+backup application that combines the best features of a mirror
+and an incremental backup.
+
+What's new in this release:
+ - Fix a few serios errors preventing safekeep from running.
+ - Better reporting of client messages and exceptions.
+
+Sources and binaries are available from the following locations:
+
+ - RedHat EL/CentOS 3,4,5,6 Fedora 8,9,10,11,12,13,14:
+ http://prdownloads.sourceforge.net/safekeep/safekeep-common-1.3.1-1.noarch.rpm
+ http://prdownloads.sourceforge.net/safekeep/safekeep-client-1.3.1-1.noarch.rpm
+ http://prdownloads.sourceforge.net/safekeep/safekeep-server-1.3.1-1.noarch.rpm
+
+ - Ubuntu Edgy, Dapper, Breezy, Hardy, Karmic, Lucid, Maverick, and Natty:
+ http://prdownloads.sourceforge.net/safekeep/safekeep-common_1.3.1_all.deb
+ http://prdownloads.sourceforge.net/safekeep/safekeep-client_1.3.1_all.deb
+ http://prdownloads.sourceforge.net/safekeep/safekeep-server_1.3.1_all.deb
+
+ - Source:
+ http://prdownloads.sourceforge.net/safekeep/safekeep-1.3.1.tar.gz
+ http://prdownloads.sourceforge.net/safekeep/safekeep-1.3.1-1.src.rpm
+
+To find out more about the project visit on our website:
+ http://safekeep.sourceforge.net
+
+--
+Dimi Paun <di...@la...>
+Lattica, Inc.
Deleted: safekeep/tags/Release-safekeep-1_3_1/Makefile
===================================================================
--- safekeep/trunk/Makefile 2010-11-22 04:32:48 UTC (rev 713)
+++ safekeep/tags/Release-safekeep-1_3_1/Makefile 2010-11-23 04:13:03 UTC (rev 725)
@@ -1,170 +0,0 @@
-name := safekeep
-timestamp := $(shell LANG=C date)
-timestamp_svn := $(shell date -u -d '$(timestamp)' '+%Y%m%dT%H%MZ')
-version_num := $(shell grep 'VERSION *=' safekeep | sed s'/[^"]*"\([^"].*\)".*/\1/')
-version_ts := $(shell date -u -d '$(timestamp)' '+%Y%m%d%H%M')
-version := $(version_num)
-release := 1
-releasename := $(name)-$(version)
-snapshotname:= $(name)-$(version).$(version_ts)
-tagname := $(shell echo Release-$(releasename) | tr . _)
-dirname := $(shell basename $(PWD))
-rpmroot := $(shell grep '%_topdir' ~/.rpmmacros 2>/dev/null | sed 's/^[^ \t]*[ \t]*//')
-svnroot := $(shell LANG=C svn info 2>/dev/null | grep Root | cut -c 18-)
-sf_login := dimi,$(name)@frs.sourceforge.net
-sf_dir := /home/frs/project/s/sa/$(name)
-releasedir := releases
-repo_srv := root@ulysses
-repo_dir := /var/www/repos/lattica
-webroot := ../../website/trunk/WebContent/
-MAN_TXT := doc/safekeep.txt doc/safekeep.conf.txt doc/safekeep.backup.txt
-DOC_MAN := doc/safekeep.1 doc/safekeep.conf.5 doc/safekeep.backup.5
-DOC_HTML := $(patsubst %.txt,%.html,$(MAN_TXT))
-
-
-all: help
-
-help:
- @echo "Targets:"
- @echo " help Displays this message"
- @echo " info Displays package information (version, etc.)"
- @echo " install Installs safekeep and the online documentation"
- @echo " docs Builds all documentation formats"
- @echo " web Updates the website to the latest documentation"
- @echo " build Builds everything needed for an installation"
- @echo " tar Builds snapshot source distribution"
- @echo " deb Builds snapshot binary and source DEBs"
- @echo " rpm Buidls snapshot binary and source RPMs"
- @echo " tag Tags the source for release"
- @echo " dist Builds release source distribution"
- @echo " distdeb Builds release binary and source DEBs"
- @echo " distrpm Buidls release binary and source RPMs"
- @echo " deploy Deployes the release RPMs to Lattica's repos"
- @echo " check Invokes a quick local test for SafeKeep"
- @echo " test Invokes a comprehensive remote test for SafeKeep"
- @echo " clean Cleans up the source tree"
-
-info:
- @echo "Release Name = $(releasename)"
- @echo "Snapshot Name = $(snapshotname)"
- @echo "Version = $(version)"
- @echo "Timestamp = $(timestamp)"
- @echo "Tag = $(tagname)"
- @echo "RPM Root = $(rpmroot)"
- @echo "SVN Root = $(svnroot)"
-
-
-build: docs
-
-release: check-info commit-release dist distrpm
-
-deploy: deploy-lattica deploy-sf
-
-commit-release:
- svn ci -m "Release $(version) (tagged as $(tagname))"
-
-tag:
- svn cp -m "Tag safekeep $(version)" . $(svnroot)/safekeep/tags/$(tagname)
-
-check-info: info
- @echo -n 'Is this information correct? (yes/No) '
- @read x; if [ "$$x" != "yes" ]; then exit 1; fi
-
-web: html
- cp doc/*.html $(webroot)
- cd $(webroot); svn ci -m "Update man pages on website to latest as of $(timestamp)"
-
-docs: html man
-
-html: $(DOC_HTML)
-
-man: $(DOC_MAN)
-
-%.html: %.txt
- asciidoc --unsafe -b html4 -d manpage -f doc/asciidoc.conf $<
-
-%.1 %.5: %.xml
- xmlto -o doc -m doc/callouts.xsl man $<
-
-%.xml: %.txt
- asciidoc --unsafe -b docbook -d manpage -f doc/asciidoc.conf $<
-
-$(DOC_HTML) $(DOC_MAN): doc/asciidoc.conf
-
-changelog:
- svn log -v --xml | svn2log.py -D 0 -u doc/users
-
-install:
- install -m 755 safekeep "/usr/bin/"
- install -d -m 755 "/etc/safekeep/backup.d/"
- install -m 755 safekeep.conf "/etc/safekeep/"
- install -m 755 doc/safekeep.1 "/usr/share/man/man1/"
- install -m 755 doc/safekeep.conf.5 "/usr/share/man/man5/"
- install -m 755 doc/safekeep.backup.5 "/usr/share/man/man5/"
-
-tar:
- svn export -r {'$(timestamp_svn)'} $(svnroot)/safekeep/trunk $(snapshotname)
- cat $(snapshotname)/$(name).spec.in | sed 's/^%define version.*/%define version $(version).$(version_ts)/' > $(snapshotname)/$(name).spec
- cat $(snapshotname)/debian/changelog.in | sed 's/^safekeep.*/safekeep ($(version).$(version_ts)) unstable; urgency=low/' > $(snapshotname)/debian/changelog
- tar cz -f $(snapshotname).tar.gz $(snapshotname)
- rm -rf $(snapshotname)
-
-deb: tar
- tar xz -C /tmp -f $(snapshotname).tar.gz
- rm -rf $(snapshotname).tar.gz
- cd /tmp/$(snapshotname) && debuild --check-dirname-regex 'safekeep(-.*)?'
-
-rpm: tar
- rpmbuild -ta $(snapshotname).tar.gz
- mv $(rpmroot)/SRPMS/$(snapshotname)-$(release)*.src.rpm .
- mv $(rpmroot)/RPMS/noarch/$(name)-*-$(version).$(version_ts)-$(release)*.noarch.rpm .
-
-dist:
- svn export $(svnroot)/safekeep/tags/$(tagname) $(releasename)
- cat $(releasename)/$(name).spec.in | sed 's/^%define version.*/%define version $(version)/' > $(releasename)/$(name).spec
- cat $(releasename)/debian/changelog.in | sed 's/^safekeep.*/safekeep ($(version)) unstable; urgency=low/' > $(releasename)/debian/changelog
- mkdir -p $(releasedir); tar cz -f $(releasedir)/$(releasename).tar.gz $(releasename)
- cd $(releasename); make docs
- rm -rf $(releasename)
-
-distdeb: dist
- tar xz -C /tmp -f $(releasedir)/$(releasename).tar.gz
- rm -rf $(releasedir)/$(releasename).tar.gz
- cd /tmp/$(releasename) && debuild --check-dirname-regex 'safekeep(-.*)?'
- mv /tmp/$(name)-*_$(version)_all.deb $(releasedir)
-
-distrpm: dist
- rpmbuild -ta $(releasedir)/$(releasename).tar.gz
- mv $(rpmroot)/SRPMS/$(releasename)-$(release)*.src.rpm $(releasedir)
- mv $(rpmroot)/RPMS/noarch/$(name)-*-$(version)-$(release)*.noarch.rpm $(releasedir)
- rpm --addsign $(releasedir)/$(releasename)-$(release)*.src.rpm $(releasedir)/$(name)-*-$(version)-$(release)*.noarch.rpm
-
-deploy-src-to-sf:
- echo -e "cd $(sf_dir)\nmkdir $(version)" | sftp -b- $(sf_login)
- scp $(releasedir)/$(releasename).tar.gz $(sf_login):$(sf_dir)/$(version)
- scp ANNOUNCE $(sf_login):$(sf_dir)/$(version)/README.txt
-
-deploy-rpms-to-sf:
- scp $(releasedir)/$(releasename)-$(release)*.src.rpm $(releasedir)/$(name)-*-$(version)-$(release)*.noarch.rpm $(sf_login):$(sf_dir)/$(version)
-
-deploy-debs-to-sf:
- scp $(releasedir)/$(name)-*_$(version)_all.deb $(sf_login):$(sf_dir)/$(version)
-
-deploy-lattica:
- scp $(releasedir)/${name}{,-common,-client,-server}-${version}-*.rpm ${repo_srv}:${repo_dir}/upload
- ssh ${repo_srv} "cd ${repo_dir}; ./deploy-rpms.sh upload/${name}-*${version}-*.rpm"
-
-deploy-sf:
- scp releases/${name}{-${version}.tar.gz,{,-common,-client,-server}-${version}-*.rpm} frs.sourceforge.net:uploads
-
-check:
- safekeep-test --local
-
-test:
- safekeep-test --remote
-
-clean:
- rm -f {.,doc,debian}/*~ *.py[co]
- rm -f $(name).spec debian/changelog
- rm -f doc/*.xml doc/*.html doc/*.[15]
- rm -f safekeep-*[.]20[01][0-9][01][0-9][0-3][0-9][012][0-9][0-5][0-9]*
Copied: safekeep/tags/Release-safekeep-1_3_1/Makefile (from rev 719, safekeep/trunk/Makefile)
===================================================================
--- safekeep/tags/Release-safekeep-1_3_1/Makefile (rev 0)
+++ safekeep/tags/Release-safekeep-1_3_1/Makefile 2010-11-23 04:13:03 UTC (rev 725)
@@ -0,0 +1,191 @@
+name := safekeep
+timestamp := $(shell LANG=C date)
+timestamp_svn := $(shell date -u -d '$(timestamp)' '+%Y%m%dT%H%MZ')
+version_num := $(shell grep 'VERSION *=' safekeep | sed s'/[^"]*"\([^"].*\)".*/\1/')
+version_ts := $(shell date -u -d '$(timestamp)' '+%Y%m%d%H%M')
+version := $(version_num)
+release := 1
+releasename := $(name)-$(version)
+snapshotname:= $(name)-$(version).$(version_ts)
+tagname := $(shell echo Release-$(releasename) | tr . _)
+dirname := $(shell basename $(PWD))
+rpmroot := $(shell grep '%_topdir' ~/.rpmmacros 2>/dev/null | sed 's/^[^ \t]*[ \t]*//')
+svnroot := $(shell LANG=C svn info 2>/dev/null | grep Root | cut -c 18-)
+deb_box := 192.168.3.202
+rpm_box := 192.168.3.242
+sf_login := dimi,$(name)@frs.sourceforge.net
+sf_dir := /home/frs/project/s/sa/$(name)/$(name)
+releasedir := releases
+repo_srv := root@ulysses
+repo_dir := /var/www/repos/lattica
+webroot := ../../website/trunk/WebContent/
+MAN_TXT := doc/safekeep.txt doc/safekeep.conf.txt doc/safekeep.backup.txt
+DOC_MAN := doc/safekeep.1 doc/safekeep.conf.5 doc/safekeep.backup.5
+DOC_HTML := $(patsubst %.txt,%.html,$(MAN_TXT))
+
+
+all: help
+
+help:
+ @echo "Targets:"
+ @echo " help Displays this message"
+ @echo " info Displays package information (version, etc.)"
+ @echo " install Installs safekeep and the online documentation"
+ @echo " docs Builds all documentation formats"
+ @echo " web Updates the website to the latest documentation"
+ @echo " build Builds everything needed for an installation"
+ @echo " tar Builds snapshot source distribution"
+ @echo " deb Builds snapshot binary and source DEBs"
+ @echo " rpm Buidls snapshot binary and source RPMs"
+ @echo " tag Tags the source for release"
+ @echo " dist Builds release source distribution"
+ @echo " distdeb Builds release binary and source DEBs"
+ @echo " distrpm Buidls release binary and source RPMs"
+ @echo " deploy Deployes the release RPMs to Lattica's repos"
+ @echo " check Invokes a quick local test for SafeKeep"
+ @echo " test Invokes a comprehensive remote test for SafeKeep"
+ @echo " clean Cleans up the source tree"
+
+info:
+ @echo "Release Name = $(releasename)"
+ @echo "Snapshot Name = $(snapshotname)"
+ @echo "Version = $(version)"
+ @echo "Timestamp = $(timestamp)"
+ @echo "Tag = $(tagname)"
+ @echo "RPM Root = $(rpmroot)"
+ @echo "SVN Root = $(svnroot)"
+
+
+build: docs
+
+release: check-info commit-release dist distrpm
+
+deploy: deploy-lattica deploy-sf
+
+commit-release:
+ svn ci -m "Release $(version) (tagged as $(tagname))"
+
+tag:
+ svn cp -m "Tag safekeep $(version)" . $(svnroot)/safekeep/tags/$(tagname)
+
+check-info: info
+ @echo -n 'Is this information correct? (yes/No) '
+ @read x; if [ "$$x" != "yes" ]; then exit 1; fi
+
+web: html
+ cp doc/*.html $(webroot)
+ cd $(webroot); svn ci -m "Update man pages on website to latest as of $(timestamp)"
+
+docs: html man
+
+html: $(DOC_HTML)
+
+man: $(DOC_MAN)
+
+%.html: %.txt
+ asciidoc --unsafe -b html4 -d manpage -f doc/asciidoc.conf $<
+
+%.1 %.5: %.xml
+ xmlto -o doc -m doc/callouts.xsl man $<
+
+%.xml: %.txt
+ asciidoc --unsafe -b docbook -d manpage -f doc/asciidoc.conf $<
+
+$(DOC_HTML) $(DOC_MAN): doc/asciidoc.conf
+
+changelog:
+ svn log -v --xml | svn2log.py -D 0 -u doc/users
+
+install:
+ install -m 755 safekeep "/usr/bin/"
+ install -d -m 755 "/etc/safekeep/backup.d/"
+ install -m 755 safekeep.conf "/etc/safekeep/"
+ install -m 755 doc/safekeep.1 "/usr/share/man/man1/"
+ install -m 755 doc/safekeep.conf.5 "/usr/share/man/man5/"
+ install -m 755 doc/safekeep.backup.5 "/usr/share/man/man5/"
+
+tar:
+ svn export -r {'$(timestamp_svn)'} $(svnroot)/safekeep/trunk $(snapshotname)
+ cat $(snapshotname)/$(name).spec.in | sed 's/^%define version.*/%define version $(version).$(version_ts)/' > $(snapshotname)/$(name).spec
+ cat $(snapshotname)/debian/changelog.in | sed 's/^safekeep.*/safekeep ($(version).$(version_ts)) unstable; urgency=low/' > $(snapshotname)/debian/changelog
+ tar cz -f $(snapshotname).tar.gz $(snapshotname)
+ rm -rf $(snapshotname)
+
+deb: tar
+ tar xz -C /tmp -f $(snapshotname).tar.gz
+ rm -rf $(snapshotname).tar.gz
+ cd /tmp/$(snapshotname) && debuild --check-dirname-regex 'safekeep(-.*)?'
+
+rpm: tar
+ rpmbuild -ta $(snapshotname).tar.gz
+ mv $(rpmroot)/SRPMS/$(snapshotname)-$(release)*.src.rpm .
+ mv $(rpmroot)/RPMS/noarch/$(name)-*-$(version).$(version_ts)-$(release)*.noarch.rpm .
+
+dist: $(releasedir)/$(releasename).tar.gz
+
+$(releasedir)/$(releasename).tar.gz:
+ svn export $(svnroot)/safekeep/tags/$(tagname) $(releasename)
+ cat $(releasename)/$(name).spec.in | sed 's/^%define version.*/%define version $(version)/' > $(releasename)/$(name).spec
+ cat $(releasename)/debian/changelog.in | sed 's/^safekeep.*/safekeep ($(version)) unstable; urgency=low/' > $(releasename)/debian/changelog
+ mkdir -p $(releasedir); tar cz -f $(releasedir)/$(releasename).tar.gz $(releasename)
+ cd $(releasename); make docs
+ rm -rf $(releasename)
+
+distdeb: distdeb-build distdeb-sign
+
+distdeb-build: $(releasedir)/$(releasename).tar.gz
+ tar xz -C /tmp -f $<
+ cd /tmp/$(releasename) && dpkg-buildpackage -us -uc
+ mv /tmp/$(name)-*_$(version)_all.deb $(releasedir)
+
+distdeb-sign:
+ debsign $(releasedir)/$(name)-*_$(version)_all.deb
+
+distrpm: distrpm-build distrpm-sign
+
+distrpm-build: $(releasedir)/$(releasename).tar.gz
+ rpmbuild -ta $<
+ mv $(rpmroot)/SRPMS/$(releasename)-$(release)*.src.rpm $(releasedir)
+ mv $(rpmroot)/RPMS/noarch/$(name)-*-$(version)-$(release)*.noarch.rpm $(releasedir)
+
+distrpm-sign:
+ rpm --addsign $(releasedir)/$(releasename)-$(release)*.src.rpm $(releasedir)/$(name)-*-$(version)-$(release)*.noarch.rpm
+
+dist-sign: distrpm-sign distdeb-sign
+
+dist-all: dist distdeb-remote fetch-debs distrpm-remote fetch-rpms dist-sign
+
+distdeb-remote:
+ ssh $(deb_box) 'cd ~/safekeep/safekeep; svn up; cd trunk; make distdeb-build'
+
+fetch-debs:
+ scp $(deb_box):~/safekeep/safekeep/trunk/$(releasedir)/$(name)-*_$(version)_all.deb $(releasedir)
+
+distrpm-remote:
+ ssh $(rpm_box) 'cd ~/safekeep/safekeep; svn up; cd trunk; make distrpm-build'
+
+fetch-rpms:
+ scp $(rpm_box):~/safekeep/safekeep/trunk/$(releasedir)/$(name)-*$(version)-$(release).*.rpm $(releasedir)
+
+deploy-lattica:
+ scp $(releasedir)/${name}{,-common,-client,-server}-${version}-*.rpm ${repo_srv}:${repo_dir}/upload
+ ssh ${repo_srv} "cd ${repo_dir}; ./deploy-rpms.sh upload/${name}-*${version}-*.rpm"
+
+deploy-sf:
+ echo -e "cd $(sf_dir)\nmkdir $(version)" | sftp -b- $(sf_login)
+ scp $(releasedir)/$(releasename).tar.gz $(sf_login):$(sf_dir)/$(version)
+ scp ANNOUNCE $(sf_login):$(sf_dir)/$(version)/README.txt
+ scp $(releasedir)/$(releasename)-$(release)*.src.rpm $(releasedir)/$(name)-*-$(version)-$(release)*.noarch.rpm $(sf_login):$(sf_dir)/$(version)
+ scp $(releasedir)/$(name)-*_$(version)_all.deb $(sf_login):$(sf_dir)/$(version)
+
+check:
+ safekeep-test --local
+
+test:
+ safekeep-test --remote
+
+clean:
+ rm -f {.,doc,debian}/*~ *.py[co]
+ rm -f $(name).spec debian/changelog
+ rm -f doc/*.xml doc/*.html doc/*.[15]
+ rm -f safekeep-*[.]20[01][0-9][01][0-9][0-3][0-9][012][0-9][0-5][0-9]*
Deleted: safekeep/tags/Release-safekeep-1_3_1/safekeep
===================================================================
--- safekeep/trunk/safekeep 2010-11-22 04:32:48 UTC (rev 713)
+++ safekeep/tags/Release-safekeep-1_3_1/safekeep 2010-11-23 04:13:03 UTC (rev 725)
@@ -1,1565 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright (C) 2006-2010 Lattica, Inc.
-#
-# SafeKeep is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-#
-# Safekeep is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Safekeep. If not, see <http://www.gnu.org/licenses/>.
-
-from __future__ import generators
-import getopt, os, os.path, re, sys, fnmatch, stat, types
-import commands, tempfile, time, traceback
-import getpass, pwd, xml.dom.minidom
-import socket, smtplib
-
-try:
- import subprocess
- from subprocess import PIPE, STDOUT
- use_subprocess = True
-except:
- PIPE = -1
- STDOUT = -2
- use_subprocess = False
-
-######################################################################
-# Python 2.2 compatibility
-######################################################################
-# There is no guarantee that we'll continue supporting Python 2.2
-# indefinitely, but we make a reasonable effor to do so as long as
-# it doesn't result in major complication/ugliness.
-
-try:
- True, False
-except NameError:
- True, False = 1, 0
-
-def enumerate(obj):
- i = -1
- for item in obj:
- i += 1
- yield i, item
-
-######################################################################
-# Global settings
-######################################################################
-
-config_file = '/etc/safekeep/safekeep.conf'
-config_ext = '.backup'
-trickle_cmd = 'trickle'
-logbuf = []
-is_client = False
-verbosity_level = 1
-verbosity_ssh = ''
-verbosity_trickle = ''
-work_user = getpass.getuser()
-backup_user = None
-home_dir = None
-base_dir = None
-default_bandwidth = {}
-cmd = "<Missing>"
-
-PROTOCOL = "1.1"
-VERSION = "1.3.0"
-VEBOSITY_BY_CLASS = {'DBG': 3, 'INFO': 2, 'WARN': 1, 'ERR': 0}
-
-######################################################################
-# Miscellaneous support functions
-######################################################################
-
-def send(msg):
- print msg.encode('utf-8')
- sys.stdout.flush()
-
-def log(msg, cls=None):
- global logbuf
- if cls:
- if is_client: cls = cls.lower()
- msg = '%s: %s' % (cls, msg)
- else:
- for c in VEBOSITY_BY_CLASS.keys():
- if msg.startswith(c + ': '):
- cls = c
- break
- else:
- cls = 'UNK'
-
- cutoff = VEBOSITY_BY_CLASS.get(cls.upper())
- if cutoff is None: cutoff = 3
- if is_client or verbosity_level >= cutoff:
- logbuf.append(msg)
- if is_client:
- send(msg)
- else:
- print >> sys.stderr, msg.encode('utf-8')
-
-def info_file(file, marker=None):
- info('# File: ' + file)
- errs = 0;
- fin = open(file, 'r')
- try:
- for line in fin.readlines():
- if marker:
- if line.startswith(marker):
- marker = None
- continue
- if line.startswith("Errors "):
- errs = int(line[6:])
- info(line.rstrip())
- finally:
- fin.close()
- return errs
-
-def stacktrace():
- return "\n" + traceback.format_exc()
-
-def debug(msg):
- log(msg, 'DBG')
-
-def info(msg):
- log(msg, 'INFO')
-
-def warn(msg):
- log(msg, 'WARN')
-
-def error(msg, ex):
- extra = ""
- if ex:
- extra = stacktrace()
- log(msg + extra, 'ERR')
-
-def do_spawn(args, stdin=None, stdout=False):
- debug('Run [' + ' '.join(args) + ']')
- _shell = isinstance(args, types.StringTypes)
- if stdin:
- _stdin = PIPE
- else:
- _stdin = None
- if stdout:
- _stderr = None
- else:
- _stderr = STDOUT
-
- if use_subprocess:
- proc = subprocess.Popen(args, bufsize=1, shell=_shell, stdin=_stdin, stdout=PIPE, stderr=_stderr, close_fds=True)
- child_in = proc.stdin
- child_out = proc.stdout
- else:
- if _shell:
- args = ["/bin/sh", "-c", args]
- if _stderr:
- (child_in, child_out) = os.popen4(args)
- else:
- (child_in, child_out) = os.popen3(args)
-
- if not stdin:
- child_in.close()
-
- if stdin:
- child_in.write(stdin)
- child_in.close()
-
- lines=[]
- for line in child_out:
- if stdout:
- lines.append(line)
- else:
- info(line.rstrip())
- child_out.close()
-
- if use_subprocess:
- return (proc.wait(), ''.join(lines))
- else:
- return (0, ''.join(lines))
-
-
-def _spawn(args, stdin=None, stdout=False):
- if isinstance(args, types.StringTypes):
- cmd = args.split(None)[0]
- else:
- cmd = args[0]
-
- try:
- rc, out = do_spawn(args, stdin, stdout)
- except OSError, ex:
- ret = "OSError: %s" % (ex)
- error('%s failed: %s' % (cmd, ret));
- return ret
-
- if not rc:
- ret = None
- elif rc > 0:
- ret = 'exited with non zero status: %d' % rc
- elif rc < 0:
- ret = 'killed by signal: %d' % -rc
- else:
- ret = 'unknown exit status: %d' % rc
- if ret:
- error('%s failed: %s' % (cmd, ret));
- return (ret, out)
-
-# this just spawns an external program (optionally through a shell)
-# and returns True it it fails, and False if it successed
-def spawn(args):
- rc, out = _spawn(args)
- return rc
-
-# this spawans an external program (optionally through a shell),
-# feeds it any input via stdin, captures the output and returns it.
-# if it fails it returns None, otherwise it returns the output
-def call(args, stdin=None):
- rc, out = _spawn(args, stdin, stdout=True)
- if not rc:
- return None
- return out
-
-def try_to_run(args):
- try:
- rc, out = do_spawn(args)
- except OSError, ex:
- return False
- return rc in (0,1)
-
-def send_notification(email, smtp):
- global logbuf
- if not logbuf: return
- info('Sending email to %s via %s' % (','.join(email), smtp))
- hostname = socket.gethostname()
- msg = 'From: SafeKeep@' + hostname + \
- '\r\nTo: ' + ', '.join(email) + \
- '\r\nSubject: SafeKeep results for ' + hostname + \
- '\r\n\r\n' + '\r\n'.join(logbuf)
- if smtp:
- server = smtplib.SMTP(smtp)
- server.sendmail('SafeKeep@' + hostname, email, msg)
- server.quit()
- else:
- cmd = ['/usr/sbin/sendmail', '-t']
- call(cmd, stdin=msg)
-
-def is_temp_root(dir):
- return dir != '/'
-
-def reroot(root, path):
- if root == '/': return path
- if root.endswith('/'): root = root[:-1]
- if not path: return root
- if path.startswith('/'): return root + path
- return os.path.join(root, path)
-
-def parse_prop_file(file):
- props = {}
- fin = open(file)
- lines = fin.readlines()
- fin.close()
- for line in lines:
- line = line.strip()
- if len(line) is 0 or line[0] is '#': continue
- if '=' in line:
- key, value = line.split('=', 1)
- props[key.strip()] = value.strip()
- else:
- props[line] = None
- return props
-
-######################################################################
-# Configuration file parser
-######################################################################
-
-class ConfigException (Exception):
- def __init__(self, value):
- self.value = value
- def __str__(self):
- return repr(self.value)
-
-def parse_dump(dump_el):
- type = dump_el.getAttribute('type')
- if not type:
- raise ConfigException('You need to specify the database type')
- if type not in ('postgres', 'postgresql', 'pgsql', 'mysql'):
- raise ConfigException('Invalid database type: %s' % type)
- db = dump_el.getAttribute('db')
- user = dump_el.getAttribute('user')
- dbuser = dump_el.getAttribute('dbuser')
- dbpasswd = dump_el.getAttribute('dbpasswd')
- opts = (dump_el.getAttribute('options') or '').split()
-
- file = dump_el.getAttribute('file')
- if not file:
- raise ConfigException('You need to specify where the database should be dumped')
- cleanup = dump_el.getAttribute('cleanup')
- return { 'type' : type, 'db' : db, 'user' : user, 'dbuser' : dbuser, 'dbpasswd': dbpasswd,
- 'opts' : opts, 'file' : file, 'cleanup' : cleanup }
-
-def parse_snap(snap_el):
- device = snap_el.getAttribute('device')
- if not device:
- raise ConfigException('Please specify the device to be snapshot')
- if device.rfind('/') == -1 or device.endswith('/'):
- raise ConfigException('The device name seems incorrect: ' + device)
- size = snap_el.getAttribute('size')
- if not size:
- raise ConfigException('Please specify the size for the snapshot')
- return { 'device' : device, 'size' : size }
-
-def parse_clude(clude_el):
- path = clude_el.getAttribute('path')
- path = path.replace('*', '\*').replace('?', '\?')
- path = path.replace('[', '\[').replace(']', '\]')
- glob = clude_el.getAttribute('glob')
- regexp = clude_el.getAttribute('regexp')
- if not path and not glob and not regexp:
- raise ConfigException('Empty ' + clude_el.tagName)
- return { 'type' : clude_el.tagName, 'path' : path, 'glob' : glob, 'regexp' : regexp }
-
-def parse_bandwidth(bw_el):
- return {
- 'overall': int(bw_el.getAttribute('overall') or 0),
- 'download': int(bw_el.getAttribute('download') or 0),
- 'upload': int(bw_el.getAttribute('upload') or 0)
- }
-
-def parse_data_attributes(data_el):
- return {
- 'exclude-devices': (data_el.getAttribute('exclude-devices') or 'false'),
- 'exclude-sockets': (data_el.getAttribute('exclude-sockets') or 'false'),
- 'exclude-fifos': (data_el.getAttribute('exclude-fifos') or 'false')
- }
-
-def parse_config(backup_el, dflt_id):
- if backup_el.tagName != 'backup':
- raise ConfigException('Invalid config file, the top level element must be <backup>')
- id = backup_el.getAttribute('id')
- if not id: id = dflt_id
-
- host_el = backup_el.getElementsByTagName('host')
- if host_el:
- host = host_el[0].getAttribute('name')
- user = host_el[0].getAttribute('user')
- nice = host_el[0].getAttribute('nice')
- key_ctrl = host_el[0].getAttribute('key-ctrl')
- key_data = host_el[0].getAttribute('key-data')
- else:
- host = user = nice = key_ctrl = key_data = None
- if host and not user:
- user = 'root'
- if host and not key_ctrl:
- key_ctrl = os.path.join('.ssh', 'safekeep-server-ctrl-key')
- if host and not key_data:
- key_data = os.path.join('.ssh', 'safekeep-server-data-key')
- if key_ctrl and not os.path.isabs(key_ctrl):
- key_ctrl = os.path.join(home_dir, key_ctrl)
- if key_data and not os.path.isabs(key_data):
- key_data = os.path.join(home_dir, key_data)
-
- bw = {}
- bw_el = backup_el.getElementsByTagName('bandwidth')
- if len(bw_el) == 1:
- bw = parse_bandwidth(bw_el[0])
- elif len(bw_el) > 1:
- raise ConfigException('Can not have more than one bandwidth element')
-
- repo_el = backup_el.getElementsByTagName('repo')
- dir = None
- retention = None
- if len(repo_el) == 1:
- dir = repo_el[0].getAttribute('path')
- retention = repo_el[0].getAttribute('retention')
- elif len(repo_el) > 1:
- raise ConfigException('Can not have more than one repo element')
- if not dir: dir = id
- dir = os.path.join(base_dir, dir)
-
- options_els = backup_el.getElementsByTagName('options')
- options = []
- if len(options_els) > 0:
- for options_el in options_els[0].childNodes:
- if options_el.nodeType != options_el.ELEMENT_NODE:
- continue
- option = options_el.nodeName
- if option == 'special-files':
- warn('options element special-files is deprecated, use data attributes instead')
- if option in ('special-files', 'rdiff-backup'):
- if options_el.hasAttributes():
- for key, value in options_el.attributes.items():
- options.append({ option : { key : value } })
- else:
- raise ConfigException('Option "%s" has no value' % option)
- else:
- raise ConfigException('Unknown option "%s"' % option)
-
- setup_el = backup_el.getElementsByTagName('setup')
- dumps = []
- snaps = []
- script = None
- if len(setup_el) > 0:
- dump_els = setup_el[0].getElementsByTagName('dump')
- for dump_el in dump_els:
- dumps.append(parse_dump(dump_el))
- snap_els = setup_el[0].getElementsByTagName('snapshot')
- for snap_el in snap_els:
- snaps.append(parse_snap(snap_el))
- script_el = setup_el[0].getElementsByTagName('script')
- if len(script_el) == 1:
- script = script_el[0].getAttribute('path')
- elif len(script_el) > 1:
- raise ConfigException('Can not have more than one setup script element')
-
- data_options = {}
- data_el = backup_el.getElementsByTagName('data')
-
- if len(data_el) == 1:
- data_options = parse_data_attributes(data_el[0])
- child_els = data_el[0].childNodes
- cludes = []
- for child_el in child_els:
- if child_el.nodeType != child_el.ELEMENT_NODE:
- continue
- if child_el.tagName not in ('include', 'exclude'):
- continue
- cludes.append(parse_clude(child_el))
- cludes.append({ 'type' : 'exclude', 'path' : '', 'glob' : '', 'regexp' : '.*' })
- elif len(data_el) > 1:
- raise ConfigException('Can not have more than one data element')
- else:
- path_xcludes = [ '/dev/', '/media/', '/mnt/', '/net/', '/proc/', '/selinux/', '/sys/',
- '/tmp/', '/var/cache', '/var/lock', '/var/run', '/var/tmp',
- '/var/named/chroot/dev', '/var/named/chroot/proc',
- '/var/named/chroot/var/run', '/var/named/chroot/var/tmp' ]
- cludes = [{ 'type' : 'exclude', 'path' : path, 'glob' : None, 'regexp' : None } for path in path_xcludes]
-
- return { 'id': id, 'host' : host, 'nice' : nice, 'user' : user, 'key_ctrl' : key_ctrl, 'key_data' : key_data,
- 'dir' : dir, 'retention' : retention, 'dumps' : dumps, 'snaps' : snaps, 'script' : script,
- 'cludes' : cludes, 'data_options' : data_options, 'options' : options, 'bw' : bw}
-
-def parse_locs(cfglocs):
- cfgfiles = []
- for cfg in cfglocs:
- if os.path.isdir(cfg):
- for ent in os.listdir(cfg):
- if not ent.endswith(config_ext):
- warn('Ignoring file %s not ending in %s' % (os.path.join(cfg, ent), config_ext))
- continue
- filepath = os.path.join(cfg, ent)
- if not os.path.isfile(filepath):
- continue
- cfgfiles.append(filepath)
- elif os.path.isfile(cfg):
- cfgfiles.append(cfg)
- else:
- warn('Inaccessible configuration, ignoring: %s' % cfg)
-
- cfgs = {}
- for filepath in cfgfiles:
- filename = os.path.splitext(os.path.basename(filepath))[0]
-
- cfg_file = open(filepath)
- cfg_str = cfg_file.read().strip()
- cfg_file.close()
-
- dom = xml.dom.minidom.parseString(cfg_str)
- try:
- cfg = parse_config(dom.documentElement, filename)
- finally:
- dom.unlink()
- cfg['text'] = cfg_str
- if cfg['id'] in cfgs:
- raise ConfigException('Duplicate client ID: %s' % cfg['id'])
- cfgs[cfg['id']] = cfg
-
- return cfgs
-
-######################################################################
-# Script, DB and SNAPSHOT support
-# setup methods can raise exception to signal errors
-# teardown methods must succeed and cleanup the state
-######################################################################
-
-def check_script_permissions(script):
- if not os.path.isfile(script):
- return '%s is not a regular file' % script
- if not os.access(script, os.X_OK):
- return '%s is not executable' % script
-
- statinfo = os.stat(script)
- if statinfo.st_uid and statinfo.st_uid != os.getuid():
- return '%s is owned by others' % script
-
- if (statinfo.st_mode & (stat.S_IWGRP | stat.S_IWOTH)):
- return '%s is writable by others' % script
-
- return None
-
-def client_side_script(step, cfg, bdir):
- debug('Do client_side_script: step %s' % step)
-
- ret = None
- script = cfg['script']
-
- if script:
- debug('client_side_script: script = %s' % script)
- if os.path.exists(script):
- ret = check_script_permissions(script)
- if not ret:
- ret = spawn([script, step, cfg['id'], bdir])
- else:
- debug('client_side_script: %s not found' % script)
-
- return ret
-
-def do_client_dbdump(cfg):
- debug('Doing DB dumps')
- for dump in cfg['dumps']:
- type = dump['type']
- opts = dump['opts']
- passwdfile = None
- if type in ('postgres', 'postgresql', 'pgsql'):
- if dump['db']:
- args = ['pg_dump']
- args.extend(['-C'])
- else:
- args = ['pg_dumpall']
- if dump['dbuser']:
- args.extend(['-U', dump['dbuser']])
- args.extend(opts)
- if dump['db']:
- args.extend([dump['db']])
- if dump['dbpasswd']:
- (fd, passwdfile) = tempfile.mkstemp()
- f = os.fdopen(fd, 'w')
- f.write(dump['dbpasswd'])
- f.close()
-
- elif type in ('mysql'):
- args = ['mysqldump']
- if dump['dbuser']:
- args.extend(['-u', dump['dbuser']])
- if dump['dbpasswd']:
- args.extend(['-p%s' % dump['dbpasswd']])
- if not dump['db']:
- args.extend(['-A'])
- args.extend(opts)
- if dump['db']:
- args.extend([dump['db']])
-
- else:
- warn('Invalid database type: ' + type)
- continue
-
- if dump['user']:
- cmd = ' '.join([commands.mkarg(arg) for arg in args])
- args = [ 'su', '-c', cmd, '-', dump['user'] ]
- cmd = ' '.join([commands.mkarg(arg) for arg in args])
- cmd = '%s > %s' % (cmd, commands.mkarg(dump['file']))
-
-
- if passwdfile:
- os.environ['PGPASSFILE'] = passwdfile
- try:
- ec = spawn(cmd)
- finally:
- if passwdfile:
- del os.environ['PGPASSFILE']
- os.remove(passwdfile)
- if ec:
- warn('Can not dump the database: %s' % dump['db'])
-
-def do_client_dbdump_teardown(cfg):
- debug('Tear down DB dumps')
- for dump in cfg['dumps']:
- if dump['cleanup'].lower() != 'true':
- continue
- try:
- os.remove(dump['file'])
- except Exception, e:
- warn('Unable to remove dump file: %s for database %s because: %s' %
- (dump['file'], dump['db'], e))
-
-def lvm_snap_information():
- lines = call(['lvs', '--separator', ':', '--noheadings']) or ''
- lvms = []
- for line in lines:
- if line.count(':') > 3:
- (volume, group, attr, blah1) = line.lstrip().split(':', 3)
- if fnmatch.fnmatch(volume, '*_snap_safekeep-*') and attr[0].lower() == 's':
- lvms.append([volume, group])
- return lvms
-
-def mount_information(reverse = False):
- lines = call(['mount']) or ''
- mounts = []
- pattern = re.compile(r"^(\S+) on (.+) type (\S+) \((\S+)\)")
- if reverse:
- lines.reverse()
- for line in lines:
- matches = pattern.match(line)
- if not matches is None:
- mounts.append(matches.groups())
- return mounts
-
-def map_lvm_device(device):
- device = device.replace('/mapper','').replace('-','/').replace('//', '-')
- return device.split('/')[-2:]
-
-def check_lvm_information(device):
- (group, volume) = map_lvm_device(device)
- for (lvm_volume, lvm_group) in lvm_snap_information():
- if lvm_group == group and lvm_volume.startswith(volume):
- return True
- return False
-
-def gather_lvm_information(device):
- (group, volume) = map_lvm_device(device)
- for (device, mountpoint, mounttype, mountoptions) in mount_information(False):
- if [group, volume] == map_lvm_device(device):
- return (group, volume, mountpoint, mounttype)
- return (None, None, None, None)
-
-def gather_snap_information(device, bdir):
- (group, volume, mountpoint, mounttype) = gather_lvm_information(device)
- if not mountpoint: return (None, None, None, None)
- lvmdev = os.path.join('/dev', group, volume)
- if bdir[-1] == '/': bdir = bdir[:-1]
- snapname = '%s_snap_%s' % (volume, os.path.basename(bdir))
- snapdev = os.path.join('/dev', group, snapname)
- if os.path.isabs(mountpoint[0]): mountpoint = mountpoint[1:]
- return (lvmdev, snapdev, os.path.join(bdir, mountpoint), mounttype)
-
-def do_client_snap(cfg, bdir):
- assert is_temp_root(bdir)
- debug('Doing FS snapshots')
- for snap in cfg['snaps']:
- device = snap['device']
- (lvmdev, snapdev, snapmnt, snaptyp) = gather_snap_information(device, bdir)
- if not snapmnt:
- warn('Cannot find the mountpoint for: %s' % device)
- continue
- args = ['lvcreate', '--snapshot', '--size', snap['size'],
- '--name', os.path.basename(snapdev), lvmdev]
- ec = spawn(args)
- if ec:
- warn('Can not snapshot the device: %s' % device)
- continue
- # no need to mkdir since the mountpoint already exists
- args = ['mount', '-t', snaptyp, snapdev, snapmnt]
- ec = spawn(args)
- if ec:
- warn('Can not mount the snapshot: %s' % device)
- ret = spawn(['lvremove', '--force', snapdev])
- if ret:
- warn('Can not tear down snapshot: %s' % device)
-
-def do_client_snap_teardown(cfg, bdir):
- assert is_temp_root(bdir)
- debug('Tear down FS snapshots dumps')
- snaps = list(cfg['snaps'])
- snaps.reverse()
- for snap in snaps:
- device = snap['device']
- (lvmdev, snapdev, snapmnt, snaptyp) = gather_snap_information(device, bdir)
- if not snapmnt:
- warn('Can not find the mountpoint for: %s' % device)
- continue
- ret = spawn(['umount', snapmnt])
- if ret:
- warn('Can not umount the snapshot: %s' % snapmnt)
-
- # stupid workaround for https://bugzilla.redhat.com/show_bug.cgi?id=577798
- for i in range(1,10):
- ret = spawn(['lvremove', '--force', snapdev])
- if not ret:
- break
-
- if ret:
- warn('Can not tear down snapshot: %s' % device)
-
-######################################################################
-# Client implementation
-######################################################################
-
-def do_client_config(cmd):
- cfgStr = ''
-
- (cfg_cmd, cnt_str, dflt_id) = cmd.split(':', 2)
- for i in xrange(int(cnt_str)):
- line = sys.stdin.readline()
- if not line: raise ConfigException('Unexpected end of file')
- cfgStr += line
-
- return do_client_config_parse(cfgStr, dflt_id)
-
-def do_client_config_parse(cfgStr, dflt_id=None):
- dom = xml.dom.minidom.parseString(cfgStr)
- try:
- return parse_config(dom.documentElement, dflt_id)
- finally:
- dom.unlink()
-
-def do_client_setup(cfg):
- debug('Do setup of %s' % cfg['host'])
-
- do_client_dbdump(cfg)
-
- if len(cfg['snaps']) > 0:
- debug('Checking FS snapshots')
- for snap in cfg['snaps']:
- device = snap['device']
- if check_lvm_information(device):
- raise Exception("Previous snapshots found for %s: run 'safekeep --server --cleanup' to correct" % device)
-
- ret = spawn(['modprobe', 'dm-snapshot'])
- if ret:
- warn('modprobe dm-snapshot failed, continuing')
- bdir = tempfile.mkdtemp("-rbind", "safekeep-", "/mnt")
- ret = spawn(['mount', '--rbind', '/', bdir])
- if ret:
- warn('mount --rbind failed, snapshotting will be disabled')
- try:
- os.rmdir(bdir)
- except Exception, e:
- warn('Failed to remove: %s' % bdir)
- bdir = '/'
- else:
- do_client_snap(cfg, bdir)
- else:
- bdir = '/'
- debug('Working root is %s' % bdir)
-
- return bdir
-
-def do_client_cleanup(cfg, bdir):
- debug('Do cleanup of %s in %s' % (cfg['host'], bdir))
- if is_temp_root(bdir):
- do_client_snap_teardown(cfg, bdir)
-
- ret = spawn(['umount', '-l', bdir])
- if ret:
- warn('Failed to unmount: %s' % bdir)
- else:
- try:
- os.rmdir(bdir)
- except Exception, e:
- warn('Unable to remove: ' + bdir)
-
- do_client_dbdump_teardown(cfg)
-
-def do_client_compat(server_versions):
- debug('Server versions: %s' % server_versions)
-
-def do_client_scrub():
- debug("Do client scrub loop")
-
- if os.getuid():
- if is_client:
- raise Exception('client not running as root')
- else:
- warn('--cleanup should be run as root on client')
- info('No cleanup performed')
- else:
- scrubbed = False
-
- if os.environ['PATH'][-1] == ':':
- os.environ['PATH'] += '/sbin:/usr/sbin:/usr/local/sbin:'
- else:
- os.environ['PATH'] += ':/sbin:/usr/sbin:/usr/local/sbin'
-
- # Go through and unmount anythings that are still hanging around
-
- debug("Cleaning up existing mounts")
- for (device, mountpoint, mounttype, mountoptions) in mount_information(True):
- if mountpoint.startswith('/mnt/safekeep-'):
- info("Removing mount %s" % mountpoint)
- if device == '/' and 'bind' in mountoptions.split(','):
- info("Removing rbind directory %s" % mountpoint)
- ret = spawn(['umount', '-l', mountpoint])
- if ret:
- warn('Failed to unmount: %s' % mountpoint)
- else:
- try:
- os.rmdir(mountpoint)
- except Exception, e:
- warn('Failed to remove: %s' % mountpoint)
- else:
- ret = spawn(['umount', mountpoint])
- if ret:
- warn('Can not unmount the snapshot: %s' % mountpoint)
- if fnmatch.fnmatch(device, '*_snap_safekeep-*'):
- info("Removing snapshot %s" % device)
- ret = spawn(['lvremove', '--force', device])
- if ret:
- warn('Can not tear down snapshot: %s' % device)
- scrubbed = True
-
- # Now cleanup any snapshots still hanging around
-
- debug("Cleaning up remaining snapshots")
- for (volume, group) in lvm_snap_information():
- device = os.path.join('/dev', group, volume)
- info("Removing snapshot %s" % device)
- ret = spawn(['lvremove', '--force', device])
- if ret:
- warn('Can not tear down snapshot: %s' % device)
- scrubbed = True
-
- # Now cleanup any safekeep directories still hanging around
-
- debug("Cleaning up remaining safekeep directories")
- if os.path.isdir('/mnt'):
- for ent in os.listdir('/mnt'):
- mountpoint = os.path.join('/mnt', ent)
- if ent.startswith('safekeep-') and os.path.isdir(mountpoint):
- info("Removing rbind directory %s" % mountpoint)
- try:
- os.rmdir(mountpoint)
- except Exception, e:
- warn('Failed to remove: %s' % mountpoint)
-
- if not scrubbed:
- info('No cleanup required')
-
-def do_client():
- debug("Do client main loop")
- should_cleanup = True
- bdir = '/'
- cfg = do_client_config_parse('<backup/>', 'def')
- ex = None
- try:
- while True:
- try:
- line = sys.stdin.readline()
- if line.startswith('ALOHA'):
- do_client_compat(line.strip().split(':', 1)[1])
- send('OK %s, %s' % (PROTOCOL, VERSION))
- elif line.startswith('CONFIG'):
- cfg = do_client_config(line)
- ret = client_side_script('STARTUP', cfg, bdir)
- if ret:
- send('ERROR Client-side setup script failed: %s' % ret)
- else:
- send('OK')
- elif line.startswith('SETUP'):
- client_side_script('PRE-SETUP', cfg, bdir)
- bdir = do_client_setup(cfg)
- client_side_script('POST-SETUP', cfg, bdir)
- send('OK ' + bdir)
- elif line.startswith('CLEANUP'):
- dir = line[7:].strip()
- if dir == bdir: should_cleanup = False
- do_client_cleanup(cfg, dir)
- client_side_script('POST-BACKUP', cfg, bdir)
- send('OK')
- elif line.startswith('SCRUB'):
- do_client_scrub()
- client_side_script('POST-SCRUB', cfg, bdir)
- send('OK')
- elif not line:
- break
- else:
- send('ERROR Unknown command: %s' % line)
- break
- except Exception, e:
- ex = e
- break
- finally:
- if should_cleanup:
- do_client_cleanup(cfg, bdir)
-
- if ex:
- send('TRACEBACK ' + ex + '>>>' + stacktrace().replace('\n', '###'))
-
-######################################################################
-# Server implementation
-######################################################################
-
-def do_server_getanswer(cout):
- while True:
- line = cout.readline()
- if line.startswith('OK'):
- return line[2:-1].strip()
- elif line.startswith('ERROR'):
- raise Exception(line[5:].strip())
- elif line.startswith('TRACEBACK'):
- i = line.find('>>>')
- error(line[i+3:].replace('###', '\n'))
- raise Exception(line[10:i].strip())
- elif not line:
- raise Exception('client died unexpectedly')
- else:
- log(line[:-1])
-
-def do_server_rdiff(cfg, bdir, nice, ionice, force):
- args = []
-
- if nice:
- args.extend(['nice', '-n' + str(nice)])
-
- ionice_cmd = 'ionice'
- if ionice and ionice != 'none':
- if try_to_run(ionice_cmd):
- if ionice is 'idle':
- args.extend([ionice_cmd, '-c3', '-t'])
- else:
- args.extend([ionice_cmd, '-c2', '-n' + ionice, '-t'])
- else:
- warn('ionice(1) not available, ignoring ionice.adjustment')
-
- args.extend(['rdiff-backup'])
-
- if cfg['host']:
- trickle = ''
-
- def get_bw(vals, dir):
- return vals.get(dir) or vals.get('overall')
-
- def get_bandwidth(cfg, dir):
- return get_bw(cfg['bw'], dir) or get_bw(default_bandwidth, dir)
-
- limit_dl = get_bandwidth(cfg, 'download')
- limit_ul = get_bandwidth(cfg, 'upload')
- if limit_dl or limit_ul:
- trickle = trickle_cmd + ' ' + verbosity_trickle
- if limit_dl:
- trickle += ' -d ' + str(limit_dl)
- if limit_ul:
- trickle += ' -u ' + str(limit_ul)
-
- if trickle:
- if not try_to_run(trickle_cmd + ' -V'):
- warn('Trickle not available, bandwidth limiting disabled')
- trickle = ''
-
- schema = '%s ssh %s -i %s %%s rdiff-backup --server' % (trickle, verbosity_ssh, cfg['key_data'])
- args.extend(['--remote-schema', schema])
-
- if force:
- args.extend(['--force'])
-
- options_append = []
-
- special_files = []
- if cfg['data_options'].get('exclude-devices').lower() == 'true':
- special_files.extend(['--exclude-device-files'])
- if cfg['data_options'].get('exclude-sockets').lower() == 'true':
- special_files.extend(['--exclude-sockets'])
- if cfg['data_options'].get('exclude-fifos').lower() == 'true':
- special_files.extend(['--exclude-fifos'])
-
- for option in cfg['options']:
- if 'special-files' in option:
- if 'include' in option['special-files']:
- if 'true' == option['special-files']['include'].lower():
- special_files = ['--include-special-files']
-
- # Note if we ever add other backends this section should only be run
- # when rback-diff is the current option.
-
- if 'rdiff-backup' in option:
- if 'append' in option['rdiff-backup']:
- options_append.extend(option['rdiff-backup']['append'].split(None))
-
- args.extend(special_files)
- args.extend(options_append)
-
- for clude in cfg['cludes']:
- opt = '--' + clude['type']
- if clude['path']:
- args.extend([opt, reroot(bdir, clude['path'])])
- if clude['glob']:
- args.extend([opt, reroot(bdir, clude['glob'])])
- if clude['regexp']:
- args.extend([opt + '-regexp', bdir + clude['regexp']])
-
- userhost = ''
- if cfg['host']:
- userhost = '%s@%s' % (cfg['user'], cfg['host'])
- args.extend([userhost + '::' + bdir, cfg['dir']])
- ret = spawn(args)
- if ret:
- raise Exception('Failed to run rdiff-backup')
-
-def do_server_rdiff_cleanup(cfg):
- args = ['rdiff-backup', '--check-destination-dir', cfg['dir']]
- ret = spawn(args)
- if ret:
- warn('Failed to cleanup old data, please fix the problem manually')
-
-def do_server_data_cleanup(cfg):
- args = ['rdiff-backup', '--force', '--remove-older-than', cfg['retention'], cfg['dir']]
- ret = spawn(args)
- if ret:
- warn('Failed to cleanup old data, please fix the problem manually')
-
-def do_server_compat(client_versions):
- (client_protocol, client_version) = client_versions.split(',')
- (client_major, client_minor) = client_protocol.strip().split('.')
- (server_major, server_minor) = PROTOCOL.split('.')
- if server_major != client_major:
- raise Exception('Incompatible protocols: %s <> %s' % (PROTOCOL, client_protocol))
- elif server_minor > client_minor:
- warn('Protocol mismatch: %s <> %s' % (PROTOCOL, client_protocol))
-
-def do_server(cfgs, ids, nice, ionice, force, cleanup):
- debug("Do server main loop")
- for cfg in cfgs.itervalues():
- id = cfg['id']
- if ids and id not in ids: continue
- info('------------------------------------------------------------------')
- info('Server backup starting for client %s' % id)
-
- cleaned_up = 0
- try:
- if cfg['host']:
- if not os.path.isfile(cfg['key_ctrl']):
- raise Exception('Client %(id)s missing ctrl key %(key_ctrl)s' % cfg)
- if not os.path.isfile(cfg['key_data']):
- raise Exception('Client %(id)s missing data key %(id)s' % cfg)
-
- datadir = os.path.join(os.getcwd(), cfg['dir'])
- if not os.path.isdir(datadir):
- try:
- os.makedirs(datadir)
- except EnvironmentError, ex:
- raise Exception('Can not create data store dir: %s' % datadir)
-
- rdiff_logdir = os.path.join(datadir, 'rdiff-backup-data')
- if cfg['retention'] and os.path.isdir(rdiff_logdir) and not cleanup:
- do_server_data_cleanup(cfg)
-
- cmd = []
- if cfg['host']:
- cmd.extend(['ssh', verbosity_ssh, '-T', '-i', cfg['key_ctrl'], '-l', cfg['user'], cfg['host']])
- cmd.extend(['safekeep', '--client'])
-
- (cin,cout) = os.popen2(cmd)
-
- cin.write('ALOHA: %s, %s\n' % (PROTOCOL, VERSION))
- cin.flush()
- client_versions = do_server_getanswer(cout)
- do_server_compat(client_versions)
-
- cin.write('CONFIG: %d: %s\n' % (len(cfg['text'].splitlines()), id))
- cin.write(cfg['text'] + '\n')
- cin.flush()
- do_server_getanswer(cout)
- if cleanup:
- cin.write('SCRUB\n')
- cin.flush()
- do_server_getanswer(cout)
- bdir = '/' # Fake directory for the rest of the cleanup
- do_server_rdiff_cleanup(cfg)
- cleaned_up = 1
- errs = 0
- else:
- cin.write('SETUP\n')
- cin.flush()
- bdir = do_server_getanswer(cout)
-
- if os.path.isdir(rdiff_logdir):
- rdiff_logpre = os.listdir(rdiff_logdir)
- else:
- rdiff_logpre = []
-
- backup_log = os.path.join(rdiff_logdir, 'backup.log')
- if os.path.isfile(backup_log):
- backup_marker = '=== Backup session on %s ===' % time.asctime()
- fbm = open(backup_log, 'a')
- fbm.write(backup_marker + '\n')
- fbm.close()
- else:
- backup_marker = None
-
- do_server_rdiff(cfg, bdir, nice, ionice, force)
-
- errs = 0
- if os.path.isdir(rdiff_logdir):
- info_file(backup_log, backup_marker)
- rdiff_logpost = os.listdir(rdiff_logdir)
- for lfn in rdiff_logpost:
- if lfn.startswith('session_statistics.') and lfn.endswith('.data') and lfn not in rdiff_logpre:
- errs += info_file(os.path.join(rdiff_logdir, lfn))
- else:
- warn('Log dir does not exist.')
-
- cin.write('CLEANUP %s\n' % bdir)
- cin.flush()
- do_server_getanswer(cout)
-
- if errs == 0:
- info('Server backup for client %s: OK' % id)
- else:
- info('Server backup for client %s: OK (%d WARNINGS)' % (id, errs))
-
- except Exception, ex:
- if cleanup and not cleaned_up:
- info('Client-side cleanup for client %s: FAILED' % id)
- do_server_rdiff_cleanup(cfg)
- else:
- error('Server backup for client %s: FAILED' % id, ex)
-
- info('------------------------------------------------------------------')
- debug('Server backup done')
-
-def do_list(cfgs, ids, list_type, list_date, list_parsable):
- debug("Do server listing main loop")
- for cfg in cfgs.itervalues():
- id = cfg['id']
- if ids and id not in ids: continue
- if list_parsable:
- info('Client: %s' % id)
- else:
- info('------------------------------------------------------------------')
- info('Server listing for client %s' % id)
-
-
- args = ['rdiff-backup']
-
- if list_type is 'increments':
- args.extend(['--list-increments'])
- elif list_type is 'sizes':
- args.extend(['--list-increment-sizes'])
- elif list_type is 'changed':
- args.extend(['--list-changed-since', list_date])
- elif list_type is 'attime':
- args.extend(['--list-at-time', list_date])
- else:
- assert False, 'Unknown list type: ' + list_type
-
- if list_parsable:
- args.extend(['--parsable-output'])
-
- args.extend([cfg['dir']])
- ret...
[truncated message content] |
