[Corredor-commits] SF.net SVN: corredor: [2] corredor/trunk
Status: Pre-Alpha
Brought to you by:
brlcad
From: <br...@us...> - 2006-09-19 06:02:54
|
Revision: 2 http://svn.sourceforge.net/corredor/?rev=2&view=rev Author: brlcad Date: 2006-09-18 23:01:40 -0700 (Mon, 18 Sep 2006) Log Message: ----------- Initial revision of Corredor. This version was implemented a couple years ago but was never committed to revision control until now. At this point, corredor correctly loads the plugin modules in correct dependency order and supports the primary start/stop/status/run commands along with all the supporting backend infrastructure (e.g. logging, environment processing, overrides, i/o, configuration management, locking, and more). Also included is the embedded benchmark suite for evaluating the performance of a shell environment. Added Paths: ----------- corredor/trunk/README corredor/trunk/Resources/ corredor/trunk/Resources/NOTES corredor/trunk/Resources/Source/ corredor/trunk/Resources/Source/Compression/ corredor/trunk/Resources/Source/Compression/Compression corredor/trunk/Resources/Source/Compression/ExecutionParameters.plist corredor/trunk/Resources/Source/Environment/ corredor/trunk/Resources/Source/Environment/Environment corredor/trunk/Resources/Source/Environment/ExecutionParameters.plist corredor/trunk/Resources/Source/Filesystem/ corredor/trunk/Resources/Source/Filesystem/ExecutionParameters.plist corredor/trunk/Resources/Source/Filesystem/Filesystem corredor/trunk/Resources/Source/Input/ corredor/trunk/Resources/Source/Input/ExecutionParameters.plist corredor/trunk/Resources/Source/Input/Input corredor/trunk/Resources/Source/Locking/ corredor/trunk/Resources/Source/Locking/ExecutionParameters.plist corredor/trunk/Resources/Source/Locking/Locking corredor/trunk/Resources/Source/Mail/ corredor/trunk/Resources/Source/Mail/ExecutionParameters.plist corredor/trunk/Resources/Source/Mail/Mail corredor/trunk/Resources/Source/Output/ corredor/trunk/Resources/Source/Output/ExecutionParameters.plist corredor/trunk/Resources/Source/Output/Output corredor/trunk/Resources/Source/Platform/ corredor/trunk/Resources/Source/Platform/ExecutionParameters.plist corredor/trunk/Resources/Source/Platform/Platform corredor/trunk/Resources/Source/ResourceFile/ corredor/trunk/Resources/Source/ResourceFile/ExecutionParameters.plist corredor/trunk/Resources/Source/ResourceFile/ResourceFile corredor/trunk/Resources/Source/RevisionControl/ corredor/trunk/Resources/Source/RevisionControl/ExecutionParameters.plist corredor/trunk/Resources/Source/RevisionControl/RevisionControl corredor/trunk/Resources/Source/Testing/ corredor/trunk/Resources/Source/Testing/ExecutionParameters.plist corredor/trunk/Resources/Source/Testing/Testing corredor/trunk/Resources/Source/Transport/ corredor/trunk/Resources/Source/Transport/ExecutionParameters.plist corredor/trunk/Resources/Source/Transport/Transport/ corredor/trunk/Resources/Source/Transport/Transport/RSH/ corredor/trunk/Resources/Source/Transport/Transport/SSH/ corredor/trunk/Resources/Source/Transport/Transport/SSH/SSH corredor/trunk/Resources/library corredor/trunk/Resources/rcHeader.txt corredor/trunk/Resources/test.sh corredor/trunk/SystemBucket/ corredor/trunk/SystemBucket/localhost/ corredor/trunk/TestBucket/ corredor/trunk/TestBucket/Input/ corredor/trunk/TestBucket/Locking/ corredor/trunk/TestBucket/Logging/ corredor/trunk/TestBucket/README corredor/trunk/TestBucket/Shell/ corredor/trunk/TestBucket/Shell/Shell corredor/trunk/TestBucket/Template/ corredor/trunk/TestBucket/Template/Template corredor/trunk/TestBucket/Variables/ corredor/trunk/corredor Added: corredor/trunk/README =================================================================== --- corredor/trunk/README (rev 0) +++ corredor/trunk/README 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,170 @@ + Corredor + Version 2006.10 + http://corredor.sf.net + +Corredor is a robust and general framework for performing automated +execution of tasks with a particular emphasis on software testing. +With a POSIX-compliant shell, Corredor makes it easier to set up, +maintain, and run scripts, supporting symmetric, simultaneous, and +distributed exectution. + +Corredor was initially created in 2002 with the design purpose of +performing automated and distributed testing of software packages. +Initially designed for testing the extensive BRL-CAD solid modeling +software suite, the framework was expanded and refined towards +providing a general execution framework that lets you run a set of +shell scripts are defined for a given project. + +The framework is written entirely in POSIX Bourne shell and makes no +assumptions about system functionality outside of the POSIX standard. +Any POSIX platform that can run a bourne shell should be able to +utilize Corredor without compilation or modification. The framework +provides a variety of routines that strive to make it as easy as +possible to build and test software packages. You write scripts that +can build and test your software, and then Corredor make it easy to +manage how those scripts are run locally or even distributed. + +As an example, once you describe even just a simple script that builds +a project, you can then tell the framework to automatically distribute +that job to an entire compilation farm of machines. Corredor then +takes care of all the details of packing everything up, getting onto +those machines, peforming the builds, analyzing whether the builds +succeed, cleaning up, and returning results back in some useful form. + +Corredor strives to make it as easy as possible to set up, maintain, +and run black box, white box, regression, unit, and build tests so +that software may be made more robust and reliable. Corredor supports +symmetric, simultaneous, and distributed testing. The system may be +invoked on-demand via automation or run interactively. Success and +failure reporting is given via direct-interactive feedback, file +report logging, and e-mail. + + +TABLE OF CONTENTS +----------------- + Introduction + Table of Contents + Getting Started + Benchmark + File Manifest + References + Contact & Contributions + + +GETTING STARTED +--------------- + +To effectively utilize Corredor as an execution framework, some +preparation is required. Namely, you will need to create execution +scripts specific to you particular environment testing needs. +Corredor may also be used to benchmark the overall performance of +shell environment. See the BENCHMARK section for details. + + +BENCHMARK +--------- + +Corredor has a built-in benchmark performance analysis functionality +for testing the run-time performance characteristics of a given +system's shell scripting environment. The benchmark exercises basic +shell functionality including the invocation of shell functions, +setting of environment variables, execution of required system +commands, and more. The benchmark reports the performance results in +terms of the number of shell operations per second. + + +FILE MANIFEST +------------- + README - this initial documentation file + Resources/ - execution framework internal logic and plugins + Resources/library - general purpose shell script library + SystemBucket/ - system directory containing settings for remote hosts + TestBucket/ - directory containing runnable scripts/tests + corredor - the Corredor test execution command tool + + +REFERENCES +---------- + +Regression testing +URL: http://whatis.techtarget.com/definition/0,,sid9_gci212884,00.html +NOTE: Regression testing is the process of testing changes to computer +programs to make sure that the older programming still works with the +new changes. Regression testing is a normal part of the program +development process and, in larger companies, is done by code testing +specialists. Test department coders develop code test scenarios and +exercises that will test new units of code after they have been +written. These test cases form what becomes the test bucket. Before a +new version of a software product is released, the old test cases are +run against the new version to make sure that all the old capabilities +still work. The reason they might not work is because changing or +adding new code to a program can easily introduce errors into code +that is not intended to be changed. + +Regression testing +URL: http://www.pcwebopaedia.com/TERM/R/regression_testing.html +NOTE: The selective retesting of a software system that has been +modified to ensure that any bugs have been fixed and that no other +previously-working functions have failed as a result of the +reparations and that newly added features have not created problems +with previous versions of the software. Also referred to as +verification testing, regression testing is initiated after a +programmer has attempted to fix a recognized problem or has added +source code to a program that may have inadvertently introduced +errors. It is a quality control measure to ensure that the +newly-modified code still complies with its specified requirements and +that unmodified code has not been affected by the maintenance +activity. + +Test-drive development +URL: http://www.thecoadletter.com/issues/testdriven/coadletter-testdriven-095_why_test.htm +NOTE: Article explains why Test-driven Development (TDD) is generally +a "good thing". + +Mac Startup Items +URL: http://developer.apple.com/techpubs/macosx/Essentials/SystemOverview/BootingLogin/The_Boot_Sequence.html +URL: http://developer.apple.com/techpubs/macosx/Essentials/SystemOverview/BootingLogin/Customization_Techniques.html +URL: http://developer.apple.com/techpubs/macosx/Darwin/howto/system_starter_howto/system_starter_howto.html + +Tetworks +URL: http://tetworks.opengroup.org/ +URL: http://tetworks.opengroup.org/Products/tet.html +NOTE: The Open Group's test execution product (16K for pro license + +6K/year). It appears to be the most robust and complete testing +framework around. The framework is used to test UNIX/POSIX +compliance, X11 API conformance and a slew of other "large" APIs. + +Gentoo Initscripts +URL: http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=5 + +Concurrent Versions System +URL: http://www.cvshome.org + +Red Hat rc.d initscripts +URL: file:///etc/rc.d + + +CONTACT & CONTRIBUTIONS +----------------------- + +Corredor is written and maintained by Christopher Sean Morrison. +Feedback and contributions are always appreciated and welcome, and the +project is always open to having others join in the direct development +and improvement of Corredor. + +Interactive support via IRC is preferred and generally available +during UTC-5 hours on the Freenode network (irc.freenode.net:6667) in +the #brlcad channel. Look for the user 'brlcad'. + +Bug reports, feature requests, and support requests may be made to the +project's Sourceforge site: http://sourceforge.net/projects/corredor + +Mailing lists are similarly available at the Sourceforge project site. + +Feel free to direct any questions or comments relating to Corredor via +e-mail to 'cor...@in...' or to any of the other +contact means described above. + +Cheers! +Sean + Property changes on: corredor/trunk/README ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/NOTES =================================================================== --- corredor/trunk/Resources/NOTES (rev 0) +++ corredor/trunk/Resources/NOTES 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,114 @@ +PROBLEM ANALYSIS +================ + +Needs +----- + for source code commits to the main CVS branch compile across multiple architectures + detection of software bugs + verification that expected APIs do not change unexpectedly + + +Problems +-------- + compilation is time intensive + manually compiling across multiple architectures is time intensive + verification that source code behaves as expected is time intensive + APIs are sometimes broken or used in unforseen manners + modification to one source module inadvertently induces an error in another source module + + +Potential General Solutions to Problems +--------------------------------------- + ignore the problems and/or deal with the issues as they arise + use a testing framework + design and implement one + buy one + impose policy on commiters + + +Realistic Tool Solution Expectations +------------------------------------ + it is simple to run tests across multiple architectures on demand + adding new tests is not complicated + cross compilation across multiple architectures may be automated + + +Current Manual Testing Costs +---------------------------- + time/cost varies per developer + useability testing is generally minimal (i.e. little time spent testing the software) + compilation testing is generally frequent (i.e. most time spent compiling) + compilation takes between 5 and 120 minutes per architecture (but is invoke-and-forget) + approximately 10 minutes per architecture per build is needed to setup/invoke/analyse build and run benchmarks (minimal functionality test + approximately 6 supported architectures (osx, linux, freebsd, sgi, solaris, aix) (windows?) + approximately 1 hour to build across all platforms and verify builds + 8 active core developers + one cross-compilation build/test by all developers is approximately 8 man hours (1 man day) + one build per week (moderate activity) is about 400 man hours (50 man days) per year (50 weeks) + compilations are less frequent after release (1/week) + compilations are frequent before release (1+/day) + + +Future (Expected) Manual Testing Costs +-------------------------------------- + mandatory cross compilations happen nightly and per/commit happen automatically + maintenance of test suite is necessary + + +TOOL DESIGN +=========== + +Initial Automated Testing Costs +------------------------------- + design and implementation of test suite (0 -- done offline) + design and implementation of tests (1 man hour per test) + setup of test suite (8 man hours) + usage learning by all developers (1 man hour per developer) + + +Future Automated Testing Costs +------------------------------ + maintenance of test suite (1 man hour per month) + maintenance of tests (variable -- 1 man hour per year per test) + design and implementation of new tests ( 1 man hour per test) + + +Tool Users +---------- + builder and tester + := those who want to use the suite to verify buildability and/or functionality + test writer + := those who contribute tests to the test suite + suite maintainer + := those who install and/or modify the suite itself, preparing the test framework + software + := cron, web browsers, other tools that may invoke the tool on demand + + +Mandatory Tool Features and Contraints +-------------------------------------- + works cross-platform on all supported architectures + must be minimally dependant upon external interfaces and tools + may be run on demand + may be automated + tool should not irreversibly modify user data by default + tool must be understood/learned by all users + requires no time expense when system is static (should not have to "clean up logs", etc) + + +Desireable Tool Features +------------------------ + should provide minimal false negatives (suite failure) + tool should be quick/simple to learn and use + may be run interactively (single stepping at different levels) + should be simple to add new tests + should be robust test framework to support evolving test style/strategy + should provide informative test reports for failure investigation + + +Maintenance of Tool +------------------- + needs to have new tests added as new features/tools are added + must be maintained as supported architectures evolve + must be ported to newly supported architectures + Property changes on: corredor/trunk/Resources/NOTES ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Compression/Compression =================================================================== --- corredor/trunk/Resources/Source/Compression/Compression (rev 0) +++ corredor/trunk/Resources/Source/Compression/Compression 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,52 @@ +#!/bin/sh +# +# C o m p r e s s i o n +# +### +# +# unpackArchive +# unpack an archive from a variety of formats (tar files, tgz, zip, etc) + + +# +# unpackArchive archive [extractDir] +# +# as it is, this function merely bombs on error, but it should return a +# truthvalue as to whether the unpacking succeeded or not and let the +# application handle the result. +unpackArchive ( ) { + [ ! "x$DEBUG" = "x" ] && log "unpackArchive( \$1=[$1] \$2=[$2] )" + + [ "x$1" = x ] && bomb "unpackArchive() arg1 mismatch -- no archive given" + __ARCHIVE="$1" + if [ "x$2" = "x" ] ; then + __SWS=`echo $__ARCHIVE | sed 's/^\/.*/___ROOTED_NAME___/'` + [ "x$__SWS" = "x___ROOTED_NAME___" ] && __DIR=`dirname $__ARCHIVE` || __DIR="./" + else + __DIR="$2" + fi + + # [ ! -f "$__ARCHIVE" ] && bomb "Unable to find archive [$__ARCHIVE] for unpacking" + # [ ! -r "$__ARCHIVE" ] && bomb "Unable to read archive [$__ARCHIVE] for unpacking" + + # XXX this is minimal checking and support for right now + # we assume that the uncompressor is available and options are supported + case "$__ARCHIVE" in + *.tar ) + tar -C "$__DIR" -xf "$__ARCHIVE" ;; + *.zip ) + unzip "$__ARCHIVE" ;; + *.tar.Z ) + uncompress -c "$__ARCHIVE" | tar -C "$__DIR" -x ;; + *.tar.gz | *.tgz ) + gzip -c -d "$__ARCHIVE" | tar -x -C "$__DIR" ;; + *.tar.bz2 ) + bunzip2 -c "$__ARCHIVE" | tar -C "$__DIR" -x ;; + * ) + bomb "Unsupported archive format" ;; + esac + + # [ ! "x$?" = "x0" ] && bomb "Archive unpacking failed" + return $? +} + Property changes on: corredor/trunk/Resources/Source/Compression/Compression ___________________________________________________________________ Name: svn:executable + * Name: svn:mime-type + text/x-sh Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Compression/ExecutionParameters.plist =================================================================== --- corredor/trunk/Resources/Source/Compression/ExecutionParameters.plist (rev 0) +++ corredor/trunk/Resources/Source/Compression/ExecutionParameters.plist 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,5 @@ +Description="Compression and archive routines"; +Provides="Compression"; +Requires="Output"; +Uses="Debug"; +OrderPreference="None"; Property changes on: corredor/trunk/Resources/Source/Compression/ExecutionParameters.plist ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Environment/Environment =================================================================== --- corredor/trunk/Resources/Source/Environment/Environment (rev 0) +++ corredor/trunk/Resources/Source/Environment/Environment 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,698 @@ +#!/bin/sh +# +# E n v i r o n m e n t +# +### +# +# initializeVariable +# initializes a variable to a value if not already set +# getInitializedVariables +# returns a list of variables that were initialized +# initializeOption +# initialize a special variable that has a defined behavior and interface +# getOptionSwitches +# outputs a list of switches associated with an option +# getOptionVariable +# outputs the variable associated with a given switch +# getOptionDescription +# outputs a description of an option +# variableIsTrue +# returns true or false if a variable is truthful +# variableIsFalse +# returns true or false if a variable is not truthful +# handleArgument +# turns an argument into the appropriate option value +# +# INTERNAL VARIABLES (do not use or rely on) +# +# __INITIALIZED_VARIABLE_GROUPS +# __INITIALIZED_VARIABLES_* +# + +# +# initializeVariable variableName [defaultValue] [categoryList] +# +# initializes a variable to a given value unless that variable is already +# defined (e.g. already initialized). this allows script variables to also be +# overridden by environment variables this is similar behavior as +# ${variableName:=defaultValue} available in some shells, except for the +# variable logging. variables may be logged in groups by giving a list of +# categories that this variable should be logged under. +# +# !!! need to optimize this routine to decrease startup times +# +initializeVariable ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "initializeVariable( \$1=[$1] \$2=[$2] \$3=[$3] )" + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "initializeVariable() arg1 mismatch" + else + __VAR="$1" + fi + # do not need to check since may be empty anyways + __DEF="$2" + __GROUP="`echo $3 | tr [:lower:]- [:upper:]_`" + + # make sure the name is valid (no dashes) + if [ "x`echo $__VARIABLE | tr - _`" != "x$__VARIABLE" ] ; then + bomb "Dashes are not allowed in variable names (underscores are, though)" + fi + + # stupid name to variable translation is a bitch..especially on sun5 + + # here, the variable is actually set, after we first make sure that the + # variable is not already set + __PVAR="echo \$$__VAR" + if [ "x`eval $__PVAR`" = "x" ] ; then + # if the value is already quoted, leave it quoted + # XXX need to check that the quoting still works + if [ "x`echo $__DEF | sed 's/^[\"].*/__QUOTED__/'`" = "x__QUOTED__" ] ; then + __CMD="$__VAR=$__DEF" + else + __CMD="$__VAR=\"$__DEF\"" + fi + + eval $__CMD + export $__VAR + fi + + # add the group to the list of groups if it is not there. this is + # primarily of performance use for routines that want to know what the + # groups are (like bomb) + if [ ! "x$__GROUP" = "x" ] ; then + for __GRP in $__GROUP ; do + __FOUNDGROUP=no + for __i in $__INITIALIZED_VARIABLE_GROUPS ; do + if [ "x$__GRP" = "x$__i" ] ; then + __FOUNDGROUP="yes" + break; + fi + done + if [ "x$__FOUNDGROUP" = "xno" ] ; then + __INITIALIZED_VARIABLE_GROUPS="$__INITIALIZED_VARIABLE_GROUPS $__GRP" + export __INTIALIZED_VARIABLE_GROUPS + fi + done + fi + + # add the variable to a list of known initialized variables (with optional + # grouping). note that __INITIALIZED_VARIABLES_ holds the + # main/default/non-categorized variables + if [ "x$__GROUP" = "x" ] ; then + __INITIALIZED_VARIABLES_="$__INITIALIZED_VARIABLES_ $__VAR" + export __INITIALIZED_VARIABLES_ + else + for __GRP in $__GROUP ; do + __PVAR="echo \$__INITIALIZED_VARIABLES_${__GRP}" + __PVAR="__INITIALIZED_VARIABLES_${__GRP}=\"`eval $__PVAR` $__VAR\"" + eval $__PVAR + export __INITIALIZED_VARIABLES_${__GRP} + done + fi + + __TEMP_DEBUG_OFF=1 + if [ ! x$DEBUG = x ] ; then + __PVAR="echo \$$__VAR" + log "initializeVariable: $__VAR=[`eval $__PVAR`]" + fi + unset __TEMP_DEBUG_OFF + + return 0 +} + + +# +# getInitializedVariables [categoryList] +# +# returns a list of the variables that have been initialized from an optional +# list of categorues (that would have been given when the variable was set). +# +# category names are case-insensitive +# +getInitializedVariables ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "getInitializedVariables( \$1=[$1] )" stderr + unset __TEMP_DEBUG_OFF + + __GROUP="`echo $1 | tr [:lower:]- [:upper:]_`" + + if [ "x$__GROUP" = "x" ] ; then + [ ! "x$DEBUG" = "x" ] && log "getInitializedVariables: returning $__INITIALIZED_VARIABLES_" + echo "$__INITIALIZED_VARIABLES_" + return 0 + fi + + __VARS="" + for __GRP in $__GROUP ; do + __PVAR="echo \$__INITIALIZED_VARIABLES_${__GRP}" + __VARS="`eval $__PVAR` $__VARS" + done + [ ! "x$DEBUG" = "x" ] && log "getInitializedVariables: returning $__VARS" + echo $__VARS + return 0 +} + + +# +# initializeOption variableName [defaultValue] [validValues] [description] [additionalSwitches] [categoryList] +# +# options are runtime switches that modify a programs runtime behavior. +# options may be set either 1) as a default value to a variable inside of the +# actual source (e.g. inside of ./corredor) program using the variable, or +# 1) as a system environment variable (e.g. PACKAGE_SOURCE=/usr/src/whatever), +# or 3) via a command-line argument to the program +# (e.g. ./corredor --package-source=/usr/src/whatever). +# precedence goes in reverse order: first to command-line args, then to +# environment vars, then to defaults. +# +# this function takes the "description" of an option, including its name, +# default value, valid values, a text description, additional switches, and a +# category. the only required data is the name of the option. valid values +# are escaped patterns that will match a case test. by default, an option may +# be specified via the command line according to this pattern +# -+OPTION[_-]NAME[= ]+\"?VALUE\"?. (e.g. option OPTION_NAME could be +# specified as --option-name=one, --option_name=two, +# -option-name="three four", -option_name five, OPTION_NAME=six, etc) that is +# to say, in english, that there are one or more preceeding dashes followed by +# case-insensitive letters, embedded underscores and dashes are +# interchangeable, and the equal sign and wrapping quotes are optional. +# (wrapping quotes are not optional if there are spaces in the value). +# +# category names are case-insensitive. +# +initializeOption ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "initializeOption( \$1=[$1] \$2=[$2] \$3=[$3] \$4=[$4] \$5=[$5] \$6=[$6] )" + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "initializeOption() arg1 mismatch" + else + __VARIABLE="`echo $1 | tr [:lower:]- [:upper:]_`" + fi + __DEFAULT="$2" + + # since valid is a matching pattern, default is match nothing if given "". + # that is how to make a read-only option (cannot be set) + __VALID="$3" + # wrap the value in single quotes to prevent replacement + # [ "x$__VALID" = "x" ] || __VALID="'$__VALID'" + + # a simple default label is given if no description is available + [ "x$4" = "x" ] && __DESC="$__VARIABLE option" || __DESC="$4" + + # there are no default "additional" switches or default grouping + __SWITCHES="$5" + # convert a category to uppercase + __CATEGORY="`echo $6 | tr [:lower:]- [:upper:]_`" + + # check if option is already registered as an option or is in use + __PVAR="echo \$$__VARIABLE" + if [ ! "x`eval $__PVAR`" = "x" ] ; then + + # this means the option name is already in use as an option or variable + + __PVAR="echo \$__${__VARIABLE}_SWITCHES" + if [ "x`eval $__PVAR`" = "x" ] ; then + __PVAR="echo \$$__VARIABLE" + warn "Unable to initialize $__VARIABLE as option -- variable already has value (`eval $__PVAR`)" + else + # the option is already registered + return + fi + else + # check for options with empty defaults + __PVAR="echo \$__${__VARIABLE}_SWITCHES" + if [ ! "x`eval $__PVAR`" = "x" ] ; then + # the option is in use (empty default) + return + + # xxx unreached xxx + # returning is normally desired to match initVar's default "quiet" + # behavior on initializing something already initialized. + warn "Option $__VARIABLE has already been initialized" + fi + fi + + # !!! need to optimize initializeOptions to not use initializeVariable for performance + + # actually intialize and record an option just as if it were a + # variable, then we do some extra bookkeeping + initializeVariable "$__VARIABLE" "$__DEFAULT" "$__CATEGORY" + + # add at least the variable name to the list of valid switches. + __SWITCHES="$__SWITCHES `echo $__VARIABLE | tr [:upper:]_ [:lower:]-`" + + # XXX should replace this whole section with a grep match for performance + # make sure another option is not set to use any of our switches + for __i in $__SWITCHES ; do + for __j in $__USED_SWITCHES ; do + if [ "x$__i" = "x$__j" ] ; then + # here we see if the matching switch is associated with *this* option (multiple init) + __SELF_SWITCHES="echo \$__${__VARIABLE}_SWITCHES" + __SELF_SWITCHES="`eval $__SELF_SWITCHES`" + __SELF_MATCH=0 + if [ ! "x$__SELF_SWITCHES" = "x" ] ; then + for __k in $__SELF_SWITCHES ; do + if [ "x$__i" = "x$__k" ] ; then + warn "Option \"$__VARIABLE\" already has switch \"$__k\"" + __SELF_MATCH=1 + break + fi + done + fi + if [ "x$__SELF_MATCH" = "x0" ] ; then + bomb "Conflict adding $__VARIABLE option, \"$__i\" switch already designated" + fi + # since we matched a used switch, no need to keep checking + # (the switch was previously added to this option) + break + fi + done + done + + # extra bookkeeping + initializeVariable "__${__VARIABLE}_SWITCHES" "$__SWITCHES" "_OPTION_SWITCHES" + initializeVariable "__${__VARIABLE}_VALUES" "$__VALID" "_OPTION_VALUES" + initializeVariable "__${__VARIABLE}_DESCRIPTION" "$__DESC" "_OPTION_DESCRIPTION" + + # keep specific track of all switches for collision detection. we do not + # do any error checking just because if we got here, new switches were + # added. this is a whole *lot* faster than getting a list of variables or + # options and checking all their switches then. saves us about a sec, at + # least. + __USED_SWITCHES="$__USED_SWITCHES $__SWITCHES" + export __USED_SWITCHES + + # we still perform basic book-keeping separate from the init vars + # bookkeeping, so we can iterate only over option variables + __INITIALIZED_OPTIONS_="$__INITIALIZED_OPTIONS_ $__VARIABLE" + export __INITIALIZED_OPTIONS_ + # if categories were given, make add the option to that category list + if [ ! "x$__CATEGORY" = "x" ] ; then + for __CAT in $__CATEGORY ; do + __PVAR="echo \$__INITIALIZED_OPTIONS_${__CAT}" + __PVAR="__INITIALIZED_OPTIONS_${__CAT}=\"`eval $__PVAR` $__VARIABLE\"" + eval $__PVAR + export __INITIALIZED_OPTIONS_${__CAT} + done + fi + + [ ! "x$DEBUG" = "x" ] && log "initializeOption: done with option \"$__VARIABLE\" (set to to \"$__DEFAULT\" in category \"$__CATEGORY\")" +} + + +# getOptionSwitches optionName [category] +# +# returns a list of the options for this switch. if the category is given, +# then the option must be in that category. +# +# category names are case-insensitive +# +getOptionSwitches ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "getOptionSwitches( \$1=[$1] \$2=[$2] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "getOptionSwitches() arg1 mismatch" + fi + __VARIABLE="$1" +# __CATEGORY="`echo $2 | tr [:lower:]- [:upper:]_`" + + # if they did not give a category, just return the switch +# if [ "x$__CATEGORY" = "x" ] ; then + eval "echo \$__${__VARIABLE}_SWITCHES" + return 0 +# fi + + # they gave a category so we must validate that the option is in that + # category + + # XXX unimplemented + + return 0 +} + + +# getOptionVariable switch [category] +# +# returns the option that is associated with a given switch. if the category +# is given, then the option must be in that category. +# +# category names are case-insensitive +# +getOptionVariable ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "getOptionVariable( \$1=[$1] \$2=[$2] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "getOptionVariable() arg1 mismatch" + fi + __SWITCH="$1" +# __CATEGORY="`echo $2 | tr [:lower:]- [:upper:]_`" + + # separate out __SWITCH if it is SWITCH=VALUE form + if [ ! "x`echo $__SWITCH | grep =`" = "x" ] ; then + __PVAR="`echo $__SWITCH | sed 's/^-*\([a-zA-Z_-]*\)=.*/\1/'`" + if [ "x$__PVAR" = "x" ] ; then + warn "could not match $__VAR (sed match error)" + else + __SWITCH="$__PVAR" + fi + fi + + # strip off leading dashes + __SWITCH="`echo $__SWITCH | sed 's/^-*\(.*\)/\1/'`" + + # convert embedded underscores to dashes and upper to lower + __SWITCH="`echo $__SWITCH | tr [:upper:]_ [:lower:]-`" + + # if they did not give a category, just return the switch + +# if [ "x$__CATEGORY" = "x" ] ; then + # make sure the switch is at least in use + if [ ! "x`echo $__USED_SWITCHES | grep -i $__SWITCH`" = "x" ] ; then + for __OPT in $__INITIALIZED_OPTIONS_ __placeholder__ ; do + __SWITCHES="`getOptionSwitches $__OPT`" + for __SH in $__SWITCHES ; do + if [ "x$__SH" = "x$__SWITCH" ] ; then + [ "x$DEBUG" = "x" ] || log "getOptionVariable: returning $__OPT" stderr + echo "$__OPT" + return 0 + fi + done + done + fi + [ "x$DEBUG" = "x" ] || log "getOptionVariable: returning" stderr + return 0 +# fi + + # they gave a category so we must validate that the option is in that + # category + + # XXX unimplemented + + [ "x$DEBUG" = "x" ] || log "getOptionVariable: returning" stderr + return 0 +} + + +# getOptionValues optionName [category] +# +# returns a list of the valid options for this switch. if a category is +# given, then the option must be in that category. +# +# category names are case-insensitive +# +getOptionValues ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "getOptionValues( \$1=[$1] \$2=[$2] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "getOptionValues() arg1 mismatch" + fi + __VARIABLE="$1" +# __CATEGORY="`echo $2 | tr [:lower:]- [:upper:]_`" + + # if they did not give a category, just return the switch +# if [ "x$__CATEGORY" = "x" ] ; then + __PVAR="echo \$__${__VARIABLE}_VALUES" + __PVAR="`eval $__PVAR`" +# __QUOTE="`echo $__PVAR | tr \' \\" | sed 's/^\"\(.*\)\"/\1/'`" +# echo $__PVAR | tr \' \" | sed 's/^\"\(.*\)\"/\1/' + echo $__PVAR + # strip off the wrapping single quotes + [ ! "x$DEBUG" = "x" ] && log "getOptionValues: returning $__PVAR" stderr + + return 0 +# fi + + # they gave a category so we must validate that the option is in that + # category + + # XXX unimplemented + + return 0 +} + + +# getOptionDescription optionName [category] +# +# returns a list of the options for this switch. if a category is given, +# then the option must be in that category. +# +# category names are case-insensitive +# +getOptionDescription ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "getOptionDescription( \$1=[$1] \$2=[$2] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "getOptionDescription() arg1 mismatch" + fi + __VARIABLE="`echo $1 | tr [:lower:]- [:upper:]_`" +# __CATEGORY="`echo $2 | tr [:lower:]- [:upper:]_`" + + # if they did not give a category, just return the switch +# if [ "x$__CATEGORY" = "x" ] ; then + __PVAL="echo \$__${__VARIABLE}_DESCRIPTION" + eval $__PVAL + return 0 +# fi + + # they gave a category so we must validate that the option is in that + # category + + # XXX unimplemented + + return 0 +} + +# +# variableIsTrue variable +# +# simply returns true if a variable is considered true and false otherwise. +# variables are true when their value is "yes", "true", or "1". +# +variableIsTrue ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "variableIsTrue( \$1=[$1] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "variableIsTrue() arg1 mismatch" + fi + __VARIABLE="$1" + + __VARVAL="echo \$$__VARIABLE" + __VARVAL="`eval $__VARVAL`" + + [ ! "x$DEBUG" = "x" ] && log "variableIsTrue: value is [$__VARVAL]" + + [ "x$__VARVAL" = "x" ] && return 1 + + # if the value is a truthful boolean, return true + case "$__VARVAL" in + y[eE][sS] | Y[eE][sS] | t[rR][uU][eE] | T[rR][uU][eE] | 1 ) + return 0 + ;; + esac + + # otherwise, return false + return 1 +} + + +# +# variableIsFalse variable +# +# simply returns true if a variable is considered false and true otherwise. +# variables are false when they are not true (grin -- see variableIsTrue). +# +variableIsFalse ( ) { + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "variableIsFalse( \$1=[$1] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "variableIsFalse() arg1 mismatch" + fi + __VARIABLE="$1" + + if `variableIsTrue $__VARIABLE` ; then + return 1 + fi + return 0 +} + + +# +# handleArgument (switch|switch=value) [value] [category] +# +# handles an argument (switch value tuple) passed through via the command +# line. the argument includes some unkown number of dashes before the +# switch name, then the switch (case insensitive with optional interchangeable +# dashes and underscores embedded), an optional equals sign, and then the +# value (case-preserving). if a category is given, then the argument must +# match a corresponding option switch in that category. otherwise, not giving +# a category means to match against *any* category. +# +# (e.g. handleArgument test="one two three" "" testCategory) +# (e.g. handleArgument --w no) +# (e.g. handleArgument --w=no -nextArg) +# +# there are several special-case values that represent special classes of +# values. passing __ANYTHING__ as the value, will allow any argument value to +# match. passing __BOOLEAN__ will allow yes/no style values such as true +# false 0 1, etc. passing __NUMBER__ will allow for positive numeric values. +# +# this function returns how many arguments (0, 1, or 2) that were handled that +# may subsequently be shift'd off the argument stack. if a value greater than +# two is returned, it means that the switch was matched, but the value was not. +# +# category names are case-insensitive +# +handleArgument ( ) { + + __TEMP_DEBUG_OFF=1 + [ ! "x$DEBUG" = "x" ] && log "handleArgument( \$1=[$1] \$2=[$2] \$3=[$3] )" stderr + unset __TEMP_DEBUG_OFF + + if [ "x$1" = "x" ] ; then + bomb "handleArgument() arg1 mismatch" + fi + __VAR="$1" + # this "may" not be the value if __VAR is in VAR=VALUE form + __VALUE="$2" + # optional category (case-insensitive) + __GROUP="`echo $3 | tr [:lower:]- [:upper:]_`" + # how many (1 or 2) arguments were processed + __RETURN="2" + + # make sure arg2 is not an option (if it is, value defaults to "true") + if [ "x`echo $__VALUE | sed 's/^\(-\).*/\1/'`" = "x-" ] ; then + __VALUE="1" + __RETURN="1" + elif [ "x$__VALUE" = "x" ] ; then + __VALUE="1" + __RETURN="1" + fi + + # separate out __VAR if it is VAR=VALUE form + if [ ! "x`echo $__VAR | grep =`" = "x" ] ; then + __PVAR="`echo $__VAR | sed 's/^-*\([a-zA-Z_-]*\)=.*/\1/'`" + __PVAL="`echo $__VAR | sed 's/^-*\([a-zA-Z_-]*\)=\(.*\)/\2/'`" + if [ "x$__PVAR" = "x" ] ; then + warn "could not match $__VAR" + else + __VAR="$__PVAR" + __VALUE="$__PVAL" + __RETURN="1" + fi + fi + + # strip off leading dashes and underscores + __VAR="`echo $__VAR | sed 's/^[-_]*\(.*\)/\1/'`" + # convert to lowercase (because switches are stored as *lowercase*) + __VAR="`echo $__VAR | tr [:upper:]_ [:lower:]-`" + + [ ! "x$DEBUG" = "x" ] && log "using $__VAR == $__VALUE as argument" stderr + + # see if there are any options (for this group) registered + __IO="echo \$__INITIALIZED_OPTIONS_${__GROUP}" + __IO="`eval $__IO`" + if [ "x$__IO" = "x" ] ; then + warn "\"$__VAR\" is not a switch in the $__GROUP category, checking other categories" +# warn "No options in the specified category [${__GROUP}]" + __IO="$__INITIALIZED_OPTIONS_" + if [ "x$__IO" = "x" ] ; then + warn "No options match (no options initialized)!" + return 0 + fi + # return 0 + fi + + # there were options found, so iterate over their switches until we find it + for __i in $__IO ; do + for __j in `getOptionSwitches "$__i" "$__GROUP"` ; do + if [ "x$__j" = "x$__VAR" ] ; then + + # we FOUND it ... now validate the value + [ ! "x$DEBUG" = "x" ] && log "handleArgument: found $__VAR as valid switch for $__i option" stderr + + __options="`getOptionValues "$__i" "$__GROUP"`" + + for __val in $__options ; do + # expand the anything value keyword to match any value. + # we set to __match so that we can know which keyword we + # are on (useful for boolean below). + if [ "x$__val" = "x__ANYTHING__" ] ; then + __match="*" + # expand the boolean value keyword to match booleans (we + # cannot do it in outer loop since __BOOLEAN__ may be + # embedded with other options) + elif [ "x$__val" = "x__BOOLEAN__" ] ; then + __match="yes no true false 0 1" + elif [ "x$__val" = "x__NUMBER__" ] ; then + __match="[0123456789]*" + else + __match="$__val" + fi + + # check what was given + case $__VALUE in + $__match ) + __PVAR="${__i}=\"${__VALUE}\"" + eval $__PVAR + export ${__i} + [ ! "x$DEBUG" = "x" ] && log "handleArugment: matched $__match case for $__VALUE, set $__PVAR, returning $__RETURN" stderr + return $__RETURN + ;; + esac + + # more extensive backup checking here for non-exact or + # boolean matches. + for __val2 in $__match ; do + + # check case-insensitive version (we set var to + # original, though -- case-preserving, unless value + # is __BOOLEAN__) + __LCVALUE="`echo $__VALUE | tr [:upper:]_ [:lower:]-`" + if [ "x`echo $__val2 | tr [:upper:]_ [:lower:]-`" = "x$__LCVALUE" ] ; then + if [ "x$__val" = "x__BOOLEAN__" ] ; then + __PVAR="${__i}=\"${__LCVALUE}\"" + else + __PVAR="${__i}=\"${__VALUE}\"" + fi + eval $__PVAR + export ${__i} + + [ ! "x$DEBUG" = "x" ] && log "handleArugment: matched $__val2 insensitive case for $__VALUE, set $__PVAR, returning $__RETURN" stderr + return $__RETURN + fi + + done + + done + + # if we get to here, then we matched, but did not find a + # valid value. + [ ! "x$DEBUG" = "x" ] && warn "handleArgument: matched (${__VAR}) switch, but did NOT find (${__VALUE}) as a valid value" stderr + return 127 + fi + done + done + + [ ! "x$DEBUG" = "x" ] && warn "handleArgument: did NOT find (${__VAR}) as a valid switch" stderr + + return 0 +} + +# this is to avoid a freebsd /bin/sh pwd problem over nfs +# initializeVariable PWD "`pwd`" __library Property changes on: corredor/trunk/Resources/Source/Environment/Environment ___________________________________________________________________ Name: svn:executable + * Name: svn:mime-type + text/x-sh Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Environment/ExecutionParameters.plist =================================================================== --- corredor/trunk/Resources/Source/Environment/ExecutionParameters.plist (rev 0) +++ corredor/trunk/Resources/Source/Environment/ExecutionParameters.plist 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,5 @@ +Description="Environement variable, option, and argument manipulation support " +Provides="Environment Variables Options Arguments" +Requires="Output" +Uses="Debug" +OrderPreference="None" Property changes on: corredor/trunk/Resources/Source/Environment/ExecutionParameters.plist ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Filesystem/ExecutionParameters.plist =================================================================== --- corredor/trunk/Resources/Source/Filesystem/ExecutionParameters.plist (rev 0) +++ corredor/trunk/Resources/Source/Filesystem/ExecutionParameters.plist 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,5 @@ +Description="Filesystem " +Provides="Filesystem Directory" +Requires="Output" +Uses="Debug" +OrderPreference="None" Property changes on: corredor/trunk/Resources/Source/Filesystem/ExecutionParameters.plist ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Filesystem/Filesystem =================================================================== --- corredor/trunk/Resources/Source/Filesystem/Filesystem (rev 0) +++ corredor/trunk/Resources/Source/Filesystem/Filesystem 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,76 @@ +#!/bin/sh +# +# F i l e s y s t e m +# +### +# +# directoryEqualsDirectory +# returns true or false if two directories are one in the same +# directoryContainsDirectory +# returns true or false if one directory contains another +# + + +# +# directoryEqualsDirectory containingDirectory directory +# +# returns a true/false depending on whether two directories are, in fact, +# one in the same. the mechanism used to detect equality are the directory +# inode numbers. +# +directoryEqualsDirectory ( ) { + [ ! "x$DEBUG" = "x" ] && log "directoryEqualsDirectory( \$1=[$1] \$2=[$2] )" + + if [ "x$1" = "x" ] ; then + bomb "directoryEqualsDirectory() arg1 mismatch" + else + __CDIR="$1" + fi + + if [ "x$2" = "x" ] ; then + bomb "directoryEqualsDirectory() arg2 mismatch" + else + __DIR="$2" + fi + + # get a unique identifier for the automation directory + CDIR_INODE="`ls -d -i $CDIR | awk '{print $1}'`" + DIR_INODE="`ls -d -i $DIR | awk '{print $1}'`" + + if [ "x$CDIR_INODE" = "x$DIR_INODE" ] ; then + return 1; + fi + return 0 +} + + +# +# directoryContainsDirectory containingDirectory directory +# +# returns a true/false depending on whether the second directory is +# contained within the first containing directory. The mechanism used +# to detect containment are the directory inode numbers. +# +directoryContainsDirectory ( ) { + [ ! "x$DEBUG" = "x" ] && log "directoryContainsDirectory( \$1=[$1] \$2=[$2] )" + + if [ "x$1" = "x" ] ; then + bomb "directoryContainsDirectory() arg1 mismatch" + else + __CDIR="$1" + fi + + if [ "x$2" = "x" ] ; then + bomb "directoryContainsDirectory() arg2 mismatch" + else + __DIR="$2" + fi + + # get a unique identifier for the automation directory + CDIR_INODE="`ls -d -i $CDIR | awk '{print $1}'`" + DIR_INODE="`ls -d -i $DIR | awk '{print $1}'`" + + + + return 0 +} Property changes on: corredor/trunk/Resources/Source/Filesystem/Filesystem ___________________________________________________________________ Name: svn:executable + * Name: svn:mime-type + text/x-sh Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Input/ExecutionParameters.plist =================================================================== --- corredor/trunk/Resources/Source/Input/ExecutionParameters.plist (rev 0) +++ corredor/trunk/Resources/Source/Input/ExecutionParameters.plist 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,5 @@ +Description="Receive and parse input from the user"; +Provides="Input"; +Requires="Output"; +Uses="Debug"; +OrderPreference="None"; Property changes on: corredor/trunk/Resources/Source/Input/ExecutionParameters.plist ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Input/Input =================================================================== --- corredor/trunk/Resources/Source/Input/Input (rev 0) +++ corredor/trunk/Resources/Source/Input/Input 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,54 @@ +#!/bin/sh +# +# I n p u t +# +### +# +# ask +# ask a yes/no question until a valid response is given +# + + +# +# ask [question] +# +# ask simply asks a yes/no question until a valid answer is given. a boolean +# value is returned (true==yes) +# +ask ( ) { + [ ! "x$DEBUG" = "x" ] && log "ask( \$1=[$1] )" + + [ "x$1" = "x" ] || __QUESTION="$1" + + [ "x$__QUESTION" = "x" ] || log "$__QUESTION" + read __answer + [ ! "x$DEBUG" = "x" ] && log "ask: user responded with [$__answer] )" + case $__answer in + y[eE][sS] | Y[eE][sS] ) + __answer="yes" + ;; + n[oO] | N[oO] ) + __answer="no" + ;; + esac + while [ "x$__answer" != "xyes" ] && [ "x$__answer" != "xno" ] ; do + log "please answer yes or no" + [ "x$__QUESTION" = "x" ] || log "$__QUESTION" + read __answer + [ ! "x$DEBUG" = "x" ] && log "ask: user responded with [$__answer] )" + case $__answer in + Yes | yEs | yeS | YEs | YeS | yES | YES ) + __answer="yes" + ;; + No | nO | NO ) + __answer="no" + ;; + esac + done + + [ ! "x$DEBUG" = "x" ] && log "ask: returning yes/no 0/1 value (yes==0) of [$__answer] )" + + [ "x$__answer" = "xyes" ] && return 0 + return 1 +} + Property changes on: corredor/trunk/Resources/Source/Input/Input ___________________________________________________________________ Name: svn:executable + * Name: svn:mime-type + text/x-sh Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Locking/ExecutionParameters.plist =================================================================== --- corredor/trunk/Resources/Source/Locking/ExecutionParameters.plist (rev 0) +++ corredor/trunk/Resources/Source/Locking/ExecutionParameters.plist 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,5 @@ +Description="File-based lock and semaphore support"; +Provides="Locking Locks Semaphores"; +Requires="Output"; +Uses="Debug"; +OrderPreference="None"; Property changes on: corredor/trunk/Resources/Source/Locking/ExecutionParameters.plist ___________________________________________________________________ Name: svn:mime-type + text/plain Name: svn:eol-style + native Added: corredor/trunk/Resources/Source/Locking/Locking =================================================================== --- corredor/trunk/Resources/Source/Locking/Locking (rev 0) +++ corredor/trunk/Resources/Source/Locking/Locking 2006-09-19 06:01:40 UTC (rev 2) @@ -0,0 +1,320 @@ +#!/bin/sh +# +# L o c k i n g +# +### +# +# acquireLock +# creates a cooperative lock file ("blocks" when one already exists) +# releaseLock +# releases and/or removes a cooperative lock file +# releaseAllLocks +# releases all locally acquired locks +# +# acquireSemaphore +# gets a cooperative file semaphore ("blocks" when one does not exist) +# releaseSemaphore +# releases and/or creates a cooperative file semaphore +# releaseAllSemaphores +# releases all locally acquired semaphores +# +# INTERNAL VARIABLES (do not use or rely on) +# +# __ACQUIRED_LOCKS +# __ACQUIRED_SEMAPHORES +# + + +# +# acquireLock [lockName] [retryCount] [waitLength] [location] +# +# will create a file that will block subsequent calls to acquireLock until the +# file lock is released. a ".lock" extension is appended automatically to +# distinguish between file locks and file semaphores. +# locks acquired are listed in the ACQUIRED_LOCKS global var +# +# if file exists then wait/fail +# else create lock file and log +# +acquireLock ( ) { + [ ! "x$DEBUG" = x ] && log "acquireLock( \$1=[$1] \$2=[$2] \$3=[$3] \$4=[$4] )" + + if [ "x$1" = x ] ; then __NAME="local" ; else __NAME="${1}" ; fi + if [ "x$2" = x ] ; then __RETRY=1 ; else __RETRY=$2 ; fi + if [ "x$3" = x ] ; then __WAIT=0 ; else __WAIT=$3 ; fi + if [ "x$4" = x ] ; then + # if we contain a slash, prefix is nothing (rooted or relative path + # name), otherwise use the current dir + __SWS=`echo $__NAME | sed 's/.*\/.*/___ROOTED_NAME___/'` + if [ "x$__SWS" = "x___ROOTED_NAME___" ] ; then __PREFIX="" ; else __PREFIX="./" ; fi + else + # if we end with a slash, no worries, otherwise, append one + __SWS=`echo $__NAME | sed 's/.*\/$/___END_SLASH___/'` + if [ "x$__SWS" = "x___END_SLASH___" ] ; then __PREFIX="$4" ; else __PREFIX="${4}/" ; fi + fi + __LOCK="${__NAME}.lock" + + [ ! "x$DEBUG" = "x" ] && log "acquireLock: before __ACQUIRED_LOCKS=[${__ACQUIRED_LOCKS}]" + + while [ -f ${__PREFIX}$__LOCK ] ; do + __RETRY=`expr $__RETRY - 1` + if [ $__RETRY -lt 1 ] ; then + warn "Unable to obtain $__LOCK in $__PREFIX" + + if [ ! "x$USER" = "x" ] ; then __USER="$USER" ; unset USER ; fi + if [ ! "x$PID" = "x" ] ; then __PID="$USER" ; unset PID ; fi + if [ ! "x$DATE" = "x" ] ; then __DATE="$USER" ; unset DATE ; fi + loadResourceFile "${__PREFIX}$__LOCK" + if [ ! "x$USER" = "x" ] ; then + log "Timed out waiting for ${USER}'s lock held since ${DATE} (pid: $PID)" + fi + if [ ! "x$__USER" = "x" ] ; then USER="$__USER" ; unset __USER ; fi + if [ ! "x$__PID" = "x" ] ; then PID="$__USER" ; unset __PID ; fi + if [ ! "x$__DATE" = "x" ] ; then DATE="$__USER" ; unset __DATE ; fi + + return 1 + fi + log "Waiting for $__LOCK ($__WAIT seconds)... $__RETRY attempts remain" + sleep $__WAIT + done + touch ${__PREFIX}$__LOCK + if [ ! -w ${__PREFIX}$__LOCK ] ; then + warn "Unable to write to lock file $__LOCK in ${__PREFIX}?" + fi + # save details about the acquiring process to the lock file + if [ ! "x$USER" = "x" ] ; then log "USER=\"$USER\"" "${__PREFIX}$__LOCK" ; else log "USER=\"unknown\"" "${__PREFIX}$__LOCK" ; fi + log "PID=\"$$\"" "${__PREFIX}$__LOCK" + log "DATE=\"`date`\"" "${__PREFIX}$__LOCK" + # keep a list of the acquired locks up to date + __ACQUIRED_LOCKS="$__ACQUIRED_LOCKS ${__PREFIX}$__NAME" ; export __ACQUIRED_LOCKS + [ ! "x$DEBUG" = "x" ] && log "acquireLock: after __ACQUIRED_LOCKS=[${__ACQUIRED_LOCKS}]" + + log "Acquired $__LOCK" + + return 0 +} + + +# +# releaseLock [lockName] [location] +# +# if file exists then remove and try unlog +# else warn and try unlog +# +releaseLock ( ) { + [ ! "x$DEBUG" = x ] && log "releaseLock( \$1=[$1] \$2=[$2] )" + + if [ "x$1" = x ] ; then __NAME="local" ; else __NAME="${1}" ; fi + if [ "x$2" = x ] ; then + # if we contain a slash, prefix is nothing (rooted or relative-path + # name), otherwise use a current dir + __SWS=`echo $__NAME | sed 's/.*\/.*/___ROOTED_NAME___/'` + if [ ! "x$DEBUG" = x ] ; then log "releaseLock: __SWS=[$__SWS]" ; fi + if [ "x$__SWS" = "x___ROOTED_NAME___" ] ; then __PREFIX="" ; else __PREFIX="./" ; fi + else + # if we end with a slash, no worries, otherwise, append one + __SWS=`echo $__NAME | sed 's/.*\/$/___END_SLASH___/'` + if [ "x$__SWS" = "x___END_SLASH___" ] ; then __PREFIX="$2" ; else __PREFIX="${2}/" ; fi + fi + __LOCK="${__NAME}.lock" + + if [ ! "x$DEBUG" = x ] ; then log "releaseLock: __PREFIX=[$__PREFIX] __LOCK=[$__LOCK]" ; fi + + [ ! "x$DEBUG" = "x" ] && log "releaseLock: before __ACQUIRED_LOCKS=[${__ACQUIRED_LOCKS}]" + + if [ ! -f "${__PREFIX}$__LOCK" ] ; then + warn "Lock [$__LOCK] is already released" + # make sure the lock is not registered + if [ ! "x$__ACQUIRED_LOCKS" = "x" ] ; then + __NEW__AL="" + for __CL in $__ACQUIRED_LOCKS ; do + if [ ! "x$__CL" = "x${__PREFIX}$__NAME" ] ; then + __NEW__AL="$__NEW__AL $__CL" + fi + done + __ACQUIRED_LOCKS="$__NEW__AL" ; export __ACQUIRED_LOCKS + fi + return 1 + fi + rm -f ${__PREFIX}$__LOCK + # unregister the lock with the __ACQUIRED_LOCKS var + if [ ! "x$__ACQUIRED_LOCKS" = "x" ] ; then + __NEW__AL="" + __FOUND_LOCK=0 + for __CL in $__ACQUIRED_LOCKS ; do + if [ "x$__CL" = "x${__PREFIX}$__NAME" ] ; then + __FOUND_LOCK=1 + else + __NEW__AL="$__NEW__AL $__CL" + fi + done + if [ ! "x$__FOUND_LOCK" = "x1" ] ; then + warn "bad bookkeeping -- __ACQUIRED_LOCKS is missing entry $__NAME [LOCK=${__PREFIX}${__NAME}] [__A_L=$__ACQUIRED_LOCKS]" + fi + __ACQUIRED_LOCKS="$__NEW__AL" ; export __ACQUIRED_LOCKS + else + warn "bad bookkeeping -- __ACQUIRED_LOCKS is empty prior to a release of $__NAME" + fi + [ ! "x$DEBUG" = "x" ] && log "releaseLock: after __ACQUIRED_LOCKS=[${__ACQUIRED_LOCKS}]" + + log "Released $__LOCK" + return 0 +} + +# +# releaseAllLocks +# +releaseAllLocks ( ) { + [ ! "x$DEBUG" = x ] && log "releaseAllLocks() __ACQUIRED_LOCKS=[$__ACQUIRED_LOCKS]" + + if [ ! "x$__ACQUIRED_LOCKS" = x ] ; then + for __LOCK in $__ACQUIRED_LOCKS ; do + releaseLock $__LOCK + done + fi +} + +# +# acquireSemaphore [semaphoreName] [retryCount] [waitLength] [location] +# +# if file exists then remove file and log +# else wait/fail +# +acquireSemaphore ( ) { + [ ! "x$DEBUG" = x ] && log "acquireSemaphore( \$1=[$1] \$2=[$2] \$3=[$3] \$4=[$4] )" + + if [ "x$1" = x ] ; then __NAME="local" ; else __NAME="${1}" ; fi + if [ "x$2" = x ] ; then __RETRY=1 ; else __RETRY=$2 ; fi + if [ "x$3" = x ] ; then __WAIT=0 ; else __WAIT=$3 ; fi + if [ "x$4" = x ] ; then + # if we start with a slash, prefix is nothing (rooted name), + # otherwise use a relative path + __SWS=`echo $__NAME | sed 's/^\/.*/___ROOTED_NAME___/'` + if [ "x$__SWS" = "x___ROOTED_NAME___" ] ; then __PREFIX="" ; else __PREFIX="./" ; fi + else + # if we end with a slash, no worries, otherwise, append one + __SWS=`echo $__NAME | sed 's/.*\/$/___END_SLASH___/'` + if [ "x$__SWS" = "x___END_SLASH___" ] ; then __PREFIX="$4" ; else __PREFIX="${4}/" ; fi + fi + __SEMAPHORE="${__NAME}.semaphore" + + while [ ! -f ${__PREFIX}$__SEMAPHORE ] ; do + __RETRY=`expr $__RETRY - 1` + if [ $__RETRY -lt 1 ] ; then + warn "Unable to obtain $__SEMAPHORE in $__PREFIX" + return 1 + fi + log "Waiting for $__SEMAPHORE ($__WAIT seconds)... $__RETRY attempts remain" + sleep $__WAIT + done + # we move instead of delete so that we can tell who has acquired the semaphore + mv -f "${__PREFIX}${__SEMAPHORE}" "${__PREFIX}${__SEMAPHORE}.acquired" + if [ ! -w "${__PREFIX}${__SEMAPHORE}.acquired" ] ; then + warn "Unable to write to acquired semaphore file ${__SEMAPHORE}.acquired in ${__PREFIX}" + fi + # sanity check - make sure file is gone + if [ -f "${__PREFIX}${__SEMAPHORE}" ] ; then + warn "Unable to properly remove/acquire semaphore file ${__SEMAPHORE} in ${__PREFIX}" + return 1 + fi + + if [ ! "x$USER" = "x" ] ; then log "USER=$USER" "${__PREFIX}${__SEMAPHORE}.acquired" ; else log "USER=unknown" "${__PREFIX}${__SEMAPHORE}.acquired" ; fi + log "PID=$$" "${__PREFIX}${__SEMAPHORE}.acquired" + log "DATE=`date`" "${__PREFIX}${__SEMAPHORE}.acquired" + + __ACQUIRED_SEMAPHORES="$__ACQUIRED_SEMAPHORES ${__PREFIX}$__NAME" ; exp... [truncated message content] |