GenWrapper - DC-API/ BOINC Generic Wrapper for legacy applications
MTA SZTAKI LPDS
BUDAPEST, 2008-2009
Introduction
GenWrapper is a generic wrapper interpreter (scriptable as the POSIX shell). It is very much based on GitBox that was extracted from git and modified to run standalone. GitBox is a stripped down version of BusyBox modified to compile under Windows with MinGW. This version should also compile on Linux and Mac OS X (and probably more but those are not tested) besides Windows. Main features are:
- Runs legacy applications without BOINCification.
- Makes BOINC API/ DC-API available in POSIX shell scripting.
- Available for Mac OS X, MS Windows, GNU/ Linux (it is based on GitBox).
- Small size (~600k).
Links
Precompiled binaries can be found at the files section on SourceForge. The homepage of GenWrapper can be found here.
Usage
Applications and work units
GenWrapper can be compiled as a native BOINC application or as a DC-API application (see here for more information on DC-API ).
GenWrapper consists of two parts: a launcher (gw_launcher{.exe}) and GitBox (gitbox{.exe}). The launcher acts as a DC-API/BOINC application, it’s task is to prepare the application and communicate with BOINC, while GitBox provides a generic DC-API/BOINC enabled scripting interface.
Any application using GenWrapper must have the following components:
- The launcher, renamed to the name of the registered application in BOINC (e.g.: numsys-1.00_i686-apple-darwin)
- Either:
- A zip file with the same name and .zip extension holding the legacy application and GitBox binaries. GitBox on Windows must be named as gitbox.exe on Linux/ Mac OS X as gitbox)
- Or the application binaries and GitBox executable unzipped. GitBox on Windows must be named as gitbox.exe on Linux/ Mac OS X as gitbox.
- An optional script insalled as part of the application named profile.sh (profile script). This script will be executed each time before the application is started. This is optional. The optional profile script should follow these rules:
- Should be a standard POSIX shell script (no bash or ksh extensions). It can be tested locally via GitBox (execute gitbox sh profile.sh).
- It is run after uncompressing the application archive, but before starting the work unit.
- Should handle all platform/architecture specific tasks required for the application to run (e.g.: set library path).
- Exported environment variables are later available for the work unit supplied script.
Work units for any GenWrapper application must follow these rules:
- They should supply a shell script (work unit supplied script), which:
- Is a standard POSIX shell script (no bash or ksh extensions). It can be tested locally via GitBox (execute gitbox sh scriptname).
- Should contain platform/ architecture independent code only.
- Should execute the legacy application(s).
- Has access to environment variables set in the profile script.
- The first command line argument of the work unit should be the name of the work unit supplied shell script.
Available shell commands
Available BOINC/ DC-API commands:
boinc resolve_filename <FILENAME> boinc fraction_done <0.00 .. 1.00> boinc fraction_done_percent <0 .. 100>
The exit <STATUS> command equals to calling the boinc_finish(<STATUS>) BOINC C API function Notes:
- gitbox, gw_launcher are BOINC compound applications.
- DC-API enabled version also uses these shell commands but instead of the BOINC API calls the appropriate DC-API functions.
- Legacy applications executed by the shell cannot contain any BOINC API / DC-API functions (especially init and finish functions).
Available non-BOINC specific shell commands:
ash, awk, basename, bunzip2, bzcat, cat, chmod, cmp, cp, cut, date, diff, dirname, dos2unix, echo, egrep, env, exit, expr, false, fgrep, find, grep, gunzip, gzip, head, ls, lzmacat, md5sum, mkdir, mv, od, patch, printenv, printf, pwd, realpath, rm, rmdir, sed, seq, sh, sleep, sort, sum, tac, tail, tar, test, touch, tr, true, uniq, unix2dos, unlzma, unzip, wc, which, yes, zcat, [, [[
Example shell script
The following shows a simple example for a work unit supplied script:
IN= ‘boinc resolve_filename in‘
OUT= ‘boinc resolve_filename out‘
NUM= ‘cat "${IN}"‘
PERCENT_PER_ITER=$((10000 / NUM))
for i in ‘seq ${NUM}‘ ; do
PERCENT_COMPLETE=$((PERCENT_INTER *i / 1000) )
boinc fraction_done_percent ${PERCENT_COMPLETE}
echo -e "I am ${PERCENT_COMPLETE}% complete." >> "${OUT}"
sleep 1 ;
done
It reads an integer from a logical file named in and runs a loop that many times. In each iteration it sleeps for a second and writes the completion status to the logical file named out while reporting it to the BOINC client. Notes:
- Every filename needs to be resolved via boinc resolve_filename.
- Variable references should be properly quoted (e.g: NUM=‘cat "${IN}"‘).
- Only the launcher is DC-API/ BOINC enabled, GitBox and shell scripts act as legacy applications.
- exit <STATUS> shell command equals to calling the boinc_finish(<STATUS>) BOINC C API function.
- Applications executed by the script cannot contain any BOINC/ DC-API functions, must be legacy applications.
Flow
A GenWrapper wrapped application works as follows:
- A BOINC client downloads an application and a work unit.
- The main executable of the application is the launcher for GenWrapper. This launcher is executed by the client.
- The launcher is a DC-API/ BOINC enabled application, thus it contacts the client after start.
- The launcher checks if a zip file named <application_name>.zip exists, if yes it unzips it.
- The launcher generates a shell script which:
- Checks if a file named profile.sh exists, if yes it sources it.
- Executes GitBox with the command line arguments of the work unit, where the first argument must be the name of the work unit supplied shell script. The launcher will resolve the name of this script, it is enough to provide the logical file name.
- The generated shell script is executed.
- When the script finishes the exit status of the work unit supplied script is set as the exit status of the work unit.
- The launcher exits and the results are uploaded to the server.
Batch mode
More legacy executables may be bundled together and run, for example one executable could use the output of another executable in a single work unit, even if there are special preparations to be done with the output before using it as input.
Compiling
GenWrapper can be compiled as standalone, BOINC enabled or DC-API enabled. It’s been tested and working on Debian GNU/Linux, MS Windows XP and Mac OS X 10.5. For compiling on Windows use the MINGW toolkit, MS Visual Studio is not supported. It uses a single Makefile, the compile options must be set there.
Configurable options in the Makefile:
# setting DCAPI to "yes" will set BOINC to "yes" also BOINC=yes DCAPI=yes ifeq ($(findstring mingw, $(TARGET)), mingw) DCAPI_HOME=C:/Projects/dcapi/trunk BOINC_CFLAGS=-IC:/Projects/boinc_mingw/include -IC:/Projects/openssl/include BOINC_LIBS=-LC:/Projects/boinc_mingw/ -lboinc -LC:/Projects/openssl/lib - lcrypto -Wl,-Bstatic -lstdc++ -Wl,-Bdynamic -lm #LAUNCHER_CFLAGS= #LAUNCHER_LDFLAGS= #only when compiling with mingw and BOINC is set to "yes" #OPENSSL_DIR=C:/Projects/openssl/lib/ else DCAPI_CFLAGS= ‘pkg-config --cflags dcapi-boinc-client‘ DCAPI_LIBS= ‘pkg-config --libs dcapi-boinc-client‘ BOINC_CFLAGS=-I/usr/include/BOINC BOINC_LIBS=-lboinc_api -lboinc -lcrypto -Wl,-Bstatic -lstdc++ -Wl,- Bdynamic -lpthread -lm endif
The BOINC=yes|no and DCAPI=yes|no options (above) decide whether DC-API or BOINC support should be compiled. On Linux/ Mac OS X no other adjustment is needed. On Windows the DCAPI_HOME, BOINC_CFLAGS and BOINC_LIBS variables should be set to the directories where the BOINC/ DC-API libraries and includes reside. Linking libraries compiled with MinGW and MS Visual C++ can be troublesome, we recommend to compile the BOINC and DC-API libs via MinGW too.
Cross-compiling
For cross-compiling you should set the TARGET variable. (Additionally setting CC, AR and STRIP or other variables may also be necessary if defaults are not sufficient).
On Unix/Linux (also includes Mac OS X) to Windows
To set up cross-compiling environment:
- Get the ’Cross-Hosted MinGW Build Tool’ script from the MinGW download page (Tested with x86-mingw32-build.sh-0.0-20061107-1 ).
- Set the package versions in x86-mingw32-build.sh.conf, tested with:
GCC_VERSION 3.4.5-20060117-1 BINUTILS_VERSION 2.18.50-20080109-2 RUNTIME_VERSION 3.14 W32API_VERSION 3.11
- Run the script to get and build cross-compiling environment
- To compile simply do:
make TARGET=/path/to/mingw/install/dir/bin/i386-mingw32
On MinGW to Windows
To compile with the Cygwin compiler install the mingw versions from setup.exe and then you can issue the following command:
make TARGET=mingw CC="gcc -mno-cygwin" AR=ar STRIP=strip
Contact and feedback
Please provide feedback to desktopgrid@… .
