pipper Git
Status: Alpha
Brought to you by:
kellrott
=============================================================================== PIPPER Python Integrated Parallel Pipeline Environment =============================================================================== http://pipper.sourceforge.net/ PIPPER is a Python pre-parser which makes it easy to generate parallel code that can be run in a cluster environment. It looks for code annotation much like OpenMPI. The main difference is that unlike OpenMPI, which is can only work on a single computer, PIPPER designed to work with a cluster of machines. =============================================================================== The helloworld.py script: =============================================================================== #!/usr/bin/python import sys import os def do_call(x,y): print x, y, os.getpid() if __name__ == '__pipper_main__': a_range = range( int(sys.argv[1]) ) """@pragma pipper_start""" for a in a_range : for b in a_range : do_call(a,b) """@pragma pipper_end""" =============================================================================== To execute the program in an MPI environment: =============================================================================== On 25 nodes (using the manga engine) mpirun -np 25 pipper -manga ./helloworld 100 =============================================================================== About PIPPER =============================================================================== PIPPER is written as a Python pre-parser, written in Python. In addition to a command line program that automates the parsing and execution of PIPPER scripts, it can be used explicitly to import and extend modules from inside of a Python script. The Python parser introduces callback functions to link in parallel code. These callbacks then link to an abstract engine interface which performs calls to parallelization engines that have been adapted to PIPPER. Currently the main engine is called Manga, and is implemented with MPI. PIPPER will work in a single CPU environment, but that mode is primarily for debugging. Build Notes: To Build: python setup.py build python setup.py install For local install (for example $HOME/my_pipper_install) python setup.py install --home=$HOME/my_pipper_install =============================================================================== The Future: =============================================================================== Future abstract engines wrappers will hopefully include: - MapReduce ( http://hadoop.apache.org/mapreduce/ ) : For Jython - Parallel Python ( http://www.parallelpython.com/ ) - papy ( http://code.google.com/p/papy/ )