SkePi

Data parallel and stream parallel skeletons implemented in erlang.

compilation, debug and cleanup

Once you have moved to ske-pi's root directory, you can compile all sources to object code.

make all or make may be used to compile with debug mode turned on.
In this mode, some information regarding inter-process communication (and timeouts, if any have occurred), as well as function calls for each module will be saved to a log file (located in "ske-pi/logs/info.log" by default).
make all DEBUG= or make DEBUG= will compile with debug mode turned off.

make clean may be used to remove the directory containing all the object files and keep the log files.
make cleanall will remove both object files and the log files previously generated.

usage examples

Each module has its own usage function printing a brief description and giving a simple usage example.

Once you have compiled ske-pi, you may:
* start the Erlang shell with cd ebin; erl from ske-pi's root directory
* call the usage function for a given module. Via the Erlang shell, use the following command:

> <module_name>:usage().

• additionally, you may test the stream parallel skeletons' performance using an example with a default configuration. Like so:

> test_stream:benchmark().

• similarly, you may run the data parallel skeletons example. Write:

> test_mapred:benchmark().

• the above is also available for the skeletons implementing the google mapreduce framework. Try it with:

> test_mapred_google:benchmark().

• for the stream parallel skeletons example, it is possible to configure the length of the list, the length of the chunks onto which the list is split, the number of worker processes and the number of schedulers used (N.B. dependent on the machine used to run the tests)
• you may also configure the length of the list, the length of chunks and the number of schedulers used for the data parallel skeletons example
• for the google mapreduce example, you may input the name of a directory containing your own test files, an atom used to perform an unix grep-like operation on the test files and the number of scherdulers used (please note that the directory with your own test files should be located in ske-pi's top directory and its name should be in string format)
• the erlang shell may be exited with the following command:

> q().

acknowledgements

The stream skeleton ("stream.erl") is taken and then modified from the skel library by the ParaPhrase group, whilst the data parallel skeletons ("mapred_google.erl" and "pmap.erl") are modified versions of Joe Armstrong's examples in his Programming Erlang 2nd ed. book.