AcoustiCloud: Research Code Files

Research code for the paper AcoustiCloud: A Cloud-Based System for Managing Large-Scale Bioacoustics Processing currently under review for the Journal of Environmental Monitoring and Software.

Note: This is research code and is not designed for use by a general audience.

To Run Main Master/Slave Test
Rebuild the MongoDB database using from the DB dump (Dump is in the 'dump' folder) (mongorestore dump)
Import relevant Python3 libraries using pip3
Modify ClientTest.py and MongoDBTest... files so that the master machines IP is given where "MASTER MACHINE" is used as a placeholder
Run ClientTest.py on slave machine/s
Run WFEval.py file on a master machine
Masters and Slaves both need the Functions.py file, only Slaves need ClientTest.py, only Masters need MongoDBTest... and WFEval.py files

Due to some audio samples in the audio test file containing licenses with no derivatives clauses, the source audio used for testing cannot be uploaded. To generate your own test audio, use the Scaper tool (https://github.com/justinsalamon/scaper) using sound samples and pink noise (generated in Audacity for tests). Annotate audio files based on the attached AnnotationsTestB.csv file (set frequency ranges to 0-11000 if you don't want to specify these, and the rest are given by scaper itself).

For Executing Pegasus Workflow:
Install Pegasus/HTCondor
Start Pegasus workflow using pegasus-init split
Replace rc.txt, tc.txt, and daxgen.py with the included files (change file paths in rc.txt and tc.txt as appropriate)
Run generate_dax.sh and then plan_dax.sh

For adding extra tasks:
Write a python function in a seperate file containing the task to add. Follow the formatting of functions in the Functions.py file.
Run FunctionInput.py and follow the prompts.
You will need to generate new allTasks.py and TypeData.py files. There is a function to do this in the MongoDBTest... file.

For adding new parameter Sets:
Run the ParamsInput.py file and follow the prompts.