glue

Glue is a job execution engine, written in Java and Groovy. workflows are written in Groovy DSL (simple statements) and use pre-developed modules to interact with external resources e.g. DBs, Hadoop, Netezza, FTP etc.

In Glue one of the main features and design goals is to always abstract configuration away from functionality. This means that no more hardcoded IPs, UserIDs and Passwords spread over 10s of hundreds of bash/python scripts.

Scripts written for one environment can be easily ported to another because the configuration is done outside of each workflow.

How is this done?

Each Module has a configuration section in the /opt/glue/conf/workflow_modules.grooy where data like hosts, ips, usernames etc are placed. This configuration is loaded and provided to the Module before starting each workflow.

Configurations can be changed dynamically and are re-read before each workflow run, such that no restart is required.