glue

BigData Workflow Engine for Hadoop, Hbase, Netezza, Pig, Hive ...

Add a Review
2 Downloads (This Week)
Last Update:

Description

Glue is a job execution engine, written in Java and Groovy. workflows are written in Groovy DSL (simple statements) and use pre-developed modules to interact with external resources e.g. DBs, Hadoop, Netezza, FTP etc.

In Glue one of the main features and design goals is to always abstract configuration away from functionality. This means that no more hardcoded IPs, UserIDs and Passwords spread over 10s of hundreds of bash/python scripts.

Scripts written for one environment can be easily ported to another because the configuration is done outside of each workflow.

How is this done?

Each Module has a configuration section in the /opt/glue/conf/workflow_modules.grooy where data like hosts, ips, usernames etc are placed. This configuration is loaded and provided to the Module before starting each workflow.

Configurations can be changed dynamically and are re-read before each workflow run, such that no restart is required.

glue Web Site

Categories

License

Apache License V2.0

Update Notifications





Write a Review

User Reviews

Be the first to post a review of glue!

Additional Project Details

Registered

2013-07-03
Screenshots can attract more users to your project.
Features can attract more users to your project.

Icons must be PNG, GIF, or JPEG and less than 1 MiB in size. They will be displayed as 48x48 images.