Apache Airflow
Programmatically author, schedule, and monitor workflows
Apache Airflow is a community-created platform for programmatically authoring, scheduling, and monitoring workflows. It’s much easier to do all these things when workloads are defined as code. They become more versionable, testable, maintainable and collaborative.
With Airflow you can author workflows as directed acyclic graphs (DAGs) of tasks. This means that your tasks are executed on an array of workers while following the specified dependencies. And with rich command line utilities...